Skip to content

Commit 947d3a1

Browse files
committed
feat(infra): a kustomize patch OVERRIDEs the objects it changes
A kustomize overlay changes its base through patches; the graph read none of the v5 `patches:` entries (path / patch / target mappings), so a patch file showed up as a second copy of the object it patches, with no edge to it. Each patch of a kustomization now OVERRIDEs, child -> base, the objects of its build it changes, with {via: patch, kustomization, patch, line}: - a patch file's documents: the object of the same group, kind and name (and namespace, when the document declares one); - an inline patch (from the kustomization's File) or a JSON6902 operation file (from its own File): what its `target` selects -- kind, group, version, name, namespace; a label or annotation selector is not evaluated and names nothing; - in the build of the kustomization stating it; a component's in the build of each kustomization including it. A patch file is an IMPORTS (local_name: patches) of its kustomization and never a member of its build: its documents select nothing, are selected by nothing. Resources now keep the version of their apiVersion (`version`), which the identity leaves out and a target may name. Each build indexes its own objects by name once: a lookup in the repository-wide name index would visit every copy of a name for every patch. The complexity fixture gains patch files (one of four documents), an inline and a kind-only patch. Measured on 17 infrastructure repositories: 126 edges, every one in an independent reading of the 173 patches (122 resolvable in the repositories); the 4 pairs without an edge use a label selector. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent 3b93db0 commit 947d3a1

9 files changed

Lines changed: 858 additions & 29 deletions

File tree

‎README.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -255,6 +255,7 @@ The install script placed beside the binary is **reported, not deleted** — uni
255255
- `DEPLOYS` — what puts manifests, kustomizations or charts on a cluster, to what it puts there: a Flux Kustomization or Argo Application to the kustomizations / manifests / chart of its path, a Flux HelmRelease to its chart (`spec.chart` or `chartRef` → OCIRepository), a shell command or Make recipe (`kubectl apply -f/-k`, `kustomize build | kubectl apply`, `helm install/upgrade/template`), a Go installer loading a chart with the Helm SDK, a Terraform `helm_release`. A chart outside the repository is a shared external `Chart`; the edge says `via` (`flux`, `argo`, `helmrelease`, `kubectl`, `kustomize`, `helm`, `helm-sdk`, `terraform`, ...)
256256
- `DEPENDS_ON` — a Helm chart to the charts it declares (the local chart for a resolvable `file://` repository, otherwise an external `Chart`; the edge keeps `repository`), and a `go.mod` / `requirements.txt` file to an external `Package`; a Dockerfile to the `Image`s it builds on (`via: from`, `copy-from`); an `Image` to the Dockerfile it is built from (`via: build`; `tool`: `skaffold`, `compose`, `docker-build` for `docker` / `podman` / `nerdctl` / `buildah` build commands in scripts, Make and CI); a compose service to the services its `depends_on` / `links` name
257257
- `CONFIGURES` — code reading an environment variable to the shared `EnvVar` of that name (`strategy: env_access`); a Kubernetes container, compose service or Dockerfile (`ENV` of the image) setting one to that container's own `EnvVar` (`strategy: env_set`, with `source`: `value`, `configMapKeyRef`, `secretKeyRef`, `fieldRef`, ..., `compose`, `dockerfile` — values are never stored); and that container's `EnvVar` to the code reading the name inside the files its image is built from (`strategy: env_read`, with `dockerfile`) — the image's files are the sources its Dockerfile's `COPY` / `ADD` put into it, less the context's `.dockerignore`
258+
- `OVERRIDE` with `via: patch` — a Kustomize patch to each object of its kustomization's build it changes (child → base): a patch file's documents to the object of the same kind and name; an inline patch (from the kustomization) or a JSON6902 operation file to what its `target` selects (group, version, kind, name, namespace); a component's in the build of each kustomization including it. The edge keeps `kustomization`, `patch` and `line`; a patch file is an `IMPORTS` (`local_name: patches`) of its kustomization, never a member of the build
258259
- `EMITS`, `LISTENS_ON` (channels)
259260
- `DATA_FLOWS` with arg-to-param mapping + field access chains
260261
- `SIMILAR_TO` (MinHash + LSH near-clone detection, Jaccard scored)
@@ -263,7 +264,7 @@ The install script placed beside the binary is **reported, not deleted** — uni
263264
### Indexing pipeline
264265
- **158 vendored tree-sitter grammars** compiled into the binary
265266
- **Generic package / module resolution** — bare specifiers like `@myorg/pkg`, `github.com/foo/bar`, `use my_crate::foo` resolved via manifest scanning (`package.json`, `go.mod`, `Cargo.toml`, `pyproject.toml`, `composer.json`, `pubspec.yaml`, `pom.xml`, `build.gradle`, `mix.exs`, `*.gemspec`)
266-
- **Infrastructure-as-code indexing** — Dockerfiles (`Dockerfile.*`, `*.Dockerfile`, `Containerfile`: bases, what the container runs, `ENV`), container images (`Image`: the normalised repository, tag and digest on the edges; a repository built from several Dockerfiles, one per component tag, gets an `Image` per component), docker compose services (`Resource`), Kubernetes manifests (`Resource`), Kustomize overlays (`IMPORTS` to their entries), Helm charts (`Chart`, `DEPENDS_ON`), Terraform / OpenTofu (`.tf`, `.tofu`, `.tfvars`) as graph nodes. A Service walks to the code it runs: Service → workload → `Image` → Dockerfile → `main`
267+
- **Infrastructure-as-code indexing** — Dockerfiles (`Dockerfile.*`, `*.Dockerfile`, `Containerfile`: bases, what the container runs, `ENV`), container images (`Image`: the normalised repository, tag and digest on the edges; a repository built from several Dockerfiles, one per component tag, gets an `Image` per component), docker compose services (`Resource`), Kubernetes manifests (`Resource`, with `group`, `version`, `kind`, `namespace`), Kustomize overlays (`IMPORTS` to their entries and patch files, `OVERRIDE` from their patches), Helm charts (`Chart`, `DEPENDS_ON`), Terraform / OpenTofu (`.tf`, `.tofu`, `.tfvars`) as graph nodes. A Service walks to the code it runs: Service → workload → `Image` → Dockerfile → `main`
267268
- **[Hybrid LSP semantic type resolution](#hybrid-lsp)** for Python, TypeScript / JavaScript / JSX / TSX, PHP, C#, Go, C, C++, Java, Kotlin, Rust, and Perl — a lightweight C implementation of language type-resolution algorithms, structurally inspired by and compatible with major language servers including tsserver / typescript-go, pyright, gopls, Roslyn, Eclipse JDT, and rust-analyzer (parameter binding, return-type inference, generic substitution, JSX component dispatch, JSDoc inference for plain JS files, namespace + trait + late-static-binding resolution for PHP, file-scoped namespaces + records + LINQ method syntax for C#, class-hierarchy + overload + lambda resolution for Java, extension-function + scope-function resolution for Kotlin, trait-method + UFCS resolution for Rust)
268269
- **RAM-first pipeline**: LZ4 compression, in-memory SQLite, single dump at end. Memory released after.
269270

‎internal/cbm/cbm.h‎

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -719,15 +719,23 @@ typedef struct {
719719
* (NULL: unnamed) from image `name` (a literal reference); via "container".
720720
* CBM_K8S_FACT_ENV: object def_index sets environment variable `name` in
721721
* its container `container` (NULL: unnamed) at `line`; via = where the
722-
* value comes from ("value", or the valueFrom kind) -- never the value. */
722+
* value comes from ("value", or the valueFrom kind) -- never the value.
723+
* CBM_K8S_FACT_PATCH: a kustomization's `patches` entry at `line` patches
724+
* the objects of its build named by kind / group / version / name / ns
725+
* (any NULL: any) -- its `target`, else an inline patch document's own
726+
* kind / group / name / ns; `path` is
727+
* the patch file (as written, relative to the kustomization), NULL for an
728+
* inline patch. kind and name both NULL: the patch file's documents name
729+
* them. def_index -1: the kustomization itself is the source. */
723730
typedef enum {
724731
CBM_K8S_FACT_REF = 0,
725732
CBM_K8S_FACT_CRD,
726733
CBM_K8S_FACT_DEPLOY,
727734
CBM_K8S_FACT_ROUTE,
728735
CBM_K8S_FACT_SOURCE,
729736
CBM_K8S_FACT_IMAGE,
730-
CBM_K8S_FACT_ENV
737+
CBM_K8S_FACT_ENV,
738+
CBM_K8S_FACT_PATCH
731739
} CBMK8sFactKind;
732740
typedef struct {
733741
CBMK8sFactKind fact;
@@ -739,7 +747,10 @@ typedef struct {
739747
const char *edge;
740748
const char *route;
741749
const char *container;
742-
const char *key; /* CONNECTS_TO: the environment variable addressing it */
750+
const char *key; /* CONNECTS_TO: the environment variable addressing it */
751+
const char *group; /* PATCH: the API group named ("core"), NULL: any */
752+
const char *version; /* PATCH: the API version named, NULL: any */
753+
const char *path; /* PATCH: the patch file, NULL: inline */
743754
int line;
744755
} CBMK8sFact;
745756

‎internal/cbm/extract_k8s.c‎

Lines changed: 254 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,15 @@
1515
#include "helpers.h"
1616
#include "tree_sitter/api.h"
1717
#include "foundation/constants.h"
18+
#include <ctype.h>
1819
#include <stdint.h>
1920
#include <stdio.h>
2021
#include <string.h>
2122

2223
/* Local constants. */
2324
enum {
2425
K8S_BUF_SIZE = 256,
26+
K8S_PROPS_KV = 3, /* extra_props: one key, its value, the NULL end */
2527
};
2628

2729
// ---------------------------------------------------------------------------
@@ -217,6 +219,240 @@ static TSNode unwrap_pair_value(TSNode pair) {
217219
return val_node;
218220
}
219221

222+
// ---------------------------------------------------------------------------
223+
// Kustomize patches
224+
// ---------------------------------------------------------------------------
225+
226+
// One `patches` entry of a kustomization (the kustomize v5 mapping form). The
227+
// patch file of `path` is an import (local name "patches"); what the entry
228+
// patches is a PATCH fact the K8s pass resolves in the kustomization's build:
229+
// its `target`, else each document of an inline `patch`; a path patch
230+
// without a target names the objects of its own documents, which the pass
231+
// reads from the patch file's Resources. A target the pass cannot evaluate
232+
// (a label or annotation selector) names nothing.
233+
234+
typedef struct {
235+
const char *kind;
236+
const char *group;
237+
const char *version;
238+
const char *name;
239+
const char *ns;
240+
bool selector; /* labelSelector / annotationSelector */
241+
} kust_target_t;
242+
243+
static void kust_target(CBMExtractCtx *ctx, TSNode map, kust_target_t *t) {
244+
CBMArena *a = ctx->arena;
245+
k8s_kids_t pairs = k8s_kids(map);
246+
TSNode pair;
247+
while (k8s_kid(&pairs, false, &pair)) {
248+
if (strcmp(ts_node_type(pair), "block_mapping_pair") != 0) {
249+
continue;
250+
}
251+
const char *key = get_scalar_text(a, ts_node_named_child(pair, 0), ctx->source);
252+
TSNode v = unwrap_pair_value(pair);
253+
const char *val = key && !ts_node_is_null(v) ? get_scalar_text(a, v, ctx->source) : NULL;
254+
if (!val || !val[0]) {
255+
continue;
256+
}
257+
if (strcmp(key, "kind") == 0) {
258+
t->kind = val;
259+
} else if (strcmp(key, "group") == 0) {
260+
t->group = val;
261+
} else if (strcmp(key, "version") == 0) {
262+
t->version = val;
263+
} else if (strcmp(key, "name") == 0) {
264+
t->name = val;
265+
} else if (strcmp(key, "namespace") == 0) {
266+
t->ns = val;
267+
} else if (strcmp(key, "labelSelector") == 0 || strcmp(key, "annotationSelector") == 0) {
268+
t->selector = true;
269+
}
270+
}
271+
}
272+
273+
// The block mapping a sequence item holds, null when it holds none (a
274+
// string entry, which emit_kustomize_sequence imports).
275+
static TSNode kust_item_mapping(TSNode item) {
276+
TSNode v = first_content_child(item);
277+
if (!ts_node_is_null(v) && strcmp(ts_node_type(v), "block_node") == 0) {
278+
v = first_content_child(v);
279+
}
280+
if (ts_node_is_null(v) || strcmp(ts_node_type(v), "block_mapping") != 0) {
281+
TSNode null_node = {0};
282+
return null_node;
283+
}
284+
return v;
285+
}
286+
287+
// The text of an inline patch: a block scalar's lines after its header line
288+
// (`|`, `|-`, `|2`...), or a one-line scalar.
289+
static const char *kust_inline_text(CBMExtractCtx *ctx, TSNode v) {
290+
if (strcmp(ts_node_type(v), "block_scalar") == 0) {
291+
const char *raw = cbm_node_text(ctx->arena, v, ctx->source);
292+
const char *nl = raw ? strchr(raw, '\n') : NULL;
293+
return nl ? nl + SKIP_ONE : NULL;
294+
}
295+
return get_scalar_text(ctx->arena, v, ctx->source);
296+
}
297+
298+
// One line of an inline patch: its indent and `key: value` (the value
299+
// unquoted, a trailing comment dropped); false for a blank or comment line.
300+
static bool kust_line(const char *p, size_t len, int *indent, char *key, size_t ksz, char *val,
301+
size_t vsz) {
302+
size_t i = 0;
303+
while (i < len && p[i] == ' ') {
304+
i++;
305+
}
306+
if (i == len || p[i] == '#') {
307+
return false;
308+
}
309+
*indent = (int)i;
310+
size_t k = i;
311+
while (k < len && p[k] != ':') {
312+
k++;
313+
}
314+
snprintf(key, ksz, "%.*s", (int)(k - i), p + i);
315+
val[0] = '\0';
316+
size_t s = k < len ? k + SKIP_ONE : len;
317+
while (s < len && p[s] == ' ') {
318+
s++;
319+
}
320+
size_t e = s;
321+
while (e < len && !(p[e] == '#' && p[e - SKIP_ONE] == ' ')) {
322+
e++;
323+
}
324+
while (e > s && (p[e - SKIP_ONE] == ' ' || p[e - SKIP_ONE] == '\r')) {
325+
e--;
326+
}
327+
if (e - s >= PAIR_LEN && (p[s] == '"' || p[s] == '\'') && p[e - SKIP_ONE] == p[s]) {
328+
s++;
329+
e--;
330+
}
331+
snprintf(val, vsz, "%.*s", (int)(e - s), p + s);
332+
return true;
333+
}
334+
335+
// Each document of an inline patch with a kind and a metadata.name: a PATCH
336+
// fact naming that object -- its kind, the group of its apiVersion ("core"
337+
// for the core group, as on a Resource), its name and namespace. A JSON6902
338+
// operation list names nothing.
339+
static const char KUST_DOC_END[] = "---";
340+
341+
static void kust_inline_facts(CBMExtractCtx *ctx, const char *text, CBMK8sFact base) {
342+
CBMArena *a = ctx->arena;
343+
CBMK8sFact f = base;
344+
int top = -1; /* indent of the document's keys */
345+
int meta = -1; /* indent of metadata's keys, -1: not inside metadata */
346+
bool in_meta = false;
347+
for (const char *p = text; p;) {
348+
const char *eol = strchr(p, '\n');
349+
size_t len = eol ? (size_t)(eol - p) : strlen(p);
350+
size_t b = strspn(p, " ");
351+
bool doc_end = b + strlen(KUST_DOC_END) <= len &&
352+
strncmp(p + b, KUST_DOC_END, strlen(KUST_DOC_END)) == 0;
353+
int ind = 0;
354+
char key[K8S_BUF_SIZE];
355+
char val[K8S_BUF_SIZE];
356+
if (!doc_end && kust_line(p, len, &ind, key, sizeof(key), val, sizeof(val))) {
357+
if (top < 0) {
358+
top = ind;
359+
}
360+
if (ind == top) {
361+
in_meta = strcmp(key, "metadata") == 0 && !val[0];
362+
meta = -1;
363+
if (strcmp(key, "kind") == 0 && val[0]) {
364+
f.kind = cbm_arena_strdup(a, val);
365+
} else if (strcmp(key, "apiVersion") == 0 && val[0]) {
366+
const char *slash = strchr(val, '/');
367+
f.group = slash ? cbm_arena_strndup(a, val, (size_t)(slash - val)) : "core";
368+
}
369+
} else if (in_meta && ind > top) {
370+
meta = meta < 0 ? ind : meta;
371+
if (ind == meta && strcmp(key, "name") == 0 && val[0]) {
372+
f.name = cbm_arena_strdup(a, val);
373+
} else if (ind == meta && strcmp(key, "namespace") == 0 && val[0]) {
374+
f.ns = cbm_arena_strdup(a, val);
375+
}
376+
}
377+
}
378+
if (doc_end || !eol) {
379+
if (f.kind && f.name) {
380+
cbm_k8sfact_push(&ctx->result->k8s_facts, a, f);
381+
}
382+
f = base;
383+
top = -1;
384+
in_meta = false;
385+
}
386+
p = eol ? eol + SKIP_ONE : NULL;
387+
}
388+
}
389+
390+
// The mapping entries of a kustomization's `patches` list.
391+
static void emit_kustomize_patches(CBMExtractCtx *ctx, TSNode seq) {
392+
CBMArena *a = ctx->arena;
393+
k8s_kids_t items = k8s_kids(seq);
394+
TSNode item;
395+
while (k8s_kid(&items, false, &item)) {
396+
TSNode map = strcmp(ts_node_type(item), "block_sequence_item") == 0
397+
? kust_item_mapping(item)
398+
: (TSNode){0};
399+
if (ts_node_is_null(map)) {
400+
continue;
401+
}
402+
const char *path = NULL;
403+
const char *inline_text = NULL;
404+
kust_target_t t = {0};
405+
bool has_target = false;
406+
k8s_kids_t pairs = k8s_kids(map);
407+
TSNode pair;
408+
while (k8s_kid(&pairs, false, &pair)) {
409+
if (strcmp(ts_node_type(pair), "block_mapping_pair") != 0) {
410+
continue;
411+
}
412+
const char *key = get_scalar_text(a, ts_node_named_child(pair, 0), ctx->source);
413+
TSNode v = unwrap_pair_value(pair);
414+
if (!key || ts_node_is_null(v)) {
415+
continue;
416+
}
417+
if (strcmp(key, "path") == 0) {
418+
path = get_scalar_text(a, v, ctx->source);
419+
} else if (strcmp(key, "patch") == 0) {
420+
inline_text = kust_inline_text(ctx, v);
421+
} else if (strcmp(key, "target") == 0 &&
422+
strcmp(ts_node_type(v), "block_mapping") == 0) {
423+
kust_target(ctx, v, &t);
424+
has_target = true;
425+
}
426+
}
427+
if (path && path[0]) {
428+
CBMImport imp = {.local_name = cbm_arena_strdup(a, "patches"),
429+
.module_path = cbm_arena_strdup(a, path)};
430+
cbm_imports_push(&ctx->result->imports, a, imp);
431+
}
432+
CBMK8sFact base = {.fact = CBM_K8S_FACT_PATCH,
433+
.def_index = -1,
434+
.via = "patch",
435+
.edge = "OVERRIDE",
436+
.path = path && path[0] ? path : NULL,
437+
.line = (int)ts_node_start_point(item).row + TS_LINE_OFFSET};
438+
if (has_target) {
439+
if (!t.selector && (t.kind || t.name)) {
440+
CBMK8sFact f = base;
441+
f.kind = t.kind;
442+
f.group = t.group;
443+
f.version = t.version;
444+
f.name = t.name;
445+
f.ns = t.ns;
446+
cbm_k8sfact_push(&ctx->result->k8s_facts, a, f);
447+
}
448+
} else if (inline_text) {
449+
kust_inline_facts(ctx, inline_text, base);
450+
} else if (base.path) {
451+
cbm_k8sfact_push(&ctx->result->k8s_facts, a, base);
452+
}
453+
}
454+
}
455+
220456
// Process a single block_mapping_pair for kustomize list keys.
221457
static void process_kustomize_pair(CBMExtractCtx *ctx, TSNode pair) {
222458
if (strcmp(ts_node_type(pair), "block_mapping_pair") != 0) {
@@ -236,6 +472,9 @@ static void process_kustomize_pair(CBMExtractCtx *ctx, TSNode pair) {
236472
return;
237473
}
238474
emit_kustomize_sequence(ctx, val_node, key_text);
475+
if (strcmp(key_text, "patches") == 0) {
476+
emit_kustomize_patches(ctx, val_node);
477+
}
239478
}
240479

241480
// Emit a "Class" def named after the document's `kind` scalar. A kustomization
@@ -1349,6 +1588,21 @@ static void extract_k8s_manifest(CBMExtractCtx *ctx) {
13491588
def.file_path = ctx->rel_path;
13501589
def.start_line = ts_node_start_point(mapping).row + TS_LINE_OFFSET;
13511590
def.end_line = ts_node_end_point(mapping).row + TS_LINE_OFFSET;
1591+
// The version of its apiVersion (`v1beta2` of cluster.x-k8s.io/v1beta2,
1592+
// `v1` of v1), which the QN leaves out (one object across versions); a
1593+
// bare group without one (`apiVersion: example.io`) has none.
1594+
const char *slash = strchr(id.api_version, '/');
1595+
const char *version = slash ? slash + SKIP_ONE : id.api_version;
1596+
bool versioned = slash || (version[0] == 'v' && isdigit((unsigned char)version[1]));
1597+
if (version[0] && versioned && id.api_version[0] != '<') {
1598+
const char **kv = (const char **)cbm_arena_alloc(a, K8S_PROPS_KV * sizeof(char *));
1599+
if (kv) {
1600+
kv[0] = "version";
1601+
kv[SKIP_ONE] = cbm_arena_strdup(a, version);
1602+
kv[PAIR_LEN] = NULL;
1603+
def.extra_props = kv;
1604+
}
1605+
}
13521606
cbm_defs_push(&ctx->result->defs, a, def);
13531607
k8s_emit_facts(ctx, mapping, group, id.kind, ctx->result->defs.count - SKIP_ONE);
13541608
}

0 commit comments

Comments
 (0)