Skip to content

Commit e58c5ba

Browse files
committed
feat(infra): a Resource keeps its labels; a patch target selects by them
A Kubernetes object's metadata.labels were not in the graph: neither a search for an object by label nor a kustomize patch whose target selects by label could find anything. A Resource now keeps its labels as `labels: "k=v,k2=v2"` (sorted; a templated label, or one a selector cannot name, is left out), and a patch target's labelSelector meets the objects of its build carrying its terms -- `k=v` / `k==v`, `k!=v`, `k`, `!k`; a set-based term names no key and meets nothing. A target with only a selector selects among every kind; an annotationSelector is not evaluated and names nothing. Measured on 17 infrastructure repositories: 1,956 Resources gain labels and nothing else; an independent reading of metadata.labels agrees on every one of the 1,804 objects both read (152 more are Go-templated, which only the graph reads, through its skeleton). The 4 label-selector patch pairs now resolve: kustomize patches 130 of 130. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent 35833e1 commit e58c5ba

6 files changed

Lines changed: 232 additions & 45 deletions

File tree

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -255,7 +255,7 @@ The install script placed beside the binary is **reported, not deleted** — uni
255255
- `DEPLOYS` — what puts manifests, kustomizations or charts on a cluster, to what it puts there: a Flux Kustomization or Argo Application to the kustomizations / manifests / chart of its path, a Flux HelmRelease to its chart (`spec.chart` or `chartRef` → OCIRepository), a shell command or Make recipe (`kubectl apply -f/-k`, `kustomize build | kubectl apply`, `helm install/upgrade/template`), a Go installer loading a chart with the Helm SDK, a Terraform `helm_release`, a kustomization inflating a chart (`helmCharts`; its values file an `IMPORTS`). A chart outside the repository is a shared external `Chart`; the edge says `via` (`flux`, `argo`, `helmrelease`, `kubectl`, `kustomize`, `helm`, `helm-sdk`, `terraform`, `helmcharts`, ...)
256256
- `DEPENDS_ON` — a Helm chart to the charts it declares (the local chart for a resolvable `file://` repository, otherwise an external `Chart`; the edge keeps `repository`), and a `go.mod` / `requirements.txt` file to an external `Package`; a Dockerfile to the `Image`s it builds on (`via: from`, `copy-from`); an `Image` to the Dockerfile it is built from (`via: build`; `tool`: `skaffold`, `compose`, `docker-build` for `docker` / `podman` / `nerdctl` / `buildah` build commands in scripts, Make and CI); a compose service to the services its `depends_on` / `links` name
257257
- `CONFIGURES` — code reading an environment variable to the shared `EnvVar` of that name (`strategy: env_access`); a Kubernetes container, compose service or Dockerfile (`ENV` of the image) setting one to that container's own `EnvVar` (`strategy: env_set`, with `source`: `value`, `configMapKeyRef`, `secretKeyRef`, `fieldRef`, ..., `compose`, `dockerfile` — values are never stored); and that container's `EnvVar` to the code reading the name inside the files its image is built from (`strategy: env_read`, with `dockerfile`) — the image's files are the sources its Dockerfile's `COPY` / `ADD` put into it, less the context's `.dockerignore`
258-
- `OVERRIDE` with `via: patch` — a Kustomize patch to each object of its kustomization's build it changes (child → base): a patch file's documents to the object of the same kind and name; an inline patch (from the kustomization) or a JSON6902 operation file to what its `target` selects (group, version, kind, name, namespace); a component's in the build of each kustomization including it. The edge keeps `kustomization`, `patch` and `line`; a patch file is an `IMPORTS` (`local_name: patches`) of its kustomization, never a member of the build
258+
- `OVERRIDE` with `via: patch` — a Kustomize patch to each object of its kustomization's build it changes (child → base): a patch file's documents to the object of the same kind and name; an inline patch (from the kustomization) or a JSON6902 operation file to what its `target` selects (group, version, kind, name, namespace, label selector); a component's in the build of each kustomization including it. The edge keeps `kustomization`, `patch` and `line`; a patch file is an `IMPORTS` (`local_name: patches`) of its kustomization, never a member of the build
259259
- `EMITS`, `LISTENS_ON` (channels)
260260
- `DATA_FLOWS` with arg-to-param mapping + field access chains
261261
- `SIMILAR_TO` (MinHash + LSH near-clone detection, Jaccard scored)
@@ -264,7 +264,7 @@ The install script placed beside the binary is **reported, not deleted** — uni
264264
### Indexing pipeline
265265
- **158 vendored tree-sitter grammars** compiled into the binary
266266
- **Generic package / module resolution** — bare specifiers like `@myorg/pkg`, `github.com/foo/bar`, `use my_crate::foo` resolved via manifest scanning (`package.json`, `go.mod`, `Cargo.toml`, `pyproject.toml`, `composer.json`, `pubspec.yaml`, `pom.xml`, `build.gradle`, `mix.exs`, `*.gemspec`)
267-
- **Infrastructure-as-code indexing** — Dockerfiles (`Dockerfile.*`, `*.Dockerfile`, `Containerfile`: bases, what the container runs, `ENV`), container images (`Image`: the normalised repository, tag and digest on the edges; a repository built from several Dockerfiles, one per component tag, gets an `Image` per component), docker compose services (`Resource`), Kubernetes manifests (`Resource`, with `group`, `version`, `kind`, `namespace`), Kustomize overlays (`IMPORTS` to their entries and patch files, `OVERRIDE` from their patches), Helm charts (`Chart`, `DEPENDS_ON`), Terraform / OpenTofu (`.tf`, `.tofu`, `.tfvars`) as graph nodes. A Service walks to the code it runs: Service → workload → `Image` → Dockerfile → `main`
267+
- **Infrastructure-as-code indexing** — Dockerfiles (`Dockerfile.*`, `*.Dockerfile`, `Containerfile`: bases, what the container runs, `ENV`), container images (`Image`: the normalised repository, tag and digest on the edges; a repository built from several Dockerfiles, one per component tag, gets an `Image` per component), docker compose services (`Resource`), Kubernetes manifests (`Resource`, with `group`, `version`, `kind`, `namespace`, `labels` as `k=v,k2=v2`), Kustomize overlays (`IMPORTS` to their entries and patch files, `OVERRIDE` from their patches), Helm charts (`Chart`, `DEPENDS_ON`), Terraform / OpenTofu (`.tf`, `.tofu`, `.tfvars`) as graph nodes. A Service walks to the code it runs: Service → workload → `Image` → Dockerfile → `main`
268268
- **[Hybrid LSP semantic type resolution](#hybrid-lsp)** for Python, TypeScript / JavaScript / JSX / TSX, PHP, C#, Go, C, C++, Java, Kotlin, Rust, and Perl — a lightweight C implementation of language type-resolution algorithms, structurally inspired by and compatible with major language servers including tsserver / typescript-go, pyright, gopls, Roslyn, Eclipse JDT, and rust-analyzer (parameter binding, return-type inference, generic substitution, JSX component dispatch, JSDoc inference for plain JS files, namespace + trait + late-static-binding resolution for PHP, file-scoped namespaces + records + LINQ method syntax for C#, class-hierarchy + overload + lambda resolution for Java, extension-function + scope-function resolution for Kotlin, trait-method + UFCS resolution for Rust)
269269
- **RAM-first pipeline**: LZ4 compression, in-memory SQLite, single dump at end. Memory released after.
270270

‎internal/cbm/cbm.h‎

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -750,12 +750,13 @@ typedef struct {
750750
const char *edge;
751751
const char *route;
752752
const char *container;
753-
const char *key; /* CONNECTS_TO: the environment variable addressing it */
754-
const char *group; /* PATCH: the API group named ("core"), NULL: any */
755-
const char *version; /* PATCH: the API version named, NULL: any */
756-
const char *path; /* PATCH: the patch file, NULL: inline */
757-
const char *url; /* DEPLOY (kustomize): the chart repository */
758-
const char *release; /* DEPLOY (kustomize): the release name */
753+
const char *key; /* CONNECTS_TO: the environment variable addressing it */
754+
const char *group; /* PATCH: the API group named ("core"), NULL: any */
755+
const char *version; /* PATCH: the API version named, NULL: any */
756+
const char *path; /* PATCH: the patch file, NULL: inline */
757+
const char *selector; /* PATCH: the target's labelSelector, NULL: none */
758+
const char *url; /* DEPLOY (kustomize): the chart repository */
759+
const char *release; /* DEPLOY (kustomize): the release name */
759760
int line;
760761
} CBMK8sFact;
761762

‎internal/cbm/extract_k8s.c‎

Lines changed: 91 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@
2323
/* Local constants. */
2424
enum {
2525
K8S_BUF_SIZE = 256,
26-
K8S_PROPS_KV = 3, /* extra_props: one key, its value, the NULL end */
26+
K8S_PROPS_KV = 5, /* extra_props: version, labels (key + value each), NULL */
2727
};
2828

2929
// ---------------------------------------------------------------------------
@@ -237,7 +237,8 @@ typedef struct {
237237
const char *version;
238238
const char *name;
239239
const char *ns;
240-
bool selector; /* labelSelector / annotationSelector */
240+
const char *selector; /* labelSelector */
241+
bool unsupported; /* annotationSelector: not evaluated */
241242
} kust_target_t;
242243

243244
static void kust_target(CBMExtractCtx *ctx, TSNode map, kust_target_t *t) {
@@ -264,8 +265,10 @@ static void kust_target(CBMExtractCtx *ctx, TSNode map, kust_target_t *t) {
264265
t->name = val;
265266
} else if (strcmp(key, "namespace") == 0) {
266267
t->ns = val;
267-
} else if (strcmp(key, "labelSelector") == 0 || strcmp(key, "annotationSelector") == 0) {
268-
t->selector = true;
268+
} else if (strcmp(key, "labelSelector") == 0) {
269+
t->selector = val;
270+
} else if (strcmp(key, "annotationSelector") == 0) {
271+
t->unsupported = true;
269272
}
270273
}
271274
}
@@ -436,13 +439,14 @@ static void emit_kustomize_patches(CBMExtractCtx *ctx, TSNode seq) {
436439
.path = path && path[0] ? path : NULL,
437440
.line = (int)ts_node_start_point(item).row + TS_LINE_OFFSET};
438441
if (has_target) {
439-
if (!t.selector && (t.kind || t.name)) {
442+
if (!t.unsupported && (t.kind || t.name || t.selector)) {
440443
CBMK8sFact f = base;
441444
f.kind = t.kind;
442445
f.group = t.group;
443446
f.version = t.version;
444447
f.name = t.name;
445448
f.ns = t.ns;
449+
f.selector = t.selector;
446450
cbm_k8sfact_push(&ctx->result->k8s_facts, a, f);
447451
}
448452
} else if (inline_text) {
@@ -613,8 +617,72 @@ typedef struct {
613617
char kind[K8S_BUF_SIZE];
614618
char name[K8S_BUF_SIZE];
615619
char ns[K8S_BUF_SIZE];
620+
const char *labels; // metadata.labels as "k=v,k2=v2" (text order), NULL: none
616621
} k8s_doc_id_t;
617622

623+
// A label a selector term can name: no templated part (a skeleton's `<...>`,
624+
// `{{`), no `,` or `=` (the separators of the list), nothing JSON escapes
625+
// (`"`, `\`, control characters -- none in a valid label).
626+
static bool k8s_label_plain(const char *s) {
627+
if (s[0] == '<' || strstr(s, "{{") || strpbrk(s, ",=\"\\")) {
628+
return false;
629+
}
630+
for (const char *p = s; *p; p++) {
631+
if ((unsigned char)*p < ' ') {
632+
return false;
633+
}
634+
}
635+
return true;
636+
}
637+
638+
static int k8s_cmp_label(const void *x, const void *y) {
639+
return strcmp(*(const char *const *)x, *(const char *const *)y);
640+
}
641+
642+
// metadata.labels as "k=v,k2=v2" in text order (the arena's), NULL when it
643+
// has none a selector can name.
644+
static const char *k8s_labels(CBMArena *a, TSNode map, const char *source) {
645+
if (strcmp(ts_node_type(map), "block_mapping") != 0) {
646+
return NULL;
647+
}
648+
int cap = (int)ts_node_named_child_count(map);
649+
const char **items = cap > 0 ? cbm_arena_alloc(a, (size_t)cap * sizeof(*items)) : NULL;
650+
int n = 0;
651+
size_t len = 0;
652+
k8s_kids_t pairs = k8s_kids(map);
653+
TSNode p;
654+
while (items && n < cap && k8s_kid(&pairs, false, &p)) {
655+
if (strcmp(ts_node_type(p), "block_mapping_pair") != 0) {
656+
continue;
657+
}
658+
const char *k = get_scalar_text(a, ts_node_named_child(p, 0), source);
659+
TSNode vn = unwrap_pair_value(p);
660+
const char *v = ts_node_is_null(vn) ? NULL : get_scalar_text(a, vn, source);
661+
if (k && k[0] && v && k8s_label_plain(k) && k8s_label_plain(v)) {
662+
items[n] = cbm_arena_sprintf(a, "%s=%s", k, v);
663+
len += items[n] ? strlen(items[n]) + SKIP_ONE : 0;
664+
n += items[n] ? SKIP_ONE : 0;
665+
}
666+
}
667+
k8s_kids_done(&pairs);
668+
if (n == 0) {
669+
return NULL;
670+
}
671+
qsort((void *)items, (size_t)n, sizeof(*items), k8s_cmp_label);
672+
char *out = cbm_arena_alloc(a, len);
673+
if (!out) {
674+
return NULL;
675+
}
676+
size_t w = 0;
677+
for (int i = 0; i < n; i++) {
678+
size_t l = strlen(items[i]);
679+
memcpy(out + w, items[i], l);
680+
w += l;
681+
out[w++] = i + SKIP_ONE < n ? ',' : '\0';
682+
}
683+
return out;
684+
}
685+
618686
// Read metadata.name and metadata.namespace from a metadata block_mapping.
619687
static void extract_metadata(CBMArena *a, TSNode meta_mapping, const char *source,
620688
k8s_doc_id_t *id) {
@@ -634,10 +702,14 @@ static void extract_metadata(CBMArena *a, TSNode meta_mapping, const char *sourc
634702
const char *mkey_text = get_scalar_text(a, mkey, source);
635703
bool is_name = mkey_text && strcmp(mkey_text, "name") == 0;
636704
bool is_ns = mkey_text && strcmp(mkey_text, "namespace") == 0;
705+
TSNode mval = unwrap_pair_value(mpair);
706+
if (mkey_text && strcmp(mkey_text, "labels") == 0 && !ts_node_is_null(mval)) {
707+
id->labels = k8s_labels(a, mval, source);
708+
continue;
709+
}
637710
if (!is_name && !is_ns) {
638711
continue;
639712
}
640-
TSNode mval = unwrap_pair_value(mpair);
641713
const char *text = ts_node_is_null(mval) ? NULL : get_scalar_text(a, mval, source);
642714
if (text) {
643715
snprintf(is_name ? id->name : id->ns, K8S_BUF_SIZE, "%s", text);
@@ -1658,14 +1730,19 @@ static void extract_k8s_manifest(CBMExtractCtx *ctx) {
16581730
const char *slash = strchr(id.api_version, '/');
16591731
const char *version = slash ? slash + SKIP_ONE : id.api_version;
16601732
bool versioned = slash || (version[0] == 'v' && isdigit((unsigned char)version[1]));
1661-
if (version[0] && versioned && id.api_version[0] != '<') {
1662-
const char **kv = (const char **)cbm_arena_alloc(a, K8S_PROPS_KV * sizeof(char *));
1663-
if (kv) {
1664-
kv[0] = "version";
1665-
kv[SKIP_ONE] = cbm_arena_strdup(a, version);
1666-
kv[PAIR_LEN] = NULL;
1667-
def.extra_props = kv;
1668-
}
1733+
const char **kv = (const char **)cbm_arena_alloc(a, K8S_PROPS_KV * sizeof(char *));
1734+
int kn = 0;
1735+
if (kv && version[0] && versioned && id.api_version[0] != '<') {
1736+
kv[kn++] = "version";
1737+
kv[kn++] = cbm_arena_strdup(a, version);
1738+
}
1739+
if (kv && id.labels) {
1740+
kv[kn++] = "labels";
1741+
kv[kn++] = id.labels;
1742+
}
1743+
if (kv && kn > 0) {
1744+
kv[kn] = NULL;
1745+
def.extra_props = kv;
16691746
}
16701747
cbm_defs_push(&ctx->result->defs, a, def);
16711748
k8s_emit_facts(ctx, mapping, group, id.kind, ctx->result->defs.count - SKIP_ONE);

0 commit comments

Comments
 (0)