Skip to content

Commit 35833e1

Browse files
committed
feat(infra): a kustomization DEPLOYS the charts it inflates
A kustomization's `helmCharts` inflate Helm charts into its build -- the fourth way a repository puts a chart on a cluster, beside a Flux HelmRelease, a Terraform helm_release and a helm command; the graph read none of them. Each entry now DEPLOYS from the kustomization -- its Kustomization Resource, else its File: the unit a Flux or Argo deploy of it reaches -- to the shared external Chart of that name, with {via: helmcharts, external, chart, url, version, release, namespace, values, line}. A chart name with a path names nothing; the values file inside the repository is an IMPORTS (local_name: valuesFile) of the kustomization. Measured on 17 infrastructure repositories: 20 edges, every one in an independent reading of the kustomizations (20 entries, cluster-api 10, k8s.io 10); no other node or edge changes. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent 947d3a1 commit 35833e1

6 files changed

Lines changed: 246 additions & 19 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -252,7 +252,7 @@ The install script placed beside the binary is **reported, not deleted** — uni
252252
- `INFRA_MAPS` — a Kubernetes Service to each workload whose pod labels carry every `key: value` of the Service's selector, within one namespace
253253
- `CONNECTS_TO` — a Kubernetes workload or compose service to the Service its containers' environment addresses (`PRODUCT_CATALOG_SERVICE_ADDR: productcatalogservice:3550`, a URL's host, `<svc>.<ns>.svc...`, a bare name on an `*_ADDR` / `*_HOST` / `*_URL` / ... key; compose: the project's `.env` interpolated): a Kubernetes Service of the same bundle and namespace, or a compose service of the same file. The edge keeps `via: env`, `key` and `container` — never the value
254254
- `USAGE` with `ref: image` — a Kubernetes workload (any pod template: containers, initContainers, CronJobs, custom resources) or a docker compose service to the `Image` it runs
255-
- `DEPLOYS` — what puts manifests, kustomizations or charts on a cluster, to what it puts there: a Flux Kustomization or Argo Application to the kustomizations / manifests / chart of its path, a Flux HelmRelease to its chart (`spec.chart` or `chartRef` → OCIRepository), a shell command or Make recipe (`kubectl apply -f/-k`, `kustomize build | kubectl apply`, `helm install/upgrade/template`), a Go installer loading a chart with the Helm SDK, a Terraform `helm_release`. A chart outside the repository is a shared external `Chart`; the edge says `via` (`flux`, `argo`, `helmrelease`, `kubectl`, `kustomize`, `helm`, `helm-sdk`, `terraform`, ...)
255+
- `DEPLOYS` — what puts manifests, kustomizations or charts on a cluster, to what it puts there: a Flux Kustomization or Argo Application to the kustomizations / manifests / chart of its path, a Flux HelmRelease to its chart (`spec.chart` or `chartRef` → OCIRepository), a shell command or Make recipe (`kubectl apply -f/-k`, `kustomize build | kubectl apply`, `helm install/upgrade/template`), a Go installer loading a chart with the Helm SDK, a Terraform `helm_release`, a kustomization inflating a chart (`helmCharts`; its values file an `IMPORTS`). A chart outside the repository is a shared external `Chart`; the edge says `via` (`flux`, `argo`, `helmrelease`, `kubectl`, `kustomize`, `helm`, `helm-sdk`, `terraform`, `helmcharts`, ...)
256256
- `DEPENDS_ON` — a Helm chart to the charts it declares (the local chart for a resolvable `file://` repository, otherwise an external `Chart`; the edge keeps `repository`), and a `go.mod` / `requirements.txt` file to an external `Package`; a Dockerfile to the `Image`s it builds on (`via: from`, `copy-from`); an `Image` to the Dockerfile it is built from (`via: build`; `tool`: `skaffold`, `compose`, `docker-build` for `docker` / `podman` / `nerdctl` / `buildah` build commands in scripts, Make and CI); a compose service to the services its `depends_on` / `links` name
257257
- `CONFIGURES` — code reading an environment variable to the shared `EnvVar` of that name (`strategy: env_access`); a Kubernetes container, compose service or Dockerfile (`ENV` of the image) setting one to that container's own `EnvVar` (`strategy: env_set`, with `source`: `value`, `configMapKeyRef`, `secretKeyRef`, `fieldRef`, ..., `compose`, `dockerfile` — values are never stored); and that container's `EnvVar` to the code reading the name inside the files its image is built from (`strategy: env_read`, with `dockerfile`) — the image's files are the sources its Dockerfile's `COPY` / `ADD` put into it, less the context's `.dockerignore`
258258
- `OVERRIDE` with `via: patch` — a Kustomize patch to each object of its kustomization's build it changes (child → base): a patch file's documents to the object of the same kind and name; an inline patch (from the kustomization) or a JSON6902 operation file to what its `target` selects (group, version, kind, name, namespace); a component's in the build of each kustomization including it. The edge keeps `kustomization`, `patch` and `line`; a patch file is an `IMPORTS` (`local_name: patches`) of its kustomization, never a member of the build

‎internal/cbm/cbm.h‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -708,7 +708,10 @@ typedef struct {
708708
* CBM_K8S_FACT_CRD: the CRD def_index declares kind `kind` in group `name`.
709709
* CBM_K8S_FACT_DEPLOY: object def_index (a Flux Kustomization, an Argo
710710
* Application) deploys the repository directory `name` into namespace
711-
* `ns` (NULL: the objects' own); via "flux" / "argo".
711+
* `ns` (NULL: the objects' own); via "flux" / "argo". via "kustomize"
712+
* (def_index -1: the kustomization itself): a `helmCharts` entry
713+
* inflates chart `name` of repository `url` at `version` as release
714+
* `release` into namespace `ns`, with values file `path`.
712715
* CBM_K8S_FACT_ROUTE: object def_index (an Ingress, an HTTPRoute) exposes
713716
* `route` ("<host or *><path>") and sends it to the kind/ns/name backend
714717
* (name NULL: none); via "ingress" / "httproute".
@@ -751,6 +754,8 @@ typedef struct {
751754
const char *group; /* PATCH: the API group named ("core"), NULL: any */
752755
const char *version; /* PATCH: the API version named, NULL: any */
753756
const char *path; /* PATCH: the patch file, NULL: inline */
757+
const char *url; /* DEPLOY (kustomize): the chart repository */
758+
const char *release; /* DEPLOY (kustomize): the release name */
754759
int line;
755760
} CBMK8sFact;
756761

‎internal/cbm/extract_k8s.c‎

Lines changed: 66 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -453,6 +453,64 @@ static void emit_kustomize_patches(CBMExtractCtx *ctx, TSNode seq) {
453453
}
454454
}
455455

456+
// The `helmCharts` list of a kustomization: each chart it inflates into its
457+
// build is a DEPLOY fact (via "kustomize") the K8s pass draws to that chart;
458+
// its values file an import (local name "valuesFile").
459+
static void emit_kustomize_charts(CBMExtractCtx *ctx, TSNode seq) {
460+
CBMArena *a = ctx->arena;
461+
k8s_kids_t items = k8s_kids(seq);
462+
TSNode item;
463+
while (k8s_kid(&items, false, &item)) {
464+
TSNode map = strcmp(ts_node_type(item), "block_sequence_item") == 0
465+
? kust_item_mapping(item)
466+
: (TSNode){0};
467+
if (ts_node_is_null(map)) {
468+
continue;
469+
}
470+
CBMK8sFact f = {.fact = CBM_K8S_FACT_DEPLOY,
471+
.def_index = -1,
472+
.via = "kustomize",
473+
.edge = "DEPLOYS",
474+
.line = (int)ts_node_start_point(item).row + TS_LINE_OFFSET};
475+
k8s_kids_t pairs = k8s_kids(map);
476+
TSNode pair;
477+
while (k8s_kid(&pairs, false, &pair)) {
478+
if (strcmp(ts_node_type(pair), "block_mapping_pair") != 0) {
479+
continue;
480+
}
481+
const char *key = get_scalar_text(a, ts_node_named_child(pair, 0), ctx->source);
482+
TSNode v = unwrap_pair_value(pair);
483+
const char *val =
484+
key && !ts_node_is_null(v) ? get_scalar_text(a, v, ctx->source) : NULL;
485+
if (!val || !val[0]) {
486+
continue;
487+
}
488+
if (strcmp(key, "name") == 0) {
489+
f.name = val;
490+
} else if (strcmp(key, "repo") == 0) {
491+
f.url = val;
492+
} else if (strcmp(key, "version") == 0) {
493+
f.version = val;
494+
} else if (strcmp(key, "releaseName") == 0) {
495+
f.release = val;
496+
} else if (strcmp(key, "namespace") == 0) {
497+
f.ns = val;
498+
} else if (strcmp(key, "valuesFile") == 0) {
499+
f.path = val;
500+
}
501+
}
502+
if (!f.name) {
503+
continue;
504+
}
505+
if (f.path) {
506+
CBMImport imp = {.local_name = cbm_arena_strdup(a, "valuesFile"),
507+
.module_path = cbm_arena_strdup(a, f.path)};
508+
cbm_imports_push(&ctx->result->imports, a, imp);
509+
}
510+
cbm_k8sfact_push(&ctx->result->k8s_facts, a, f);
511+
}
512+
}
513+
456514
// Process a single block_mapping_pair for kustomize list keys.
457515
static void process_kustomize_pair(CBMExtractCtx *ctx, TSNode pair) {
458516
if (strcmp(ts_node_type(pair), "block_mapping_pair") != 0) {
@@ -463,14 +521,20 @@ static void process_kustomize_pair(CBMExtractCtx *ctx, TSNode pair) {
463521
return;
464522
}
465523
const char *key_text = get_scalar_text(ctx->arena, key_node, ctx->source);
466-
if (!key_text || !is_kustomize_list_key(key_text)) {
524+
if (!key_text) {
467525
return;
468526
}
469-
470527
TSNode val_node = unwrap_pair_value(pair);
471528
if (ts_node_is_null(val_node) || strcmp(ts_node_type(val_node), "block_sequence") != 0) {
472529
return;
473530
}
531+
if (strcmp(key_text, "helmCharts") == 0) {
532+
emit_kustomize_charts(ctx, val_node);
533+
return;
534+
}
535+
if (!is_kustomize_list_key(key_text)) {
536+
return;
537+
}
474538
emit_kustomize_sequence(ctx, val_node, key_text);
475539
if (strcmp(key_text, "patches") == 0) {
476540
emit_kustomize_patches(ctx, val_node);

‎src/pipeline/pass_k8s.c‎

Lines changed: 107 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1226,6 +1226,18 @@ typedef struct {
12261226
int line;
12271227
} k8s_patch_t;
12281228

1229+
/* A chart a kustomization inflates into its build (`helmCharts`). */
1230+
typedef struct {
1231+
char *kust; /* the kustomization file */
1232+
char *chart; /* the chart name */
1233+
char *url; /* its repository, NULL when unset */
1234+
char *version; /* NULL when unset */
1235+
char *release; /* the release name, NULL when unset */
1236+
char *ns; /* the release namespace, NULL when unset */
1237+
char *values; /* the values file (repository path), NULL when none */
1238+
int line;
1239+
} k8s_kchart_t;
1240+
12291241
typedef struct {
12301242
k8s_ref_t *refs;
12311243
int ref_n;
@@ -1251,6 +1263,9 @@ typedef struct {
12511263
k8s_patch_t *patches;
12521264
int patch_n;
12531265
int patch_cap;
1266+
k8s_kchart_t *kcharts;
1267+
int kchart_n;
1268+
int kchart_cap;
12541269
} k8s_facts_t;
12551270

12561271
static void k8s_facts_free(k8s_facts_t *f) {
@@ -1296,6 +1311,17 @@ static void k8s_facts_free(k8s_facts_t *f) {
12961311
cbm_free(CBM_MEM_CLASS_OTHER, p->ns);
12971312
}
12981313
cbm_free(CBM_MEM_CLASS_OTHER, f->patches);
1314+
for (int i = 0; i < f->kchart_n; i++) {
1315+
k8s_kchart_t *c = &f->kcharts[i];
1316+
cbm_free(CBM_MEM_CLASS_OTHER, c->kust);
1317+
cbm_free(CBM_MEM_CLASS_OTHER, c->chart);
1318+
cbm_free(CBM_MEM_CLASS_OTHER, c->url);
1319+
cbm_free(CBM_MEM_CLASS_OTHER, c->version);
1320+
cbm_free(CBM_MEM_CLASS_OTHER, c->release);
1321+
cbm_free(CBM_MEM_CLASS_OTHER, c->ns);
1322+
cbm_free(CBM_MEM_CLASS_OTHER, c->values);
1323+
}
1324+
cbm_free(CBM_MEM_CLASS_OTHER, f->kcharts);
12991325
cbm_free(CBM_MEM_CLASS_OTHER, f->envs);
13001326
cbm_free(CBM_MEM_CLASS_OTHER, f->hrs);
13011327
cbm_free(CBM_MEM_CLASS_OTHER, f->srcs);
@@ -3423,6 +3449,45 @@ static void k8s_link_chart_refs(k8s_dep_link_t *l, const k8s_hr_t *oci) {
34233449
}
34243450
}
34253451

3452+
/* `, "<key>":"<val>"` (escaped) at out + *w when `val` is set. */
3453+
static void k8s_prop_add(char *out, size_t out_sz, int *w, const char *key, const char *val) {
3454+
if (!val || *w < 0 || (size_t)*w >= out_sz) {
3455+
return;
3456+
}
3457+
char esc[CBM_SZ_1K];
3458+
cbm_json_escape(esc, (int)sizeof(esc), val);
3459+
*w += snprintf(out + *w, out_sz - (size_t)*w, ",\"%s\":\"%s\"", key, esc);
3460+
}
3461+
3462+
/* A chart a kustomization inflates (`helmCharts`) is deployed with its
3463+
* build: DEPLOYS {via: helmcharts, external, chart, url, version, release,
3464+
* namespace, values, line} from the kustomization (its Kustomization
3465+
* Resource, else its File -- the unit a Flux / Argo deploy reaches) to the
3466+
* shared external Chart of that name. */
3467+
static void k8s_link_kcharts(k8s_dep_link_t *l, const k8s_facts_t *f) {
3468+
for (int i = 0; i < f->kchart_n; i++) {
3469+
const k8s_kchart_t *c = &f->kcharts[i];
3470+
if (!c->chart || strchr(c->chart, '/') || strchr(c->chart, '$')) {
3471+
continue;
3472+
}
3473+
int64_t src = k8s_first_id(l->kusts, c->kust);
3474+
src = src > 0 ? src : k8s_first_id(l->files, c->kust);
3475+
char props[CBM_SZ_4K];
3476+
int w = snprintf(props, sizeof(props), "{\"via\":\"helmcharts\",\"external\":true");
3477+
k8s_prop_add(props, sizeof(props), &w, "chart", c->chart);
3478+
k8s_prop_add(props, sizeof(props), &w, "url", c->url);
3479+
k8s_prop_add(props, sizeof(props), &w, "version", c->version);
3480+
k8s_prop_add(props, sizeof(props), &w, "release", c->release);
3481+
k8s_prop_add(props, sizeof(props), &w, "namespace", c->ns);
3482+
k8s_prop_add(props, sizeof(props), &w, "values", c->values);
3483+
if (w < 0 || (size_t)w >= sizeof(props) - CBM_SZ_32) {
3484+
continue;
3485+
}
3486+
snprintf(props + w, sizeof(props) - (size_t)w, ",\"line\":%d}", c->line);
3487+
k8s_deploys_edge(l, src, k8s_external_chart(l, c->chart, c->kust), props);
3488+
}
3489+
}
3490+
34263491
/* A container image an object runs: USAGE {ref:image, container} to its
34273492
* Image node -- the node a Dockerfile building or building on that image
34283493
* joins; the container names whose environment runs it. */
@@ -3463,25 +3528,50 @@ static char *k8s_dup_opt(const char *s) {
34633528
return s ? cbm_mem_strdup(CBM_MEM_CLASS_OTHER, s) : NULL;
34643529
}
34653530

3466-
/* Keep a kustomization's PATCH facts, each patch file as a repository path
3467-
* (one that leaves the repository, or a URL, names no file of it). */
3468-
static void k8s_patches_keep(k8s_facts_t *f, const char *kust, const CBMFileResult *res) {
3531+
/* The repository path of `rel`, relative to directory `dir`; false when it
3532+
* leaves the repository (or is a URL). */
3533+
static bool k8s_repo_path(const char *dir, const char *rel, char *out, size_t out_sz) {
3534+
char joined[CBM_SZ_2K];
3535+
snprintf(joined, sizeof(joined), "%s%s%s", dir, dir[0] ? "/" : "", rel);
3536+
return k8s_norm_dir(joined, out, out_sz);
3537+
}
3538+
3539+
static void k8s_kchart_keep(k8s_facts_t *f, const char *kust, const CBMK8sFact *x,
3540+
const char *values) {
3541+
if (!k8s_grow((void **)&f->kcharts, &f->kchart_cap, f->kchart_n, sizeof(*f->kcharts))) {
3542+
return;
3543+
}
3544+
f->kcharts[f->kchart_n++] = (k8s_kchart_t){.kust = cbm_mem_strdup(CBM_MEM_CLASS_OTHER, kust),
3545+
.chart = k8s_dup_opt(x->name),
3546+
.url = k8s_dup_opt(x->url),
3547+
.version = k8s_dup_opt(x->version),
3548+
.release = k8s_dup_opt(x->release),
3549+
.ns = k8s_dup_opt(x->ns),
3550+
.values = k8s_dup_opt(values),
3551+
.line = x->line};
3552+
}
3553+
3554+
/* Keep a kustomization's facts: its patches (PATCH) and the charts it
3555+
* inflates (DEPLOY via "kustomize"), each file as a repository path -- a
3556+
* patch file outside the repository (or a URL) names no file of it, a values
3557+
* file there is left out. */
3558+
static void k8s_kustomize_keep(k8s_facts_t *f, const char *kust, const CBMFileResult *res) {
34693559
char dir[CBM_SZ_1K];
34703560
k8s_dirname(kust, dir, sizeof(dir));
34713561
for (int i = 0; res && i < res->k8s_facts.count; i++) {
34723562
const CBMK8sFact *x = &res->k8s_facts.items[i];
3473-
char path[CBM_SZ_1K];
3474-
if (x->fact != CBM_K8S_FACT_PATCH) {
3563+
bool chart = x->fact == CBM_K8S_FACT_DEPLOY && x->via && strcmp(x->via, "kustomize") == 0;
3564+
if (x->fact != CBM_K8S_FACT_PATCH && !chart) {
34753565
continue;
34763566
}
3477-
if (x->path) {
3478-
char joined[CBM_SZ_2K];
3479-
snprintf(joined, sizeof(joined), "%s%s%s", dir, dir[0] ? "/" : "", x->path);
3480-
if (!k8s_norm_dir(joined, path, sizeof(path))) {
3481-
continue;
3482-
}
3567+
char path[CBM_SZ_1K];
3568+
bool in_repo = x->path && k8s_repo_path(dir, x->path, path, sizeof(path));
3569+
if (chart) {
3570+
k8s_kchart_keep(f, kust, x, in_repo ? path : NULL);
3571+
continue;
34833572
}
3484-
if (!k8s_grow((void **)&f->patches, &f->patch_cap, f->patch_n, sizeof(*f->patches))) {
3573+
if ((x->path && !in_repo) ||
3574+
!k8s_grow((void **)&f->patches, &f->patch_cap, f->patch_n, sizeof(*f->patches))) {
34853575
continue;
34863576
}
34873577
f->patches[f->patch_n++] =
@@ -3767,7 +3857,7 @@ static void k8s_link_patches(cbm_pipeline_ctx_t *ctx, const k8s_kusts_t *ks, k8s
37673857

37683858
static void k8s_link_deploys(cbm_pipeline_ctx_t *ctx, k8s_kusts_t *ks, k8s_bundles_t *b,
37693859
const k8s_facts_t *f) {
3770-
if (f->dep_n == 0 && f->hr_n == 0 && f->src_n == 0) {
3860+
if (f->dep_n == 0 && f->hr_n == 0 && f->src_n == 0 && f->kchart_n == 0) {
37713861
return;
37723862
}
37733863
k8s_dep_link_t l = {.ctx = ctx,
@@ -3841,6 +3931,9 @@ static void k8s_link_deploys(cbm_pipeline_ctx_t *ctx, k8s_kusts_t *ks, k8s_bundl
38413931
for (int s = 0; l.files && l.charts && s < f->src_n; s++) {
38423932
k8s_link_chart_refs(&l, &f->srcs[s]);
38433933
}
3934+
if (l.files && l.kusts) {
3935+
k8s_link_kcharts(&l, f);
3936+
}
38443937
cbm_free(CBM_MEM_CLASS_OTHER, (void *)members.items);
38453938
cbm_free(CBM_MEM_CLASS_OTHER, (void *)units.items);
38463939
cbm_free(CBM_MEM_CLASS_OTHER, (void *)manifests.items);
@@ -3918,7 +4011,7 @@ int cbm_pipeline_pass_k8s(cbm_pipeline_ctx_t *ctx, const cbm_file_info_t *files,
39184011
case K8S_KUSTOMIZE:
39194012
k8s_kusts_push(&kusts, rel, k8s_kustomize_namespace(p->source),
39204013
k8s_kustomize_component(p->source));
3921-
k8s_patches_keep(&facts, rel, p->res);
4014+
k8s_kustomize_keep(&facts, rel, p->res);
39224015
break;
39234016
case K8S_MANIFEST:
39244017
handle_k8s_manifest(ctx, rel, p->source, p->res, &recs, &facts);

‎src/pipeline/pipeline_internal.h‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1497,7 +1497,8 @@ int cbm_pipeline_build_fresh_semantic_manifest(cbm_pipeline_t *p, const char *pr
14971497
* curl / wget HTTP_CALLS the Route of the URL it states (pass_shell.c).
14981498
* A kustomize patch OVERRIDEs the objects of its build it changes, its
14991499
* file an IMPORTS of the kustomization; a Resource keeps the version of
1500-
* its apiVersion (extract_k8s.c, pass_k8s.c). An index built before
1500+
* its apiVersion; a kustomization DEPLOYS the charts it inflates
1501+
* (helmCharts) (extract_k8s.c, pass_k8s.c). An index built before
15011502
* holds none of them, so it rebuilds once on upgrade. */
15021503
enum { CBM_SEMANTIC_INDEX_VERSION = 9 };
15031504

0 commit comments

Comments
 (0)