Skip to content

Commit 1669626

Browse files
committed
feat(infra): an object a kustomization renames has a variant Resource
Cluster API deploys its manager as `capi-controller-manager` in `capi-system`: config/default/kustomization.yaml puts `namePrefix: capi-` and `namespace: capi-system` on the `controller-manager` of config/manager/manager.yaml. The graph only had the declared name, so the name the cluster runs -- the one docs, scripts and tests quote -- matched no node. A kustomization with a namePrefix or nameSuffix now gives every object of its build a variant: a Resource named with the deployed name, in the namespace the build gives it, `ecosystem: kustomize`, which the kustomization's File DEFINES and which OVERRIDEs {via: namePrefix | nameSuffix, kustomization, line} the object. Prefixes of nested kustomizations compose outermost first, suffixes innermost first, and the outermost namespace wins. CustomResourceDefinitions, Namespaces and apiregistration APIServices keep their names, as kustomize's own transformers skip them. Variants are minted after the Kubernetes joins: inside a build, references use the declared names. Measured on 17 infrastructure repositories: 56 of 56 renamed objects in the 6 renaming kustomizations against an independent reading, with deployed name, namespace and base object; nothing else changed (56 nodes, 112 edges added, no property of an existing node); deterministic over three runs; the incremental re-index equals a full index when a prefix changes, goes, nests, or a renamed object is renamed or deleted. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent b1d93cc commit 1669626

4 files changed

Lines changed: 317 additions & 14 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -256,6 +256,7 @@ The install script placed beside the binary is **reported, not deleted** — uni
256256
- `DEPENDS_ON` — a Helm chart to the charts it declares (the local chart for a resolvable `file://` repository, otherwise an external `Chart`; the edge keeps `repository`), and a `go.mod` / `requirements.txt` file to an external `Package`; a Dockerfile to the `Image`s it builds on (`via: from`, `copy-from`); an `Image` to the Dockerfile it is built from (`via: build`; `tool`: `skaffold`, `compose`, `docker-build` for `docker` / `podman` / `nerdctl` / `buildah` build commands in scripts, Make and CI); a compose service to the services its `depends_on` / `links` name
257257
- `CONFIGURES` — code reading an environment variable to the shared `EnvVar` of that name (`strategy: env_access`); a Kubernetes container, compose service or Dockerfile (`ENV` of the image) setting one to that container's own `EnvVar` (`strategy: env_set`, with `source`: `value`, `configMapKeyRef`, `secretKeyRef`, `fieldRef`, ..., `compose`, `dockerfile` — values are never stored); and that container's `EnvVar` to the code reading the name inside the files its image is built from (`strategy: env_read`, with `dockerfile`) — the image's files are the sources its Dockerfile's `COPY` / `ADD` put into it, less the context's `.dockerignore`
258258
- `OVERRIDE` with `via: patch` — a Kustomize patch to each object of its kustomization's build it changes (child → base): a patch file's documents to the object of the same kind and name; an inline patch (from the kustomization) or a JSON6902 operation file to what its `target` selects (group, version, kind, name, namespace, label selector); a component's in the build of each kustomization including it. The edge keeps `kustomization`, `patch` and `line`; a patch file is an `IMPORTS` (`local_name: patches`) of its kustomization, never a member of the build
259+
- `OVERRIDE` with `via: namePrefix` / `nameSuffix` — an object a kustomization renames, from its variant: a `Resource` named with the deployed name (`capi-` + `controller-manager`; the prefixes of nested kustomizations outermost first, suffixes innermost first) in the namespace that build gives it, `ecosystem: kustomize`, which the kustomization's file `DEFINES`. CRDs, Namespaces and API services keep their names, as kustomize does. The edge keeps `kustomization` and `line`
259260
- `USAGE` with `via: terraform` — a Terraform / OpenTofu reference (`var.X`, `local.X`, `module.X`, `data.T.N`, `T.N`) from the top-level block holding it to the block of its module — the `.tf` / `.tofu` files of its directory — that declares it (a `locals` entry: its `locals` block); an entry of the block's `depends_on` list is a `DEPENDS_ON` instead. The edge keeps `ref` (`var`, `local`, `module`, `data`, `resource`) and `line`; a name only mentioned in string text is no reference. A module call with a local source (`module "net" { source = "./modules/net" }`) `IMPORTS` (`local_name`: the module's name) the `Folder` of that directory (the root directory: the `Project`), and `module.net.id` (also `module.net[0].id`, `module.net[*].id`) `USAGE`s (`ref: output`) the `output "id"` there, beside `module.net`
260261
- `OVERRIDE` with `via: tfvars` — a `.tfvars` assignment to a variable the module of its own directory declares, from the file's `Variable` to that `variable` block (the edge keeps `line`); a `.tfvars` file whose own directory declares none of its names sets the nearest ancestor module's that does, the edge marked `inferred` (such a file is applied by a `-var-file` the repository does not state)
261262
- Terragrunt (`via: terragrunt`) — a unit (`terragrunt.hcl`), evaluated in its own context: an `include` `IMPORTS` (`local_name`) the file its `path` names; a `read_terragrunt_config` `IMPORTS` (`read`: the file holding the call) the file it names, from the block holding the call or, for a call in an included file, from the unit's `include` block that brings it in; a `dependency` `DEPENDS_ON` the other unit; a local `terraform` `source` `IMPORTS` its `Folder`. Paths may use `find_in_parent_folders`, `dirname` and `get_terragrunt_dir`; anything else is unknown and links nothing

‎src/pipeline/pass_k8s.c‎

Lines changed: 214 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -733,6 +733,9 @@ typedef struct {
733733
int file_cap;
734734
bool referenced; /* included by another kustomization */
735735
bool component; /* `kind: Component`: patches the build including it */
736+
char *prefix; /* its `namePrefix:`, NULL when unset */
737+
char *suffix; /* its `nameSuffix:`, NULL when unset */
738+
int rename_line; /* the line of its namePrefix, else nameSuffix; 0: neither */
736739
} k8s_kust_t;
737740

738741
typedef struct {
@@ -776,25 +779,35 @@ static bool k8s_grow(void **items, int *cap, int count, size_t elem) {
776779
return true;
777780
}
778781

779-
/* The value of a kustomization's top-level `key:` line into `val`; false
780-
* when it has none, or an empty one. */
781-
static bool k8s_kustomize_top(const char *source, const char *key, char *val, size_t val_sz) {
782+
/* The value of a kustomization's top-level `key:` line into `val`, its
783+
* 1-based line into `*line` (when not NULL); false when it has none, or an
784+
* empty one. */
785+
static bool k8s_kustomize_top_at(const char *source, const char *key, char *val, size_t val_sz,
786+
int *line) {
782787
char head[CBM_SZ_64];
783788
int hl = snprintf(head, sizeof(head), "%s:", key);
784-
for (const char *p = source; p && *p && hl > 0;) {
789+
int at = SKIP_ONE;
790+
for (const char *p = source; p && *p && hl > 0; at++) {
785791
const char *eol = strchr(p, '\n');
786792
size_t len = eol ? (size_t)(eol - p) : strlen(p);
787793
if (strncmp(p, head, (size_t)hl) == 0) {
788-
char line[CBM_SZ_512];
789-
snprintf(line, sizeof(line), "%.*s", (int)len, p);
794+
char text[CBM_SZ_512];
795+
snprintf(text, sizeof(text), "%.*s", (int)len, p);
790796
char k[CBM_SZ_64];
791-
return k8s_split_kv(line, k, sizeof(k), val, val_sz) && val[0];
797+
if (line) {
798+
*line = at;
799+
}
800+
return k8s_split_kv(text, k, sizeof(k), val, val_sz) && val[0];
792801
}
793802
p = eol ? eol + SKIP_ONE : NULL;
794803
}
795804
return false;
796805
}
797806

807+
static bool k8s_kustomize_top(const char *source, const char *key, char *val, size_t val_sz) {
808+
return k8s_kustomize_top_at(source, key, val, val_sz, NULL);
809+
}
810+
798811
/* The top-level `namespace:` of a kustomization, NULL when it sets none. */
799812
static char *k8s_kustomize_namespace(const char *source) {
800813
char val[K8S_LABEL_LEN];
@@ -810,19 +823,39 @@ static bool k8s_kustomize_component(const char *source) {
810823
return k8s_kustomize_top(source, "kind", val, sizeof(val)) && strcmp(val, "Component") == 0;
811824
}
812825

813-
static void k8s_kusts_push(k8s_kusts_t *ks, const char *path, char *ns, bool component) {
826+
/* A kustomization's `namePrefix:` / `nameSuffix:` (`key`), NULL when it
827+
* sets none -- or one with `/` or a blank in it, which is no name part; its
828+
* line into `*line` when that is still 0. */
829+
static char *k8s_kustomize_name_part(const char *source, const char *key, int *line) {
830+
char val[K8S_LABEL_LEN];
831+
int at = 0;
832+
if (!k8s_kustomize_top_at(source, key, val, sizeof(val), &at) || strpbrk(val, "/ \t")) {
833+
return NULL;
834+
}
835+
if (*line == 0) {
836+
*line = at;
837+
}
838+
return cbm_mem_strdup(CBM_MEM_CLASS_OTHER, val);
839+
}
840+
841+
static void k8s_kusts_push(k8s_kusts_t *ks, const char *path, const char *source) {
814842
if (!k8s_grow((void **)&ks->items, &ks->cap, ks->count, sizeof(*ks->items))) {
815-
cbm_free(CBM_MEM_CLASS_OTHER, ns);
816843
return;
817844
}
818-
ks->items[ks->count++] = (k8s_kust_t){
819-
.path = cbm_mem_strdup(CBM_MEM_CLASS_OTHER, path), .ns = ns, .component = component};
845+
k8s_kust_t k = {.path = cbm_mem_strdup(CBM_MEM_CLASS_OTHER, path),
846+
.ns = k8s_kustomize_namespace(source),
847+
.component = k8s_kustomize_component(source)};
848+
k.prefix = k8s_kustomize_name_part(source, "namePrefix", &k.rename_line);
849+
k.suffix = k8s_kustomize_name_part(source, "nameSuffix", &k.rename_line);
850+
ks->items[ks->count++] = k;
820851
}
821852

822853
static void k8s_kusts_free(k8s_kusts_t *ks) {
823854
for (int i = 0; i < ks->count; i++) {
824855
cbm_free(CBM_MEM_CLASS_OTHER, ks->items[i].path);
825856
cbm_free(CBM_MEM_CLASS_OTHER, ks->items[i].ns);
857+
cbm_free(CBM_MEM_CLASS_OTHER, ks->items[i].prefix);
858+
cbm_free(CBM_MEM_CLASS_OTHER, ks->items[i].suffix);
826859
cbm_free(CBM_MEM_CLASS_OTHER, ks->items[i].kids);
827860
cbm_free(CBM_MEM_CLASS_OTHER, (void *)ks->items[i].files);
828861
}
@@ -3960,6 +3993,174 @@ static void k8s_link_patches(cbm_pipeline_ctx_t *ctx, const k8s_kusts_t *ks, k8s
39603993
}
39613994
}
39623995

3996+
/* kustomize's prefix / suffix transformers leave these names alone
3997+
* (api/internal/builtins/PrefixTransformer.go prefixFieldSpecsToSkip, the
3998+
* same list in SuffixTransformer.go). */
3999+
static bool k8s_rename_skips(const char *group, const char *kind) {
4000+
return strcmp(kind, "CustomResourceDefinition") == 0 || strcmp(kind, "Namespace") == 0 ||
4001+
(strcmp(kind, "APIService") == 0 && strcmp(group, "apiregistration.k8s.io") == 0);
4002+
}
4003+
4004+
/* A kustomization on the way down a renaming build: the prefix and suffix
4005+
* the levels above and its own put around its objects' names, and the
4006+
* namespace the outermost level setting one gives them (NULL: their own). */
4007+
typedef struct {
4008+
int kust;
4009+
const char *ns;
4010+
char pre[K8S_LABEL_LEN];
4011+
char suf[K8S_LABEL_LEN];
4012+
} k8s_rename_frame_t;
4013+
4014+
/* The variant of object `obj` the renaming build of `root` deploys: a
4015+
* Resource `Kind/<deployed name>` declared by the kustomization (`base`:
4016+
* its File QN without `.__file__`), ecosystem kustomize; the kustomization
4017+
* DEFINES it and it OVERRIDEs {via: namePrefix | nameSuffix, kustomization,
4018+
* line} the object. */
4019+
static void k8s_rename_variant(cbm_pipeline_ctx_t *ctx, const k8s_kust_t *root, const char *base,
4020+
int64_t kust_file, const k8s_rename_frame_t *f,
4021+
const cbm_gbuf_node_t *obj) {
4022+
char kind[K8S_LABEL_LEN];
4023+
char group[K8S_LABEL_LEN];
4024+
char version[K8S_LABEL_LEN];
4025+
char declared[K8S_LABEL_LEN];
4026+
const char *name = NULL;
4027+
if (!k8s_kind_name(obj, kind, sizeof(kind), &name)) {
4028+
return;
4029+
}
4030+
k8s_node_prop(obj, "group", group, sizeof(group));
4031+
if (!group[0] || k8s_rename_skips(group, kind)) {
4032+
return;
4033+
}
4034+
k8s_node_prop(obj, "version", version, sizeof(version));
4035+
k8s_declared_ns(obj->qualified_name, declared, sizeof(declared));
4036+
const char *ns = k8s_cluster_scoped(kind) || !f->ns ? declared : f->ns;
4037+
char deployed[K8S_LABEL_LEN];
4038+
int dl = snprintf(deployed, sizeof(deployed), "%s%s%s", f->pre, name, f->suf);
4039+
char qn[CBM_SZ_2K];
4040+
char rname[CBM_SZ_256];
4041+
if (dl <= 0 || (size_t)dl >= sizeof(deployed) ||
4042+
snprintf(qn, sizeof(qn), "%s::%s/%s/%s/%s", base, group, kind, ns[0] ? ns : "-",
4043+
deployed) >= (int)sizeof(qn) ||
4044+
snprintf(rname, sizeof(rname), "%s/%s", kind, deployed) >= (int)sizeof(rname)) {
4045+
return;
4046+
}
4047+
char *props = k8s_resource_props(qn, NULL, version, NULL, "kustomize");
4048+
int64_t id = props ? cbm_gbuf_upsert_node(ctx->gbuf, "Resource", rname, qn, root->path,
4049+
root->rename_line, root->rename_line, props)
4050+
: 0;
4051+
cbm_free(CBM_MEM_CLASS_OTHER, props);
4052+
if (id <= 0 || id == obj->id) {
4053+
return;
4054+
}
4055+
if (kust_file > 0) {
4056+
cbm_gbuf_insert_edge(ctx->gbuf, kust_file, id, "DEFINES", "{}");
4057+
}
4058+
char ek[CBM_SZ_2K];
4059+
cbm_json_escape(ek, (int)sizeof(ek), root->path);
4060+
char eprops[CBM_SZ_2K + CBM_SZ_256];
4061+
snprintf(eprops, sizeof(eprops), "{\"via\":\"%s\",\"kustomization\":\"%s\",\"line\":%d}",
4062+
root->prefix ? "namePrefix" : "nameSuffix", ek, root->rename_line);
4063+
cbm_gbuf_insert_edge(ctx->gbuf, id, obj->id, "OVERRIDE", eprops);
4064+
}
4065+
4066+
/* Every object the build of renaming kustomization `root` reaches gets its
4067+
* variant: down the kustomizations it includes, each level's prefix inside
4068+
* the ones above, its namespace where none above sets one (a component's
4069+
* own transforms are not followed). `by_file`: Resource ids by manifest. */
4070+
static void k8s_rename_build(cbm_pipeline_ctx_t *ctx, const k8s_kusts_t *ks, int root,
4071+
CBMHashTable *by_file, int *seen, int mark) {
4072+
static const char FILE_SUFFIX[] = ".__file__";
4073+
const k8s_kust_t *r = &ks->items[root];
4074+
const cbm_gbuf_node_t *kf = cbm_pipeline_file_node(ctx->gbuf, ctx->project_name, r->path);
4075+
size_t ql = kf && kf->qualified_name ? strlen(kf->qualified_name) : 0;
4076+
size_t sl = sizeof(FILE_SUFFIX) - SKIP_ONE;
4077+
char base[CBM_SZ_1K];
4078+
if (ql <= sl || strcmp(kf->qualified_name + ql - sl, FILE_SUFFIX) != 0 ||
4079+
ql - sl >= sizeof(base)) {
4080+
return;
4081+
}
4082+
snprintf(base, sizeof(base), "%.*s", (int)(ql - sl), kf->qualified_name);
4083+
k8s_rename_frame_t *stack = NULL;
4084+
int depth = 0;
4085+
int cap = 0;
4086+
if (!k8s_grow((void **)&stack, &cap, depth, sizeof(*stack))) {
4087+
return;
4088+
}
4089+
stack[depth] = (k8s_rename_frame_t){.kust = root, .ns = r->ns};
4090+
snprintf(stack[depth].pre, sizeof(stack[depth].pre), "%s", r->prefix ? r->prefix : "");
4091+
snprintf(stack[depth].suf, sizeof(stack[depth].suf), "%s", r->suffix ? r->suffix : "");
4092+
depth++;
4093+
while (depth > 0) {
4094+
k8s_rename_frame_t f = stack[--depth];
4095+
if (seen[f.kust] == mark) {
4096+
continue;
4097+
}
4098+
seen[f.kust] = mark;
4099+
const k8s_kust_t *k = &ks->items[f.kust];
4100+
for (int i = 0; i < k->file_n; i++) {
4101+
const k8s_ids_t *objs = cbm_ht_get(by_file, k->files[i]);
4102+
for (int o = 0; objs && o < objs->count; o++) {
4103+
const cbm_gbuf_node_t *n = cbm_gbuf_find_by_id(ctx->gbuf, objs->items[o]);
4104+
if (n) {
4105+
k8s_rename_variant(ctx, r, base, kf->id, &f, n);
4106+
}
4107+
}
4108+
}
4109+
for (int i = 0; i < k->kid_n; i++) {
4110+
const k8s_kust_t *kid = &ks->items[k->kids[i]];
4111+
const char *kp = kid->component || !kid->prefix ? "" : kid->prefix;
4112+
const char *kx = kid->component || !kid->suffix ? "" : kid->suffix;
4113+
if (!k8s_grow((void **)&stack, &cap, depth, sizeof(*stack))) {
4114+
break;
4115+
}
4116+
k8s_rename_frame_t *c = &stack[depth];
4117+
*c = (k8s_rename_frame_t){.kust = k->kids[i],
4118+
.ns = f.ns || kid->component ? f.ns : kid->ns};
4119+
if (snprintf(c->pre, sizeof(c->pre), "%s%s", f.pre, kp) < (int)sizeof(c->pre) &&
4120+
snprintf(c->suf, sizeof(c->suf), "%s%s", kx, f.suf) < (int)sizeof(c->suf)) {
4121+
depth++;
4122+
}
4123+
}
4124+
}
4125+
cbm_free(CBM_MEM_CLASS_OTHER, stack);
4126+
}
4127+
4128+
/* Kustomize namePrefix / nameSuffix -> a variant Resource per object a
4129+
* kustomization renames, with its deployed name, OVERRIDE -> the object.
4130+
* Minted after the joins: references inside a build use the declared names
4131+
* (kustomize rewrites them), so nothing joins a variant. */
4132+
static void k8s_link_renames(cbm_pipeline_ctx_t *ctx, const k8s_kusts_t *ks) {
4133+
bool any = false;
4134+
for (int i = 0; i < ks->count && !any; i++) {
4135+
any = !ks->items[i].component && (ks->items[i].prefix || ks->items[i].suffix);
4136+
}
4137+
if (!any) {
4138+
return;
4139+
}
4140+
CBMHashTable *by_file = cbm_ht_create(0);
4141+
int *seen = cbm_calloc(CBM_MEM_CLASS_OTHER, (size_t)ks->count * sizeof(int));
4142+
const cbm_gbuf_node_t **res = NULL;
4143+
int res_n = 0;
4144+
if (by_file && seen && cbm_gbuf_find_by_label(ctx->gbuf, "Resource", &res, &res_n) == 0) {
4145+
for (int i = 0; i < res_n; i++) {
4146+
if (res[i]->file_path) {
4147+
k8s_index_add(by_file, res[i]->file_path, res[i]->id);
4148+
}
4149+
}
4150+
int mark = 0;
4151+
for (int i = 0; i < ks->count; i++) {
4152+
if (!ks->items[i].component && (ks->items[i].prefix || ks->items[i].suffix)) {
4153+
k8s_rename_build(ctx, ks, i, by_file, seen, ++mark);
4154+
}
4155+
}
4156+
}
4157+
cbm_free(CBM_MEM_CLASS_OTHER, seen);
4158+
if (by_file) {
4159+
cbm_ht_foreach(by_file, k8s_free_ids_entry, NULL);
4160+
cbm_ht_free(by_file);
4161+
}
4162+
}
4163+
39634164
static void k8s_link_deploys(cbm_pipeline_ctx_t *ctx, k8s_kusts_t *ks, k8s_bundles_t *b,
39644165
const k8s_facts_t *f) {
39654166
if (f->dep_n == 0 && f->hr_n == 0 && f->src_n == 0 && f->kchart_n == 0) {
@@ -4114,8 +4315,7 @@ int cbm_pipeline_pass_k8s(cbm_pipeline_ctx_t *ctx, const cbm_file_info_t *files,
41144315
helm_count++;
41154316
break;
41164317
case K8S_KUSTOMIZE:
4117-
k8s_kusts_push(&kusts, rel, k8s_kustomize_namespace(p->source),
4118-
k8s_kustomize_component(p->source));
4318+
k8s_kusts_push(&kusts, rel, p->source);
41194319
k8s_kustomize_keep(&facts, rel, p->res);
41204320
break;
41214321
case K8S_MANIFEST:
@@ -4160,6 +4360,7 @@ int cbm_pipeline_pass_k8s(cbm_pipeline_ctx_t *ctx, const cbm_file_info_t *files,
41604360
k8s_link_images(ctx, &facts);
41614361
k8s_link_envs(ctx, &facts);
41624362
k8s_link_patches(ctx, &kusts, &facts);
4363+
k8s_link_renames(ctx, &kusts);
41634364
k8s_facts_free(&facts);
41644365
k8s_records_free(&recs);
41654366
/* Link `file://` chart dependencies now that every Chart.yaml is read,

‎src/pipeline/pipeline_internal.h‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1521,7 +1521,10 @@ int cbm_pipeline_build_fresh_semantic_manifest(cbm_pipeline_t *p, const char *pr
15211521
* A kustomize patch OVERRIDEs the objects of its build it changes, its
15221522
* file an IMPORTS of the kustomization; a Resource keeps the version of
15231523
* its apiVersion and its labels; a kustomization DEPLOYS the charts it inflates
1524-
* (helmCharts) (extract_k8s.c, pass_k8s.c). A Terraform reference
1524+
* (helmCharts); an object a namePrefix / nameSuffix renames has a
1525+
* variant Resource {ecosystem:kustomize} with its deployed name, which
1526+
* OVERRIDEs {via:namePrefix|nameSuffix} it (extract_k8s.c,
1527+
* pass_k8s.c). A Terraform reference
15251528
* USAGEs {ref, via:terraform}, from its block, the block of its module
15261529
* (directory) declaring it (one in depends_on: DEPENDS_ON), instead of
15271530
* the generic name matcher's USAGE / CALLS; a bare block label names

0 commit comments

Comments
 (0)