Skip to content

Commit eca1a00

Browse files
committed
feat(configlink): TOML and INI keys link by exact name in their component
A config key linked to every code symbol whose name contained it. In YAML, JSON and HCL files that guess was retired in WP0.9; TOML and INI files kept it, and in 17 infrastructure repositories every one of their 72 key_symbol edges was a coincidence: `.gitleaks.toml useDefault` linked to TestConfig_UseDefault and three other symbols, a toml-merge tool's test data `config_path` to 46 `*ConfigPath` constants all over deckhouse, `grafana.ini admin_user` to user-cluster constants, a Rust Cargo.toml dependency to other services' go.mod requirements. A .toml, .ini, .cfg or .conf key now links like an infrastructure key: by exact name (camelCase and snake_case still normalize to one key) and only to code inside the config file's component, its directory or the one around a config/, conf/, settings/ or etc/ directory. .env and .properties keys keep substring matching; the tests that covered it on a TOML key now use a .properties key. Measured on the 17 repositories: key_symbol 95 -> 23, the 23 being the infrastructure edges WP0.9 reviewed, the 72 removed all TOML/INI and all coincidences, no other edge or node changed; deterministic over three runs; the incremental re-index equals a full index for INI, TOML, code and tfvars edits. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent dec56c9 commit eca1a00

3 files changed

Lines changed: 70 additions & 18 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -336,6 +336,7 @@ The install script placed beside the binary is **reported, not deleted**: uninst
336336
- `DEPLOYS`: what puts manifests, kustomizations or charts on a cluster, to what it puts there: a Flux Kustomization or Argo Application to the kustomizations / manifests / chart of its path, a Flux HelmRelease to its chart (`spec.chart` or `chartRef` → OCIRepository), a shell command or Make recipe (`kubectl apply -f/-k`, `kustomize build | kubectl apply`, `helm install/upgrade/template`), a Go installer loading a chart with the Helm SDK, a Terraform `helm_release`, a Terraform `kubernetes_*` resource the object it declares (a `Resource` like a manifest's, `ecosystem: terraform`, when its name is a literal), a kustomization inflating a chart (`helmCharts`; its values file an `IMPORTS`). A chart outside the repository is an external `Chart`, keyed by its publisher when its repository URL is stated (`__chart__<repository>/<name>`, the same node in every project, the URL in `repository`), else one per project and name; the edge says `via` (`flux`, `argo`, `helmrelease`, `kubectl`, `kustomize`, `helm`, `helm-sdk`, `terraform`, `helmcharts`, ...)
337337
- `DEPENDS_ON`: a Helm chart to the charts it declares (the local chart for a resolvable `file://` repository, otherwise an external `Chart`; the edge keeps `repository`), and a `go.mod` / `requirements.txt` file to an external `Package`; a Dockerfile to the `Image`s it builds on (`via: from`, `copy-from`); an `Image` to the Dockerfile it is built from (`via: build`; `tool`: `skaffold`, `compose`, `docker-build` for `docker` / `podman` / `nerdctl` / `buildah` build commands in scripts, Make and CI); a compose service to the services its `depends_on` / `links` name
338338
- `CONFIGURES`: code reading an environment variable to the shared `EnvVar` of that name (`strategy: env_access`); a Kubernetes container, compose service or Dockerfile (`ENV` of the image) setting one to that container's own `EnvVar` (`strategy: env_set`, with `source`: `value`, `configMapKeyRef`, `secretKeyRef`, `fieldRef`, ..., `compose`, `dockerfile`; values are never stored); and that container's `EnvVar` to the code reading the name inside the files its image is built from (`strategy: env_read`, with `dockerfile`): the image's files are the sources its Dockerfile's `COPY` / `ADD` put into it, less the context's `.dockerignore`
339+
- `CONFIGURES` with `strategy: key_symbol`: a code symbol to the config key it names (`config_key`). A YAML, JSON, HCL (Terraform, OpenTofu, tfvars), TOML or INI key links by exact name (`maxRetryCount` = `max_retry_count`) and only to code inside the config file's component: its directory, or the one around a `config/`, `configs/`, `conf/`, `settings/` or `etc/` directory. A `.env` or `.properties` key also links by substring (confidence 0.75)
339340
- `OVERRIDE` with `via: patch`: a Kustomize patch to each object of its kustomization's build it changes (child → base): a patch file's documents to the object of the same kind and name; an inline patch (from the kustomization) or a JSON6902 operation file to what its `target` selects (group, version, kind, name, namespace, label selector); a component's in the build of each kustomization including it. The edge keeps `kustomization`, `patch` and `line`; a patch file is an `IMPORTS` (`local_name: patches`) of its kustomization, never a member of the build
340341
- `OVERRIDE` with `via: namePrefix` / `nameSuffix`: an object a kustomization renames, from its variant: a `Resource` named with the deployed name (`capi-` + `controller-manager`; the prefixes of nested kustomizations outermost first, suffixes innermost first) in the namespace that build gives it, `ecosystem: kustomize`, which the kustomization's file `DEFINES`. CRDs, Namespaces and API services keep their names, as kustomize does. The edge keeps `kustomization` and `line`
341342
- `USAGE` with `via: terraform`: a Terraform / OpenTofu reference (`var.X`, `local.X`, `module.X`, `data.T.N`, `T.N`) from the top-level block holding it to the block of its module (the `.tf` / `.tofu` files of its directory) that declares it (a `locals` entry: its `locals` block); an entry of the block's `depends_on` list is a `DEPENDS_ON` instead. The edge keeps `ref` (`var`, `local`, `module`, `data`, `resource`) and `line`; a name only mentioned in string text is no reference. A module call with a local source (`module "net" { source = "./modules/net" }`) `IMPORTS` (`local_name`: the module's name) the `Folder` of that directory (the root directory: the `Project`), and `module.net.id` (also `module.net[0].id`, `module.net[*].id`) `USAGE`s (`ref: output`) the `output "id"` there, beside `module.net`

‎src/pipeline/pass_configlink.c‎

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -116,15 +116,18 @@ static const cbm_gbuf_node_t **canonical_node_copy(const cbm_gbuf_node_t *const
116116
return sorted;
117117
}
118118

119-
/* Infrastructure files: YAML, JSON and HCL (Terraform, OpenTofu, tfvars). A
120-
* key_symbol pair with either side in one links by exact name and only inside
121-
* the config file's component (user decision 2026-10-10): in 17 infra
122-
* repositories 4 of 40 sampled key_symbol edges were right, all of them exact
123-
* and within one directory tree; the rest were substring and cross-component
124-
* coincidences. Other config formats keep exact + substring matching. */
119+
/* Files whose key_symbol pairs link by exact name and only inside the config
120+
* file's component: infrastructure files -- YAML, JSON, HCL (Terraform,
121+
* OpenTofu, tfvars) -- (user decision 2026-10-10: in 17 infra repositories 4
122+
* of 40 sampled key_symbol edges were right, all of them exact and within one
123+
* directory tree; the rest were substring and cross-component coincidences),
124+
* and TOML and INI files (user decision 2026-10-14: all 72 of their substring
125+
* or cross-component edges in those repositories were coincidences, such as
126+
* `.gitleaks.toml useDefault` <- TestConfig_UseDefault). Other config formats
127+
* (.env, .properties) keep exact + substring matching. */
125128
static bool cl_is_infra_path(const char *path) {
126-
static const char *const exts[] = {".yaml", ".yml", ".json", ".tf",
127-
".hcl", ".tofu", ".tfvars", NULL};
129+
static const char *const exts[] = {".yaml", ".yml", ".json", ".tf", ".hcl", ".tofu",
130+
".tfvars", ".toml", ".ini", ".cfg", ".conf", NULL};
128131
const char *dot = path ? strrchr(path, '.') : NULL;
129132
const char *slash = path ? strrchr(path, '/') : NULL;
130133
if (!dot || (slash && dot < slash)) {

‎tests/test_configlink.c‎

Lines changed: 58 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,8 @@ static void rm_rf(const char *path) {
8585
/* ── Strategy 1: Config Key → Code Symbol ───────────────────────── */
8686

8787
/* Go: TestConfigKeySymbol_ExactMatch
88-
* config.toml has max_connections, main.go has getMaxConnections() */
88+
* config.toml has max_connections, main.go has maxConnections(). (A TOML key
89+
* links by exact name only, see configlink_key_symbol_toml_ini_exact_scoped.) */
8990
TEST(configlink_key_symbol_exact_match) {
9091
cbm_gbuf_t *gb = cbm_gbuf_new("test", "/tmp/test");
9192

@@ -94,9 +95,9 @@ TEST(configlink_key_symbol_exact_match) {
9495
"test.config.max_connections", "config.toml", 0, 0, NULL);
9596
ASSERT_GT(cfg_id, 0);
9697

97-
/* Code Function node: getMaxConnections from main.go */
98-
int64_t func_id = cbm_gbuf_upsert_node(gb, "Function", "getMaxConnections",
99-
"test.main.getMaxConnections", "main.go", 0, 0, NULL);
98+
/* Code Function node: maxConnections from main.go */
99+
int64_t func_id = cbm_gbuf_upsert_node(gb, "Function", "maxConnections",
100+
"test.main.maxConnections", "main.go", 0, 0, NULL);
100101
ASSERT_GT(func_id, 0);
101102

102103
/* Run configlink */
@@ -113,12 +114,14 @@ TEST(configlink_key_symbol_exact_match) {
113114
}
114115

115116
/* Go: TestConfigKeySymbol_SubstringMatch
116-
* config.toml has request_timeout, main.go has getRequestTimeoutSeconds() */
117+
* app.properties has request_timeout, main.go has getRequestTimeoutSeconds().
118+
* (Formats outside the exact + component rule -- .properties, .env -- keep
119+
* substring matching.) */
117120
TEST(configlink_key_symbol_substring_match) {
118121
cbm_gbuf_t *gb = cbm_gbuf_new("test", "/tmp/test");
119122

120-
cbm_gbuf_upsert_node(gb, "Variable", "request_timeout", "test.config.request_timeout",
121-
"config.toml", 0, 0, NULL);
123+
cbm_gbuf_upsert_node(gb, "Variable", "request_timeout", "test.app.request_timeout",
124+
"app.properties", 0, 0, NULL);
122125

123126
cbm_gbuf_upsert_node(gb, "Function", "getRequestTimeoutSeconds",
124127
"test.main.getRequestTimeoutSeconds", "main.go", 0, 0, NULL);
@@ -281,6 +284,48 @@ TEST(configlink_key_symbol_terraform_variable) {
281284
PASS();
282285
}
283286

287+
/* TOML and INI keys (.toml, .ini, .cfg, .conf) link like infrastructure keys:
288+
* by exact name and only inside the config file's component. In 17 infra
289+
* repositories all 72 TOML/INI key_symbol edges were substring or
290+
* cross-component coincidences (`.gitleaks.toml useDefault` <-
291+
* TestConfig_UseDefault, `grafana.ini admin_user` <- GenAdminUser). Each
292+
* format's key below reaches only svc-a's exact `requestTimeout`: not svc-a's
293+
* substring `getRequestTimeoutSeconds`, not svc-b's exact `requestTimeout`. */
294+
TEST(configlink_key_symbol_toml_ini_exact_scoped) {
295+
static const char *const files[] = {"svc-a/config.toml", "svc-a/settings.ini",
296+
"svc-a/setup.cfg", "svc-a/app.conf"};
297+
enum { FILES = 4 };
298+
cbm_gbuf_t *gb = cbm_gbuf_new("test", "/tmp/test");
299+
int64_t cfg[FILES];
300+
for (int i = 0; i < FILES; i++) {
301+
char qn[CBM_SZ_128];
302+
snprintf(qn, sizeof(qn), "test.cfg%d.request_timeout", i);
303+
cfg[i] = cbm_gbuf_upsert_node(gb, "Variable", "request_timeout", qn, files[i], 0, 0, NULL);
304+
}
305+
int64_t inside = cbm_gbuf_upsert_node(gb, "Function", "requestTimeout",
306+
"test.svc_a.main.requestTimeout", "svc-a/main.go", 0, 0,
307+
NULL);
308+
cbm_gbuf_upsert_node(gb, "Function", "getRequestTimeoutSeconds",
309+
"test.svc_a.client.getRequestTimeoutSeconds", "svc-a/client.go", 0, 0,
310+
NULL);
311+
cbm_gbuf_upsert_node(gb, "Function", "requestTimeout", "test.svc_b.main.requestTimeout",
312+
"svc-b/main.go", 0, 0, NULL);
313+
run_configlink(gb, "test", NULL);
314+
const cbm_gbuf_edge_t **edges = NULL;
315+
int count = 0;
316+
cbm_gbuf_find_edges_by_type(gb, "CONFIGURES", &edges, &count);
317+
ASSERT_EQ(count, FILES);
318+
for (int i = 0; i < FILES; i++) {
319+
bool found = false;
320+
for (int e = 0; e < count; e++) {
321+
found = found || (edges[e]->source_id == inside && edges[e]->target_id == cfg[i]);
322+
}
323+
ASSERT_TRUE(found);
324+
}
325+
cbm_gbuf_free(gb);
326+
PASS();
327+
}
328+
284329
/* No count of config keys decides what links: the collector stopped at 4,096. */
285330
TEST(configlink_key_symbol_no_config_cap) {
286331
enum { KEYS = 4200 };
@@ -469,9 +514,11 @@ enum { CL_DET_CANDIDATES = 9000 }; /* > the 8192 code-entry cap */
469514
static cbm_gbuf_t *build_oversized_corpus(bool reverse) {
470515
cbm_gbuf_t *gb = cbm_gbuf_new("test", "/tmp/test");
471516

472-
/* One config key every candidate can match on: "max_connections". */
473-
cbm_gbuf_upsert_node(gb, "Variable", "max_connections", "test.config.max_connections",
474-
"config.toml", 0, 0, NULL);
517+
/* One config key every candidate can match on by substring:
518+
* "max_connections", in a format that keeps substring matching (TOML and
519+
* INI keys link by exact name only). */
520+
cbm_gbuf_upsert_node(gb, "Variable", "max_connections", "test.app.max_connections",
521+
"app.properties", 0, 0, NULL);
475522

476523
for (int k = 0; k < CL_DET_CANDIDATES; k++) {
477524
int i = reverse ? (CL_DET_CANDIDATES - 1 - k) : k;
@@ -529,6 +576,7 @@ SUITE(configlink) {
529576
RUN_TEST(configlink_key_symbol_config_dir_component);
530577
RUN_TEST(configlink_key_symbol_no_config_cap);
531578
RUN_TEST(configlink_key_symbol_terraform_variable);
579+
RUN_TEST(configlink_key_symbol_toml_ini_exact_scoped);
532580

533581
/* Strategy 2: Dependency → Import */
534582
RUN_TEST(configlink_dep_import_package_json);

0 commit comments

Comments
 (0)