Skip to content

Commit bd928b3

Browse files
committed
feat(infra): CONNECTS_TO -- a service address in a container's environment
A workload's environment names the services it talks to (PRODUCT_CATALOG_SERVICE_ADDR: productcatalogservice:3550, a URL, a <svc>.<ns>.svc name); the graph had no edge for it, so the service topology of an application was invisible. A Kubernetes workload or a compose service now CONNECTS_TO the Service its containers' environment addresses, with {via: env, key, container} -- never the value: - the address: the host of a URL or of host:port, a <svc>.<ns>.svc name in the value, or the whole value when the key names an address (*_ADDR, *_ADDRESS, *_HOST, *_URL, *_ENDPOINT, *_URI) and it is a bare DNS name; a templated value, localhost or an IP addresses nothing, and only service DNS forms count (svc, svc.ns, svc.ns.svc...), never another domain's host (cbm_env_address, shared by both sides); - Kubernetes: a reference resolved like every other one -- a Service of the same bundle, in the named namespace or the object's own; - compose: a service of the same file; a variable without a value takes the project's .env, whose values compose expands against the lines before them (CART_ADDR=cart:${CART_PORT}) -- the dotenv reader does that now too. Consumers: trace_path's topology mode, the CLI's edge list and the README (which also gains DEPLOYS and the env join of the previous commit). Measured on 17 infrastructure repositories: 129 edges, every one correct by hand; an independent reading of the values finds 133 pairs repo-wide, the 5 without an edge belong to another bundle or namespace. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent 6316733 commit bd928b3

12 files changed

Lines changed: 402 additions & 27 deletions

File tree

‎README.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -250,10 +250,11 @@ The install script placed beside the binary is **reported, not deleted** — uni
250250
- `HTTP_CALLS`, `ASYNC_CALLS` (cross-service)
251251
- `SPAWNS` — the caller starts another program (`subprocess.run`, `exec.Command`, `posix_spawn`, `Process.Start`, a shell command or Make recipe running a program by path, ...); the target is a `Process` node named after the program, or `<dynamic>` when the program is not a literal. When a shell script, Make recipe, GitHub Actions step or Prow job runs a script of the repository (`./hack/x.sh`, `bash hack/x.sh`), the target is that script's `Module` (else its `File`); `make TARGET` (also `$(MAKE)`, `-C`, `-f`) targets the rule of that target, followed through included makefiles; edges from CI carry `ci` and `job`. A Dockerfile's last stage `ENTRYPOINT` (else `CMD`) spawns what the container runs (`api: dockerfile`): the repository script or source copied into the image, the Go `main` of a binary a stage builds with `go build -o`, else a `Process`
252252
- `INFRA_MAPS` — a Kubernetes Service to each workload whose pod labels carry every `key: value` of the Service's selector, within one namespace
253+
- `CONNECTS_TO` — a Kubernetes workload or compose service to the Service its containers' environment addresses (`PRODUCT_CATALOG_SERVICE_ADDR: productcatalogservice:3550`, a URL's host, `<svc>.<ns>.svc...`, a bare name on an `*_ADDR` / `*_HOST` / `*_URL` / ... key; compose: the project's `.env` interpolated): a Kubernetes Service of the same bundle and namespace, or a compose service of the same file. The edge keeps `via: env`, `key` and `container` — never the value
253254
- `USAGE` with `ref: image` — a Kubernetes workload (any pod template: containers, initContainers, CronJobs, custom resources) or a docker compose service to the `Image` it runs
254255
- `DEPLOYS` — what puts manifests, kustomizations or charts on a cluster, to what it puts there: a Flux Kustomization or Argo Application to the kustomizations / manifests / chart of its path, a Flux HelmRelease to its chart (`spec.chart` or `chartRef` → OCIRepository), a shell command or Make recipe (`kubectl apply -f/-k`, `kustomize build | kubectl apply`, `helm install/upgrade/template`), a Go installer loading a chart with the Helm SDK, a Terraform `helm_release`. A chart outside the repository is a shared external `Chart`; the edge says `via` (`flux`, `argo`, `helmrelease`, `kubectl`, `kustomize`, `helm`, `helm-sdk`, `terraform`, ...)
255256
- `DEPENDS_ON` — a Helm chart to the charts it declares (the local chart for a resolvable `file://` repository, otherwise an external `Chart`; the edge keeps `repository`), and a `go.mod` / `requirements.txt` file to an external `Package`; a Dockerfile to the `Image`s it builds on (`via: from`, `copy-from`); an `Image` to the Dockerfile it is built from (`via: build`; `tool`: `skaffold`, `compose`, `docker-build` for `docker` / `podman` / `nerdctl` / `buildah` build commands in scripts, Make and CI); a compose service to the services its `depends_on` / `links` name
256-
- `CONFIGURES` — code reading an environment variable to the shared `EnvVar` of that name (`strategy: env_access`); a Kubernetes container, compose service or Dockerfile (`ENV` of the image) setting one to that container's own `EnvVar` (`strategy: env_set`, with `source`: `value`, `configMapKeyRef`, `secretKeyRef`, `fieldRef`, ..., `compose`, `dockerfile` — values are never stored)
257+
- `CONFIGURES` — code reading an environment variable to the shared `EnvVar` of that name (`strategy: env_access`); a Kubernetes container, compose service or Dockerfile (`ENV` of the image) setting one to that container's own `EnvVar` (`strategy: env_set`, with `source`: `value`, `configMapKeyRef`, `secretKeyRef`, `fieldRef`, ..., `compose`, `dockerfile` — values are never stored); and that container's `EnvVar` to the code reading the name inside the files its image is built from (`strategy: env_read`, with `dockerfile`) — the image's files are the sources its Dockerfile's `COPY` / `ADD` put into it, less the context's `.dockerignore`
257258
- `EMITS`, `LISTENS_ON` (channels)
258259
- `DATA_FLOWS` with arg-to-param mapping + field access chains
259260
- `SIMILAR_TO` (MinHash + LSH near-clone detection, Jaccard scored)
@@ -765,7 +766,7 @@ mode. `index_status` keeps describing the published graph and its freshness.
765766

766767
### Edge Types
767768

768-
`CONTAINS_FOLDER`, `CONTAINS_FILE`, `DEFINES`, `DEFINES_METHOD`, `IMPORTS`, `CALLS`, `CALL_REFERENCE`, `USAGE`, `INHERITS`, `IMPLEMENTS`, `OVERRIDE`, `DECORATES`, `RAISES`, `THROWS`, `WRITES`, `DATA_FLOWS`, `HTTP_CALLS`, `GRPC_CALLS`, `ASYNC_CALLS`, `EMITS`, `LISTENS_ON`, `HANDLES`, `SPAWNS`, `CONFIGURES`, `INFRA_MAPS`, `DEPENDS_ON`, `REFERENCES_FILE`, `MENTIONS`, `TESTS`, `TESTS_FILE`, `SIMILAR_TO`, `SEMANTICALLY_RELATED`, `FILE_CHANGES_WITH`, `HAS_BRANCH`, and the cross-repo `CROSS_HTTP_CALLS`, `CROSS_GRPC_CALLS`, `CROSS_GRAPHQL_CALLS`, `CROSS_TRPC_CALLS`, `CROSS_ASYNC_CALLS`, `CROSS_CHANNEL`
769+
`CONTAINS_FOLDER`, `CONTAINS_FILE`, `DEFINES`, `DEFINES_METHOD`, `IMPORTS`, `CALLS`, `CALL_REFERENCE`, `USAGE`, `INHERITS`, `IMPLEMENTS`, `OVERRIDE`, `DECORATES`, `RAISES`, `THROWS`, `WRITES`, `DATA_FLOWS`, `HTTP_CALLS`, `GRPC_CALLS`, `ASYNC_CALLS`, `EMITS`, `LISTENS_ON`, `HANDLES`, `SPAWNS`, `CONFIGURES`, `INFRA_MAPS`, `CONNECTS_TO`, `DEPLOYS`, `DEPENDS_ON`, `REFERENCES_FILE`, `MENTIONS`, `TESTS`, `TESTS_FILE`, `SIMILAR_TO`, `SEMANTICALLY_RELATED`, `FILE_CHANGES_WITH`, `HAS_BRANCH`, and the cross-repo `CROSS_HTTP_CALLS`, `CROSS_GRPC_CALLS`, `CROSS_GRAPHQL_CALLS`, `CROSS_TRPC_CALLS`, `CROSS_ASYNC_CALLS`, `CROSS_CHANNEL`
769770

770771
`MENTIONS` links a documented definition to the code its doc comment references (C# `<see cref>`, `<seealso cref>`, `<exception cref>`, `<inheritdoc cref>`), one edge per pair with `via`, `syntax`, `tier` (`exact` or `unique`), first `line` and `count`. A reference is bound only when it names exactly one definition under the language's own lookup rules; the rest are counted by reason in `index_status` under `doc_links` (samples with `diagnostics='full'`).
771772

‎internal/cbm/cbm.h‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -701,8 +701,10 @@ typedef struct {
701701
/* A fact a Kubernetes manifest states about other objects (extract_k8s.c),
702702
* resolved by the K8s pass against the objects of the source's bundle.
703703
* CBM_K8S_FACT_REF: object def_index names a kind/ns/name object at a
704-
* reference path (via), drawn as edge (USAGE, INFRA_MAPS or DEPENDS_ON);
705-
* ns NULL = the source's own namespace.
704+
* reference path (via), drawn as edge (USAGE, INFRA_MAPS, DEPENDS_ON or
705+
* CONNECTS_TO); ns NULL = the source's own namespace. A CONNECTS_TO
706+
* reference (via "env") is the Service the value of environment variable
707+
* `key` of container `container` addresses -- the value is not kept.
706708
* CBM_K8S_FACT_CRD: the CRD def_index declares kind `kind` in group `name`.
707709
* CBM_K8S_FACT_DEPLOY: object def_index (a Flux Kustomization, an Argo
708710
* Application) deploys the repository directory `name` into namespace
@@ -737,6 +739,7 @@ typedef struct {
737739
const char *edge;
738740
const char *route;
739741
const char *container;
742+
const char *key; /* CONNECTS_TO: the environment variable addressing it */
740743
int line;
741744
} CBMK8sFact;
742745

‎internal/cbm/extract_k8s.c‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1064,6 +1064,32 @@ static const char *k8s_env_source(k8s_walk_t *w, TSNode item) {
10641064
return "value";
10651065
}
10661066

1067+
// The Service a container's literal env value addresses (`host:port`, a URL,
1068+
// `<svc>.<ns>.svc...`, a bare name on an address key): a CONNECTS_TO
1069+
// reference, resolved in the bundle like every other. Only the service's
1070+
// name and namespace are kept -- never the value.
1071+
static void k8s_env_address(k8s_walk_t *w, TSNode item, const char *key, const char *cname) {
1072+
enum { K8S_DNS_LABEL = 64 };
1073+
const char *value = k8s_text(w, item, "value");
1074+
char svc[K8S_DNS_LABEL];
1075+
char ns[K8S_DNS_LABEL];
1076+
if (!value || cbm_env_address(key, value, svc, sizeof(svc), ns, sizeof(ns)) == 0) {
1077+
return;
1078+
}
1079+
CBMArena *a = w->ctx->arena;
1080+
CBMK8sFact f = {.fact = CBM_K8S_FACT_REF,
1081+
.def_index = w->def_index,
1082+
.kind = "Service",
1083+
.ns = ns[0] ? cbm_arena_strdup(a, ns) : NULL,
1084+
.name = cbm_arena_strdup(a, svc),
1085+
.via = "env",
1086+
.edge = "CONNECTS_TO",
1087+
.container = cname,
1088+
.key = key,
1089+
.line = (int)ts_node_start_point(item).row + SKIP_ONE};
1090+
cbm_k8sfact_push(&w->ctx->result->k8s_facts, a, f);
1091+
}
1092+
10671093
// A container (`path` ends in containers[], initContainers[] or
10681094
// ephemeralContainers[], any depth: pod templates, CronJobs, the pod
10691095
// templates of custom resources): its literal image, and the environment
@@ -1108,6 +1134,7 @@ static void k8s_container(k8s_walk_t *w, const char *path, TSNode container) {
11081134
.via = k8s_env_source(w, item),
11091135
.line = (int)ts_node_start_point(item).row + SKIP_ONE};
11101136
cbm_k8sfact_push(&w->ctx->result->k8s_facts, w->ctx->arena, f);
1137+
k8s_env_address(w, item, key, cname);
11111138
}
11121139
}
11131140

‎internal/cbm/helpers.c‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2140,6 +2140,170 @@ static bool is_env_var_pattern(const char *s, int len) {
21402140
return has_upper && has_underscore;
21412141
}
21422142

2143+
/* ── Service addresses in environment values ──────────────────────── */
2144+
2145+
static bool env_addr_key(const char *key) {
2146+
static const char *const SUFFIXES[] = {"_ADDR", "_ADDRESS", "_HOST",
2147+
"_URL", "_ENDPOINT", "_URI"};
2148+
size_t kl = strlen(key);
2149+
for (size_t i = 0; i < sizeof(SUFFIXES) / sizeof(SUFFIXES[0]); i++) {
2150+
size_t sl = strlen(SUFFIXES[i]);
2151+
if (kl >= sl && strcmp(key + kl - sl, SUFFIXES[i]) == 0) {
2152+
return true;
2153+
}
2154+
}
2155+
return false;
2156+
}
2157+
2158+
static bool env_label_char(char c) {
2159+
return islower((unsigned char)c) || isdigit((unsigned char)c) || c == '-';
2160+
}
2161+
2162+
/* A bare DNS name: `[a-z0-9][a-z0-9-]*(\.[a-z0-9-]+)*`, not only digits. */
2163+
static bool env_bare_name(const char *v) {
2164+
bool digits_only = true;
2165+
if (!env_label_char(v[0]) || v[0] == '-') {
2166+
return false;
2167+
}
2168+
for (const char *p = v; *p; p++) {
2169+
if (*p == '.' ? (p[SKIP_ONE] == '\0' || p[SKIP_ONE] == '.') : !env_label_char(*p)) {
2170+
return false;
2171+
}
2172+
digits_only = digits_only && (isdigit((unsigned char)*p) || *p == '.');
2173+
}
2174+
return !digits_only;
2175+
}
2176+
2177+
static bool env_copy(char *out, size_t sz, const char *s, size_t n) {
2178+
if (n == 0 || n >= sz) {
2179+
return false;
2180+
}
2181+
memcpy(out, s, n);
2182+
out[n] = '\0';
2183+
return true;
2184+
}
2185+
2186+
/* The host of a URL value (after any user@, before :port). */
2187+
static bool env_url_host(const char *v, char *host, size_t sz) {
2188+
const char *q = v;
2189+
if (!islower((unsigned char)*q)) {
2190+
return false;
2191+
}
2192+
while (islower((unsigned char)*q) || isdigit((unsigned char)*q) || *q == '+' || *q == '.' ||
2193+
*q == '-') {
2194+
q++;
2195+
}
2196+
if (strncmp(q, "://", strlen("://")) != 0) {
2197+
return false;
2198+
}
2199+
const char *h = q + strlen("://");
2200+
size_t hl = strcspn(h, "/ \t\r\n\"'");
2201+
for (size_t i = hl; i > 0; i--) {
2202+
if (h[i - SKIP_ONE] == '@') {
2203+
hl -= i;
2204+
h += i;
2205+
break;
2206+
}
2207+
}
2208+
size_t colon = strcspn(h, ":");
2209+
return env_copy(host, sz, h, colon < hl ? colon : hl);
2210+
}
2211+
2212+
/* `host:port` with a 2-5 digit port. */
2213+
static bool env_hostport(const char *v, char *host, size_t sz) {
2214+
enum { PORT_MIN = 2, PORT_MAX = 5 };
2215+
const char *c = strrchr(v, ':');
2216+
size_t pl = c ? strlen(c + SKIP_ONE) : 0;
2217+
if (!c || c == v || !isalnum((unsigned char)v[0]) || pl < PORT_MIN || pl > PORT_MAX ||
2218+
strspn(c + SKIP_ONE, "0123456789") != pl) {
2219+
return false;
2220+
}
2221+
for (const char *p = v; p < c; p++) {
2222+
if (!isalnum((unsigned char)*p) && *p != '.' && *p != '-') {
2223+
return false;
2224+
}
2225+
}
2226+
return env_copy(host, sz, v, (size_t)(c - v));
2227+
}
2228+
2229+
/* A `<svc>.<ns>.svc` name anywhere in the value. */
2230+
static bool env_svc_dns(const char *v, char *host, size_t sz) {
2231+
for (const char *m = strstr(v, ".svc"); m; m = strstr(m + SKIP_ONE, ".svc")) {
2232+
char after = m[strlen(".svc")];
2233+
if (isalnum((unsigned char)after) || after == '_') {
2234+
continue;
2235+
}
2236+
const char *ns = m;
2237+
while (ns > v && env_label_char(ns[-SKIP_ONE])) {
2238+
ns--;
2239+
}
2240+
if (ns == m || ns == v || ns[-SKIP_ONE] != '.' || ns[0] == '-') {
2241+
continue;
2242+
}
2243+
const char *name = ns - SKIP_ONE;
2244+
while (name > v && env_label_char(name[-SKIP_ONE])) {
2245+
name--;
2246+
}
2247+
bool word_start =
2248+
name == v || !(isalnum((unsigned char)name[-SKIP_ONE]) || name[-SKIP_ONE] == '_');
2249+
if (name == ns - SKIP_ONE || name[0] == '-' || !word_start) {
2250+
continue;
2251+
}
2252+
return env_copy(host, sz, name, (size_t)(m + strlen(".svc") - name));
2253+
}
2254+
return false;
2255+
}
2256+
2257+
int cbm_env_address(const char *key, const char *value, char *svc, size_t svc_sz, char *ns,
2258+
size_t ns_sz) {
2259+
char host[CBM_SZ_256];
2260+
if (!key || !value || !svc || !ns || svc_sz == 0 || ns_sz == 0) {
2261+
return 0;
2262+
}
2263+
bool found = env_addr_key(key) && env_bare_name(value) &&
2264+
env_copy(host, sizeof(host), value, strlen(value));
2265+
if (!found && (strstr(value, "{{") || strchr(value, '$'))) {
2266+
return 0;
2267+
}
2268+
found = found || env_url_host(value, host, sizeof(host)) ||
2269+
env_hostport(value, host, sizeof(host)) || env_svc_dns(value, host, sizeof(host));
2270+
if (!found) {
2271+
return 0;
2272+
}
2273+
int labels = 0;
2274+
const char *third = NULL;
2275+
for (const char *p = host; p;) {
2276+
const char *dot = strchr(p, '.');
2277+
size_t len = dot ? (size_t)(dot - p) : strlen(p);
2278+
if (labels == 0 && !env_copy(svc, svc_sz, p, len)) {
2279+
return 0;
2280+
}
2281+
if (labels == SKIP_ONE && !env_copy(ns, ns_sz, p, len)) {
2282+
return 0;
2283+
}
2284+
if (labels == PAIR_LEN) {
2285+
third = p;
2286+
}
2287+
labels++;
2288+
p = dot ? dot + SKIP_ONE : NULL;
2289+
}
2290+
if (labels == SKIP_ONE) {
2291+
ns[0] = '\0';
2292+
}
2293+
for (const char *p = svc; *p; p++) {
2294+
if (!env_label_char(*p)) {
2295+
return 0;
2296+
}
2297+
}
2298+
if (strcmp(svc, "localhost") == 0 || strspn(svc, "0123456789") == strlen(svc)) {
2299+
return 0;
2300+
}
2301+
bool svc_form =
2302+
labels <= PAIR_LEN || (third && strncmp(third, "svc", strlen("svc")) == 0 &&
2303+
(third[strlen("svc")] == '\0' || third[strlen("svc")] == '.'));
2304+
return svc_form ? labels : 0;
2305+
}
2306+
21432307
int cbm_classify_string(const char *str, int len) {
21442308
if (!str || len < PAIR_LEN) {
21452309
return NOT_FOUND;

‎internal/cbm/helpers.h‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,17 @@ bool cbm_is_keyword(const char *name, CBMLanguage lang);
2323
// stay in sync with kPyBuiltinNodes in internal/cbm/lsp/py_builtins.c.
2424
bool cbm_is_resolvable_builtin(const char *name, CBMLanguage lang);
2525

26+
// The service an environment variable's value addresses -- never the value
27+
// itself: the host of a URL, of `host:port`, a `<svc>.<ns>.svc[.<domain>]`
28+
// name inside it, or the whole value when `key` names an address (*_ADDR,
29+
// *_ADDRESS, *_HOST, *_URL, *_ENDPOINT, *_URI) and it is a bare DNS name. A
30+
// templated value, localhost or an IP addresses nothing. Writes the host's
31+
// first label to `svc` and its second (`svc.ns`, `svc.ns.svc...`) to `ns`
32+
// ("" for one label), and returns the label count when the host is a
33+
// service DNS name (`svc`, `svc.ns`, `svc.ns.svc[...]`), else 0.
34+
int cbm_env_address(const char *key, const char *value, char *svc, size_t svc_sz, char *ns,
35+
size_t ns_sz);
36+
2637
// Classify a string literal as URL, config, or neither.
2738
// Returns CBM_STRREF_URL (0), CBM_STRREF_CONFIG (1), or -1 for neither.
2839
int cbm_classify_string(const char *str, int len);

‎src/cli/cli.c‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1568,7 +1568,9 @@ static const char skill_content[] =
15681568
"HANDLES, IMPLEMENTS, OVERRIDE, USAGE, CALL_REFERENCE, CONFIGURES, REFERENCES_FILE,\n"
15691569
"MENTIONS (doc comment -> referenced code), FILE_CHANGES_WITH, SIMILAR_TO,\n"
15701570
"SEMANTICALLY_RELATED, CONTAINS_FILE, CONTAINS_FOLDER,\n"
1571-
"INFRA_MAPS (K8s Service -> selected workloads), DEPENDS_ON (chart / manifest -> dependency)\n"
1571+
"INFRA_MAPS (K8s Service -> selected workloads), DEPENDS_ON (chart / manifest -> dependency),\n"
1572+
"DEPLOYS (what puts manifests / charts on a cluster -> what it puts there),\n"
1573+
"CONNECTS_TO (workload / compose service -> the Service its environment addresses)\n"
15721574
"\n"
15731575
"## Cypher Examples (for query_graph)\n"
15741576
"```\n"

‎src/mcp/mcp.c‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8726,10 +8726,11 @@ static yyjson_doc *resolve_trace_edge_types(const char *args, const char *mode,
87268726
"CROSS_GRAPHQL_CALLS", "CROSS_TRPC_CALLS"};
87278727
/* topology: traffic (INFRA_MAPS), object references (USAGE), ordering
87288728
* (DEPENDS_ON), controllers (HANDLES), what an object declares
8729-
* (DEFINES: an Ingress's routes) and what it deploys (DEPLOYS: a Flux
8730-
* Kustomization's path, a HelmRelease's chart) */
8731-
static const char *mode_topology[] = {"INFRA_MAPS", "USAGE", "DEPENDS_ON",
8732-
"HANDLES", "DEFINES", "DEPLOYS"};
8729+
* (DEFINES: an Ingress's routes), what it deploys (DEPLOYS: a Flux
8730+
* Kustomization's path, a HelmRelease's chart) and the services its
8731+
* containers' environment addresses (CONNECTS_TO) */
8732+
static const char *mode_topology[] = {"INFRA_MAPS", "USAGE", "DEPENDS_ON", "HANDLES",
8733+
"DEFINES", "DEPLOYS", "CONNECTS_TO"};
87338734
/* deploy: what a CI job, script or Make rule starts (SPAWNS: scripts,
87348735
* make targets, programs), calls (CALLS: shell functions) and puts on a
87358736
* cluster (DEPLOYS) -- outbound from the job, inbound from a chart */

0 commit comments

Comments
 (0)