Skip to content

Commit ecbec5e

Browse files
si1verwind17claude
andcommitted
Fixes from mapping public repos: scanner 0.10.0, Rules 4, lanes, Airflow demo, eval
Nine public repos were mapped with the skill; every fix is general, none names a framework taken from them. - Scanner 0.8–0.10: one rule for generated and vendored code; proto messages as classes and gRPC ends in every language; a queue needs a producer and a consumer in repo code; no false SQL writes or tables (statement shape, keyword stop-list, interpolated names); C# global usings; route handlers passed by reference or nested, same-file mounts composed, route keys normalised to {id}; calls through typed fields and interfaces two levels down, Go method sets, Python bases; Rust test code dropped; exclude: remembered; field visibility; a "Not scanned" section and deployable.unscanned. - Rescans go full only when a deployable, module or the scanner changes; stale facts are found by line and say why; evidence moves with the code. - Rules 4 (SKILL.md): secrets read as key names only, compose files included; .gitattributes written into the output directory; deployables, gateways and job queues worded generically; external.unused and the route warnings. - Audit: budget counted in boxes, deterministic; --full-audit; lane checks (lane-drop, gap-stub, lane-gap) and lane names in text.over-line. - Viewer: taps drop straight onto storage lanes again, as in 1.1.1. - Demo: the features and forecast pipelines run on Airflow, not Flyte. - Eval: scenarios 1–7 rerun (scenario 3 gains a secrets check); see eval/README.md. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
1 parent eee4566 commit ecbec5e

125 files changed

Lines changed: 6688 additions & 3232 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎README.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -60,8 +60,9 @@ Later, run it again, or ask:
6060

6161
> Refresh the code map for the latest commit.
6262
63-
The map lands in `docs/viz-code/`. Open `docs/viz-code/index.html`, and commit `code-map.json`, `stack.yaml` and
64-
`README.md` from that folder; the rest are working files, already gitignored.
63+
The map lands in `docs/viz-code/`, and nothing outside it is touched. Open `docs/viz-code/index.html`, and commit
64+
`code-map.json`, `stack.yaml`, `README.md`, `.gitignore` and `.gitattributes` from that folder; the rest are working
65+
files, already gitignored.
6566

6667
**Found something it gets wrong?** Run the skill with `send-feedback`
6768
(`/codebase-map:visualizing-codebases send-feedback`), or ask your agent to send feedback on the code map. It drafts

‎demo/parking-city/.gitattributes‎

Lines changed: 0 additions & 1 deletion
This file was deleted.

‎demo/parking-city/README.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ code-map viewer can be demonstrated and tested on
77
something public, with every kind of factory element present at least once.
88

99
Eleven runtime services (Scala 3 + ZIO 2 + tapir, and Python), two shared
10-
libraries, one Airflow project and two Flyte projects.
10+
libraries and three Airflow projects.
1111

1212
## How it is wired
1313

@@ -20,7 +20,7 @@ libraries, one Airflow project and two Flyte projects.
2020
`FINES_API_URL`, `SMTP_HOST`, `SENSORCLOUD_WS_URL`, ...) or fixed vendor hosts.
2121
- One Postgres holds all tables (`sql/schema.sql` shows the owner of each);
2222
the analytics warehouse is BigQuery.
23-
- The Airflow project and the two Flyte projects are deployed by registration
23+
- The three Airflow projects are deployed by registration into one DAG bucket
2424
(`deploy/register.sh`, `jobs/README.md`, `forecast/README.md`,
2525
`features/README.md`).
2626

@@ -40,8 +40,8 @@ libraries, one Airflow project and two Flyte projects.
4040
| `analytics-loader` | Python | `analytics_loader/` |
4141
| `reporting-api` | Python | `reporting_api/` |
4242
| `nightly-jobs` (Airflow) | Python | `jobs/` |
43-
| `demand-forecast` (Flyte) | Python | `forecast/` |
44-
| `feature-store` (Flyte) | Python | `features/` |
43+
| `demand-forecast` (Airflow) | Python | `forecast/` |
44+
| `feature-store` (Airflow) | Python | `features/` |
4545

4646
Libraries: `common` (Scala: JDBC helper, Kafka producer, config; used by
4747
reservations, pricing, billing) and `common_py` (Python: Pub/Sub wrapper,
@@ -59,7 +59,7 @@ Redis cache helper, Twilio SMS client, SMTP mailer, DB connect).
5959
| Pull subscription | `analytics_loader/app/main.py` + `common_py/common_py/pubsub.py` `Puller.pull`; subscription `bay-status-analytics` |
6060
| Dead letters | `billing/.../DeadLetters.scala` produces to `reservation-events.dlq` when event handling fails (`ReservationEventsConsumer.scala`) |
6161
| Rail yard (busy batch destinations) | `jobs/dags/*.py`: 10 DAGs, 4 to BigQuery, 3 to Postgres, 2 to S3, 1 to billing; several run many times a day (`*/15`, hourly) |
62-
| Cross-project batch | `forecast/workflows.py` `reference_task` to `compute_zone_features` in `features/workflows.py` |
62+
| Cross-project batch | `forecast/demand_forecast.py` `TriggerDagRunOperator` to the `zone_features` DAG in `features/zone_features.py` |
6363
| Warehouse + lane (database) | Postgres tables `reservations`, `reservation_outbox`, `tariffs`, `zones`, `invoices`, `payments`, `bay_status`, `daily_revenue`, `zone_utilisation`, `demand_forecast` (`sql/schema.sql`); BigQuery `analytics.bay_events` and friends |
6464
| Own cache | `reservations/.../QuoteCache.scala` on `REDIS_URL`, read by nobody else |
6565
| Shared cache | `SHARED_CACHE_URL`: written by `occupancy` (`common_py/common_py/cache.py`), read by `pricing/.../SharedCache.scala` and `city_dashboard/app/main.py` |
@@ -68,7 +68,7 @@ Redis cache helper, Twilio SMS client, SMTP mailer, DB connect).
6868
| Inbound webhook | `payments_webhook/app/main.py` `POST /webhooks/stripe` |
6969
| Read-only service | `reporting_api/app/main.py`: only SELECTs on `reservations`, `invoices`, `bay_status` |
7070
| Libraries (parts racks) | `common/` (sbt module) and `common_py/` (package) |
71-
| Twin | `pipelines_shared/` registered into both Flyte projects, `deploy/register.sh` |
71+
| Twin | `pipelines_shared/` synced by both the `features` and the `forecast` registration, `deploy/register.sh` |
7272
| Replicas | `deploy.replicas` in `docker-compose.yml`: `notifier` 2, `enforcement-worker` 3 |
7373
| Deliberate smell | see below |
7474

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
11
#!/usr/bin/env bash
22
# Registers the orchestrated projects. Runtime services are deployed from
33
# docker-compose.yml; these three are deployed by registration.
4+
# All three share one Airflow DAG bucket, each project in its own folder.
45
set -euo pipefail
56

67
# Airflow project -> service "nightly-jobs"
7-
gsutil -m rsync -r -d jobs/dags "gs://${AIRFLOW_DAGS_BUCKET}/dags"
8+
gsutil -m rsync -r -d -x '^(features|forecast|pipelines_shared)/' jobs/dags "gs://${AIRFLOW_DAGS_BUCKET}/dags"
89

9-
# Flyte project "features" -> service "feature-store"
10-
pyflyte register --project features --domain production --image "${PIPELINE_IMAGE}" features pipelines_shared
10+
# Airflow project "features" -> service "feature-store"
11+
gsutil -m rsync -r -d features "gs://${AIRFLOW_DAGS_BUCKET}/dags/features" && gsutil -m rsync -r -d pipelines_shared "gs://${AIRFLOW_DAGS_BUCKET}/dags/pipelines_shared"
1112

12-
# Flyte project "forecast" -> service "demand-forecast"
13-
pyflyte register --project forecast --domain production --image "${PIPELINE_IMAGE}" forecast pipelines_shared
13+
# Airflow project "forecast" -> service "demand-forecast"
14+
gsutil -m rsync -r -d forecast "gs://${AIRFLOW_DAGS_BUCKET}/dags/forecast" && gsutil -m rsync -r -d pipelines_shared "gs://${AIRFLOW_DAGS_BUCKET}/dags/pipelines_shared"

‎demo/parking-city/docker-compose.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
# are NOT started here: they are reached through the env keys below
33
# (DB_HOST, REDIS_URL, SHARED_CACHE_URL, KAFKA_BROKERS, ...).
44
#
5-
# The Airflow project (jobs/) and the Flyte projects (forecast/, features/)
5+
# The three Airflow projects (jobs/, forecast/, features/)
66
# are deployed by registration, see deploy/register.sh.
77

88
x-db: &db
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
code-map.json linguist-generated=true -diff

‎demo/parking-city/docs/viz-code/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ How to read the map: the field guide at https://si1verwind17.github.io/codebase-
1010
| `stack.yaml` | libraries and route styles this repo uses that the scanner does not know built in | the agent; you may edit | yes |
1111
| `map-notes.md` | your standing notes on the map | you | yes |
1212
| `README.md` | this file | `assemble.py` | yes |
13+
| `.gitignore` | keeps the files not committed out of git | `assemble.py`; you may add lines | yes |
14+
| `.gitattributes` | marks `code-map.json` generated, so diffs fold it | `assemble.py`; you may add lines | yes |
1315
| `index.html` | the map viewer page | `assemble.py`, `view.py` | no |
1416
| `audit.md` | drawing and content checks, findings | `assemble.py`, `audit.py` | no |
1517
| `structure.json`, `summary.md` | the scanner's facts, and the digest the agent reads | `extract.py` | no |

0 commit comments

Comments
 (0)