Skip to content

Split exact-match NameRes lookups into their own log-analysis notebook #139

Description

@gaurav

log-analysis/nameres/analyze_nameres_logs.py was narrowed to autocomplete-only in #107. Exact-match (autocomplete=False) lookups now have no home.

They were split out because mixing the two made every aggregate a weighted average of two populations that behave nothing alike. Measured over a 16,461-row mixed export:

mode n p50 Solr wait p95 p99 max
autocomplete 504 48.5 ms 2,803 ms 263,722 ms 321,254 ms
exact 15,957 14.4 ms 42.7 ms 486 ms 29,830 ms

Exact lookups are ~97% of traffic and the far healthier population, but they are what most NameRes callers actually use, so they still want characterizing before the ElasticSearch comparison.

Suggested shape:

  • A sibling marimo notebook over the general data/log-analysis/nameres-*.json exports (the ones the autocomplete notebook's glob deliberately excludes).
  • Reuse the existing parser rather than copying it. That probably means promoting QueryLogEntry / parse_record / LOOKUP_LINE_RE out of the notebook cell into src/babel_validation/, at which point tests/log_analysis/test_nameres_log_notebook.py can import them directly instead of going through Cell.run().
  • Typing chains do not apply here; the interesting splits are result limit, filter usage, and query length.

Note that the general exports are capped 10,000-row CloudWatch samples, not complete populations — unlike the autocomplete-only export, whose filter is selective enough to fit under the cap.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions