CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

snip is a CLI proxy written in Go that reduces LLM token consumption by 60-90% by filtering shell output before it reaches the LLM context. Inspired by rtk (Rust Token Killer), snip improves on the concept with a declarative filter DSL — filters are YAML config files, not compiled code.

Key Concept

The binary (snip) is the engine. Filters are data files. The two evolve independently. Anyone can contribute a filter without knowing Go.

Repository Structure

cmd/snip/main.go        # Entry point
embed.go                # Embedded default filters (go:embed)
filters/*.yaml          # Declarative filter definitions (132 filters)
internal/
  cli/                  # CLI routing, flag parsing
  config/               # TOML config loading (~/.config/snip/config.toml)
  display/              # Lipgloss terminal styling, gain report
  engine/               # Command execution (goroutines), pipeline orchestration
  filter/               # DSL types, 20 built-in actions, YAML parser, registry
  hook/                 # Claude Code PreToolUse hook handler (native Go, no bash/jq)
  initcmd/              # Multi-agent hook installation (Claude Code, Cursor, Codex, Windsurf, Cline)
  discover/             # Session history scanner for missed savings
  learn/                # Scan session history for failed commands, generate CLI-correction rules
  verify/               # Run the inline `tests:` blocks of filter YAML files
  trust/                # SHA-256 trust store for user filter files (trust/untrust)
  economics/            # Model pricing tiers, $ savings estimates (cc-economics)
  hookaudit/            # Audit log of hook rewrite decisions (hook-audit)
  inspect/              # Code quality checks on snip's own Go source
  tracking/             # SQLite token tracking (pure Go, no CGO)
  tee/                  # Raw output recovery on failure
  utils/                # Truncate, StripANSI, EstimateTokens, LazyRegex
tests/fixtures/         # Test fixtures for integration tests

Architecture

Core Loop

1. Intercept command via snip hook (Claude Code PreToolUse, native Go)
2. Route to matching filter (O(1) registry lookup)
3. Execute original command, capture stdout/stderr via goroutines
4. Apply declarative filter pipeline (regex-based: keep/remove lines, reformat, template)
5. Output filtered result, track token savings in SQLite

Why Go over Rust

• Static binaries, no runtime dependencies, trivial cross-compilation
• Goroutines naturally solve the stdout/stderr concurrent read problem (vs 2 OS threads in rtk)
• Lower barrier to entry for community contributions
• Pure Go SQLite driver (no CGO needed)

Development Commands

make build               # Build static binary (CGO_ENABLED=0)
make build-lite          # Build without SQLite tracking (-tags lite, ~5MB smaller)
make test                # Run all tests with coverage
make test-race           # Run tests with race detector
make lint                # go vet + golangci-lint (version pinned in .golangci-lint-version)
make verify              # Run the inline `tests:` blocks of filters/*.yaml
make vulncheck           # govulncheck ./...
make ci                  # Pre-PR gate: test-race + verify + lint + vulncheck
make install             # Install using GOBIN or the Go environment
make install-lite        # Install lite variant
make upgrade             # Replace active snip (GOBIN overrides)
make upgrade-lite        # Replace active snip with lite variant
go test -run TestName ./internal/filter/...   # Single test
goreleaser release --snapshot --clean          # Test release build locally

Working in this repo

• snip is usually installed as a hook here, so it filters go and git output while you work. For raw output: /usr/bin/git ..., or printf '#!/bin/sh\nexec go "$@"\n' > /tmp/rawgo && chmod +x /tmp/rawgo
• snip verify runs the embedded filter set: rebuild the binary after editing anything in filters/. make verify goes through go run, so it always sees the current filters/
• Filter inline tests: blocks are CI-enforced by the Verify filters step in .github/workflows/ci.yaml

Design Constraints

• Startup < 10ms — snip intercepts every shell command; latency is critical
• Hook path is fast — snip hook loads filters + registry only (no SQLite, no tracking)
• SQLite init cost — modernc.org/sqlite init() adds ~3.4ms; use NewLazyTracker + WarmUp() to overlap with command execution
• Graceful degradation — if a filter fails, fall back to raw command output
• Exit code preservation — always propagate the underlying tool's exit code
• No async runtime — goroutines are sufficient; avoid heavy dependencies
• Lazy compilation — compile regex once (sync.Once), reuse across invocations
• Minimal memory — stream and filter line-by-line, don't buffer entire output

Build Variants

Two build modes via Go build tags:

• Full (default): includes modernc.org/sqlite for token tracking via internal/tracking/driver.go
• Lite (-tags lite): excludes SQLite, uses driver_lite.go stub — startup ~3ms faster

Tests requiring SQLite must have //go:build !lite tag. Check tracking.DriverAvailable at runtime.

Filter DSL

Filters are declarative YAML files with 20 built-in actions: keep_lines, remove_lines, truncate_lines, truncate_bytes, strip_ansi, head, tail, group_by, dedup, json_extract, json_schema, ndjson_stream, regex_extract, state_machine, aggregate, format_template, compact_path, replace, match_output, on_empty

Gotchas that have caused real bugs:

• aggregate replaces the lines it counted unless append: true (caused #134, #136 in three filters)
• {{.count}} is the number of lines reaching the template, not entities: wrong after any stage emitting a summary, an overflow marker or a cap (caused #125). Prefer the tool's own count over recomputing one
• Any stage that drops or counts lines must sit where payload and metadata are still distinguishable — a remove_lines after a state machine will happily delete panic text
• exclude_flags and skip_if_present match with strings.HasPrefix; skip_if_present is all-or-nothing, one match disables the whole injection
• inject.args may reformat the answer, never suppress part of it. A flag that hides content (--stat, --no-merges) needs an escape in exclude_flags and must be named in the output (#124)

Release Workflow

Uses GoReleaser (.goreleaser.yaml) + GitHub Actions (.github/workflows/release.yaml). A push of any v* tag triggers CI to build cross-platform binaries and create a GitHub release.

Semver Tagging

Every push that changes behavior must include a version tag:

• Patch (v0.1.1): bug fixes, no API change
• Minor (v0.2.0): new features, backward-compatible
• Major (v1.0.0): breaking changes

git tag -a v0.1.1 -m "fix: description" && git push origin v0.1.1

Checklist

1. make test passes
2. Commit with conventional prefix (fix:, feat:, breaking:)
3. Create annotated tag following semver
4. Push tag — CI handles release automatically
5. For first-time or local validation: goreleaser release --snapshot --clean

Conventions

• All code, comments, variable names, commits, and documentation files must be in English
• Direct communication style — no hedging, state facts and solutions
• TDD workflow: write test first, implement, refactor
• Use context wrapping on errors: fmt.Errorf("operation: %w", err)
• When adding a new built-in subcommand: add it to both the switch in cli.go AND isBuiltInCommand in flags.go (the authoritative list of builtins)
• When changing user-facing behavior, filters, or architecture: update the GitHub wiki (git clone https://github.com/edouard-claude/snip.wiki.git) to stay in sync
• Reviewing a fix: revert the production hunk, re-run the new test, confirm it fails. Two of six PRs this month shipped tests that passed either way
• Before calling a defect a regression, reproduce it on master too — several reported this month were pre-existing, which changes the verdict from "blocker" to "out of scope"