Skip to content

2026-07-14-cycle2-design.md

Latest commit

 

History

History
259 lines (219 loc) · 17 KB

2026-07-14-cycle2-design.md

File metadata and controls

259 lines (219 loc) · 17 KB
Raw

Cycle 2 — btc-longhorizon-predictor — Design Spec (REGISTERED)

Status: REGISTERED at cycle2-registration-v1 (2026-07-14). Signed off; the three open decisions (§9) are resolved and frozen. The authoritative literals now live in config/gates.yaml, config/costs.yaml, config/params.yaml; the verbatim clauses and reconciliations are in docs/registration_log.md. No model has been trained and the held-out block has not been touched. Per the build order the sequence from here is: plan → TDD build, pausing at each ⏸ checkpoint.

Cycle 1 context: repo btc-dollarbar-predictor, tag modeling-cycle-1-closed. Verdict KILL-for-now — a real ~57%-accuracy directional signal worth $0 after 11 bps taker costs, fold-unstable, held-out never opened. Cycle 2 is the one follow-on licensed by Cycle 1's registered reopen conditions (a) materially lower costs and (c) a new hypothesis class, and it carries a pre-committed exit (§D9) that is part of the registration, not a mood.

1. Hypothesis (register verbatim)

At holding periods of ~8 hours to ~2 days, two things change simultaneously relative to Cycle 1:

Costs shrink in relative terms and in absolute terms. Typical moves grow ~√time while fees are fixed; and because entries are no longer time-critical, maker execution becomes feasible, roughly halving the cost floor (taker 4.5 bps/side → maker ~1.8 bps/side, BNB-discounted).

The feature timescale matches the label timescale. Cycle 1 asked 15-minute microstructure features to predict multi-hour outcomes only at the top of its horizon grid. Cycle 2 builds features natively at 1h/4h resolution — vol regimes, multi-day momentum, funding history, session/weekday structure.

The null hypothesis remains the favorite: BTC may be efficient at these horizons too. A second honest kill is a valid, valuable output and triggers the exit.

2. Contamination guard (register verbatim, per the Cycle-1 closeout)

"The temporal-decay table from the 15m cycle is hypothesis-generating background only; this cycle's gates, horizons, and regime clauses are set independently of it. The G4 regime clause carries over precisely because the prior cycle's recent-positive pattern is suspected trend-dependence."

3. What carries over unchanged (do not redesign)

  • Sealed held-out boundary: heldout.start_ms = 1752696900000 (2025-07-16T20:15:00Z). Same block, still sealed, opened at most once at the end of this cycle, only for a config that passes ALL validation gates. On the 1h grid this literal is not a bar edge; it snaps to first-sealed-bar open 21:00 (a clean 1-bar split — data ⏸ report §3, log B6.1). The final-exam evaluator inherits the same per-horizon developability rule (log B7): when the held-out is finally opened, its own first developable bar is 21:00, and each held-out bar needs h bars of runway inside the sealed block to carry a label at horizon h — so the last h bars of the held-out tail are undevelopable at h, exactly as the dev-region tail is. Stated now to avoid re-litigating the seam at exam time.
  • Rigor spine: pre-registration commits before implementation (tag cycle2-registration-v1); purged + embargoed anchored expanding walk-forward; train-only selection of everything (h, θ-variant params, hyperparameters, p*); out-of-sample calibration with no in-sample fallback feeding the decisive statistic (fold-1's calibration comes from a burn-in scheme — §D6); block-bootstrap CIs; non-overlapping evaluation subsample; gates before models; TDD with leakage tests; registration_log.md for every mid-build mechanic decision.
  • Verdict contract: KILL / POSITIVE-BUT-UNCERTAIN / CLEAR PASS with committed responses, plus the validation-insufficiency rule (no config passing all gates with the minimum OOS trade count → KILL-for-now, held-out sealed).
  • Amendment 2: accuracy / F1 / AUC are diagnostics only, never verdicts. G5 net expectancy decides.

4. Registration decisions D1–D9

D1 — Substrate

Primary: 1h klines (Binance USDⓈ-M BTCUSDT, monthly + daily from data.binance.vision, same downloader pattern as Cycle 1; ~56k bars 2020 → boundary). Secondary ablation: 4h bars derived by aggregating the 1h parquet (never downloaded separately — one source of truth). 15m data is NOT used for features in this cycle.

D2 — Horizon grid

h ∈ {8, 16, 24, 48} in 1h bars (8h, 16h, 1d, 2d). Cap rationale: at h=48 the pre-held-out region still holds ~1,000 non-overlapping windows. Train-only selection per fold.

D3 — Feature panel (frozen at registration; hard cap 14; no selection step; permutation importance diagnostic-only)

  1. σ_robust_1h = EWMA of 1h log-returns, span 24 (one day) — the single volatility definition of this cycle.
  2. Vol-regime ratios on EWMA spans {6, 24, 168} (6h, 1d, 1w): rv_fast/rv_mid, rv_mid/rv_slow, |log_ret|/rv_mid. (3 features)
  3. Standardized momentum at lookbacks {4, 24, 72, 168} bars: cum log-return / (σ_robust·√L). (4 features)
  4. taker_buy_ratio (1h), and its z-score over 168 bars. (2 features)
  5. vol_z: z-score of ln(quote_volume) over 168 bars. (1 feature)
  6. Funding features (new, native to this timescale): last settled funding rate; cumulative funding over the past 24h. Source: Binance fundingRate history (public, monthly files). Strictly as-of: only funding stamps with settlement time ≤ bar close. (2 features)
  7. tod_sin / tod_cos + day-of-week one-hot (weekend structure matters at 1–2 day holds).
  8. range_norm = (high−low)/close ÷ σ_robust.

All rolling windows prior-bar only; gap-spanning rule carries over; causality tests (truncated-recompute bit-identity) mandatory.

Implementation note (blocking — needs a mechanic ruling at registration, not a redesign): the panel enumerates to 16 scalar features before day-of-week is encoded at all: σ_robust_1h(1) + vol-regime ratios(3) + momentum(4) + taker_buy_ratio+z(2) + vol_z(1) + funding(2) + tod_sin/tod_cos(2) + range_norm(1) = 16. Day-of-week then adds +7 (one-hot) or +1 (is_weekend), giving 23 or 17. Every encoding overshoots the hard cap of 14. This is a real arithmetic conflict in the frozen literal, not a redesign question, and I will not silently resolve it (see §9, Decision 1): either raise the cap to the exact registered count, or name which features/encodings collapse or drop to reach 14.

D4 — Labels

Fixed-horizon 3-class on ln(close_{t+h}/close_t), economic θ primary, vol-scaled (k·σ_robust·√h, k train-only) secondary — same structure as Cycle 1. MFE/MAE persisted per (t,h) from 1h highs/lows with the same conservative both-touched ambiguity rule. Triple-barrier remains gated behind the same two-branch promotion rule; its parameterization from Cycle 1 carries over with σ_robust_1h substituted.

D5 — Cost model (the load-bearing change; register with extra care)

Two execution models, BOTH priced for every config; the gate structure makes them jointly decisive.

  • Taker model (conservative anchor): 4.5 bps/side + 1 bp slippage/side → 11 bps round trip, as Cycle 1.
  • Maker-entry model: maker fee 1.8 bps/side (0.02% − 10% BNB) + 0 spread-crossing on entry, BUT with an honest fill haircut — register: assumed fill probability φ = 0.7 for a limit at touch on 1h decision bars; unfilled entries are missed trades (expectancy contribution 0), NOT chase-to-taker. Exit at horizon is taker (time-bound exits can't wait on a fill) → maker-taker round trip ≈ 1.8 + 4.5 + 1 slippage ≈ 7.3 bps, applied to the φ-weighted trade stream. φ is an assumption, not a measurement — the cycle's weakest registered constant and a Track-B measurement target.
  • Funding in expectancy: sign-aware, accrued per settled stamp actually crossed during the hold (now material: a 2-day hold crosses ~6 stamps; use the realized historical funding series, not a constant, since it's known data, not a forecast).
  • θ_econ(h) = 2 × taker-model round trip + expected funding magnitude for h — the label threshold stays anchored to the conservative model so labels don't get easier under optimistic execution assumptions.

D6 — Calibration

OvR isotonic on the traded classes, fit strictly out-of-sample. Fold-1 fix (the Cycle-1 lesson, promoted to design): reserve the first 20% of fold-1's validation block as calibration burn-in — its trades are excluded from all decisive statistics; fold-1's scored remainder is calibrated on the burn-in. "All decisive statistics" is enumerated (log B3): (i) the G4 fold-1 informative 30-trade floor is assessed on POST-burn-in trades; (ii) the G5 200-trade floor; (iii) the p*-selection pool — plus expectancy/CI, which already run on the decisive stream. No in-sample calibration anywhere in the decisive path. The calibration-provenance unit test and the de-calibration robustness probe from Cycle 1 are REQUIRED tests from day one, not audit-time additions.

D7 — Gates (G1–G5; all literals in gates.yaml before any training)

  • G1: ≥ 3pp over majority baseline, non-overlapping metric, held-out.
  • G2: block-bootstrap (block = h) 95% CI excludes baseline.
  • G3: per-traded-class Brier ≤ 0.24, slope ∈ [0.8, 1.2], out-of-sample calibration only.
  • G4 (both clauses, harmonized floors): ≥ 5/6 informative folds positive, where a fold with < 30 trades is UNINFORMATIVE (neither pass nor fail — Cycle-1 harmonization carries over); AND mean_net > 0 over the registered non-trending calendar windows (2022 ∪ 2023 literals carry over) with ≥ 30 in-regime trades, else clause = not-clear.
  • G5 (decisive):
    • Under the TAKER model: net expectancy > 0 AND CI-low > −half the taker round trip (−5.5e-4) AND ≥ 200 OOS trades (lower than Cycle 1's 300: 1h bars mechanically produce fewer decisions; rationale registered).
    • Under the MAKER model: reported alongside; a config whose taker-G5 is insufficient but maker-G5 is clean is a registered POSITIVE-BUT-UNCERTAIN outcome (verdict: proceed only to a fill-measurement / paper-trading spec, never live) — this classification is decided now, in the registration, not when a config lands there.
  • Selector: filter on all validation gates, rank by pooled mean_net (taker model), ties → longer h → more trades. p* selection = max mean_net s.t. trades ≥ G5 floor; insufficient → max-trades p*. Both carried from Cycle 1's logged mechanics.

D8 — Baselines

Majority, last-bar-sign momentum, 4-state Markov (train-fold quantiles) — plus one new baseline this substrate demands: buy-and-hold over the same horizon windows, costed identically. At 1–2 day holds in a structurally appreciating asset, a long-only drift baseline is the honest benchmark a directional model must beat to justify existing. Register: beating majority but not costed buy-and-hold at the same trade count = NOT a pass narrative.

D9 — The exit (register verbatim)

"If Cycle 2's verdict is KILL: the quant-research line is shelved as a solved question — 'the edge floor at retail-accessible costs and data sits above what BTC klines yield at 15m–2d horizons' — with held-out still sealed. No Cycle 3 is licensed by this result. Reopening requires a categorically different input (measured maker fills from live paper infrastructure, trade-level data, a different market), registered blind as a new project. POSITIVE-BUT-UNCERTAIN under the maker model licenses exactly one thing: a fill-measurement paper-trading spec, no capital."

5. Build order (checkpoints ⏸)

  1. Spec → sign-off ⏸ → registration commit (cycle2-registration-v1: gates.yaml, costs.yaml with both execution models, params.yaml literals, D9 exit, contamination-guard sentence) → plan → TDD build.
  2. Data: 1h klines + funding-rate history download, validation, quality report. ⏸ Sign-off.
  3. Features + labels + MFE/MAE + reports (label distributions per h under θ_econ — expect far fewer flats than 15m; class balance shifts materially at 22+ bps thresholds on 1–2 day moves). ⏸ Sign-off.
  4. Baselines including costed buy-and-hold. ⏸ Report before ML.
  5. Tree grid (XGB/RF/LGBM, Cycle-1 hyperparameter literals carried over; LSTM again reserved to a checkpoint ruling). Both cost models priced for every config. The validation report must include a φ-sensitivity row: maker-G5 reported at φ ∈ {0.5, 0.7, 0.9} (φ = 0.7 stays the registered headline; a config surviving only at φ = 0.9 is thereby visibly fragile). This is reporting, not a gate change — added post-freeze per log B4, touches no frozen literal. ⏸ Full validation report.
  6. If and only if a config passes ALL validation gates: single held-out touch, pre-selected by the registered selector. Else: verdict per contract, held-out sealed.
  7. Memo + inflation-style honesty ledger (smaller this time: naive vs full protocol, plus taker vs maker sensitivity) + tag modeling-cycle-2-closed.

6. Compute & parallelism policy (NEW — addresses the standing instruction)

The Cycle-1 mistakes to not repeat: a null test ran single-worker (45 min) and an audit ran single-threaded. Cycle 2 default: the WFV grid runs parallel (process pool, BLAS pinned to 1 thread/worker — the Cycle-1 fix), never single-worker. GPU is used where it is deterministic, because the seed-17 reproducibility spine outranks speed and is non-negotiable:

  • XGBoost: tree_method='hist', device='cuda', seed 17. GPU hist is reproducible GPU-to-GPU on a single device with a fixed seed; it is not bit-identical to CPU (FP reduction order differs).
  • LightGBM: GPU with deterministic=true, force_row_wise=true, seed 17.
  • RandomForest: sklearn is CPU-only; run n_jobs=-1 (parallel trees, deterministic under random_state=17 regardless of worker count).
  • Guard: a test_gpu_determinism bit-identity test (two consecutive fits under seed 17 → identical predictions) runs per estimator at build time. Any estimator that fails falls back to CPU for the registered run. The device actually used for the decisive run is pinned in registration_log.md. This gives us GPU speed where it's safe and CPU determinism everywhere else — honoring "use GPU if possible, else CPU" without ever weakening reproducibility.

(This policy is a §9 decision — I recommend the determinism-gated form above.)

7. Code reuse from Cycle 1 (proposed)

The rigor spine is substrate-agnostic and battle-tested; §3 says "do not redesign" it. Proposal: port (copy + adapt) from btc-dollarbar-predictor into this repo — purged/embargoed WFV split logic, block-bootstrap, gate-metric functions, the calibration harness, and the leakage/calibration- provenance tests — and rewrite only the substrate-specific parts: the 1h + funding downloader, the D3 feature panel, the D4/D5 labels+cost models, and θ_econ(h). Copying files out of Cycle 1 into this repo does not commit anything into btc-dollarbar-predictor — its sealed, closed status is untouched. (This is a §9 decision — I recommend porting over rewriting-from-scratch.)

8. Registration artifacts to be committed at cycle2-registration-v1

  • config/gates.yaml — G1–G5 literals (incl. −5.5e-4 CI floor, 200-trade floor, G4 30-trade UNINFORMATIVE floor, 2022 ∪ 2023 regime windows), verdict contract, insufficiency rule.
  • config/costs.yaml — both execution models (taker 11 bps; maker 7.3 bps round-trip with φ = 0.7), funding = realized series, θ_econ(h) anchored to taker.
  • config/params.yaml — substrate (1h; 4h-agg ablation), horizon grid {8,16,24,48}, feature panel (frozen), heldout.start_ms = 1752696900000, seed 17, WFV/purge/embargo/burn-in literals.
  • This spec, the D9 exit verbatim, the contamination-guard sentence verbatim.
  • docs/registration_log.md seeded with the compute-policy and code-reuse rulings.

9. Decisions — RESOLVED at registration (2026-07-14)

  1. D3 feature cap — RESOLVED: cap 14 → 17, keep the full enumerated panel, day-of-week as a single is_weekend column. Logged as a pre-freeze reconciliation (registration_log R2.1), dated before the freeze, no data seen. Cap=17 is now immutable; feature #18 → deferred_features.
  2. Compute policy — RESOLVED: determinism-gated GPU (registration_log R2.2). GPU for XGB/LGBM only where a committed bit-identity test (GPU==GPU and GPU==CPU under seed 17) passes; RF CPU; grid parallel, BLAS pinned; never single-worker.
  3. Code reuse — RESOLVED: port the Cycle-1 rigor spine from btc-dollarbar-predictor @ e8d4c57 (registration_log R2.3–R2.4), tests included, adaptations enumerated, held-out guard ported with the same literal + injection test.

All of D1–D9 is frozen. Everything downstream is plan + TDD build against these files.