|
| 1 | +# Causal Feature Engineering for Quantitative Trading |
| 2 | + |
| 3 | +## Scientific Domain |
| 4 | + |
| 5 | +**Domain:** Quantitative Finance |
| 6 | +**Field:** Systematic Trading |
| 7 | +**Subfield:** Machine-Learned Alpha Research |
| 8 | + |
| 9 | +## Scientific Problem |
| 10 | + |
| 11 | +Quantitative researchers rarely succeed by fitting one model and accepting the |
| 12 | +largest public-period score. Minute-level market data are noisy, predictive |
| 13 | +relationships are weak, regimes change, and apparently strong signals can |
| 14 | +disappear after turnover, execution costs, or exposure normalization. The |
| 15 | +research problem is to discover causal predictive structure that remains useful |
| 16 | +outside the period in which it was designed. |
| 17 | + |
| 18 | +This task compresses that research loop into a fixed, reproducible environment. |
| 19 | +The agent is expected to inspect market data, form feature hypotheses, run |
| 20 | +controlled chronological experiments, compare predictive and economic |
| 21 | +diagnostics, reject unstable candidates, and select one strategy for hidden |
| 22 | +future scoring. Public metrics may conflict: a candidate can have attractive |
| 23 | +information coefficient but poor after-cost performance, or a high Sharpe on |
| 24 | +one interval with weak sign and regime stability. |
| 25 | + |
| 26 | +The final predictive estimator is deliberately fixed to an exact Elastic Net so |
| 27 | +that improvements must come primarily from causal feature construction, |
| 28 | +preprocessing, selection, regularization, and experimental judgment rather than |
| 29 | +from switching to a more powerful model family. Symbol identities, calendar |
| 30 | +coordinates, and target semantics are obscured to reduce historical lookup and |
| 31 | +force evidence-based research on the supplied panel. |
| 32 | + |
| 33 | +## Workflow Details |
| 34 | + |
| 35 | +The agent phase has two cooperating containers. The `main` container contains |
| 36 | +the public Parquet panel, task instructions, and the Python data-science stack. |
| 37 | +The `mcp-server` sidecar owns model fitting, artifact storage, public |
| 38 | +backtesting, research budgets, and final submission promotion. The agent |
| 39 | +interacts with that sidecar through five MCP tools: |
| 40 | + |
| 41 | +- `train_model` |
| 42 | +- `get_train_model_result` |
| 43 | +- `backtest` |
| 44 | +- `get_backtest_result` |
| 45 | +- `submit_strategy` |
| 46 | + |
| 47 | +The research workflow has four stages: |
| 48 | + |
| 49 | +1. **Public research.** Inspect the 2022–2023 panel chronologically and by |
| 50 | + symbol. Form hypotheses for past-only transformations such as returns, lags, |
| 51 | + rolling statistics, expanding histories, price-range structure, activity, |
| 52 | + and order-flow imbalance. Keep every symbol at a timestamp in the same |
| 53 | + chronological fold and fit learned preprocessing on training rows only. |
| 54 | +2. **Source-bound training.** Send complete Python source defining |
| 55 | + `train_model(X, y)` to the asynchronous training tool. The returned fitted |
| 56 | + object must be an exact `ElasticNet` or an exact sklearn `Pipeline` ending in |
| 57 | + an exact `ElasticNet`. It must use `weight_std_dollar_vol` as fitting weight, |
| 58 | + remain serializable with Joblib, and expose a stable inference schema. |
| 59 | +3. **Controlled public backtesting.** Backtest accepted models on public windows |
| 60 | + strictly later than their training windows. Compare candidates on consistent |
| 61 | + periods using information coefficient, error, prediction dispersion, |
| 62 | + turnover, after-cost Sharpe, CAGR, and maximum drawdown. Use later slices to |
| 63 | + challenge apparently strong candidates rather than optimizing one interval. |
| 64 | +4. **Candidate selection.** Submit the `strategy_id` from one successful |
| 65 | + backtest after all asynchronous operations are idle. The sidecar promotes one |
| 66 | + immutable bundle containing the selected source, fitted public artifact, |
| 67 | + inference contract, strategy settings, and integrity hashes. |
| 68 | + |
| 69 | +Training and backtesting share a budget of 100 accepted research attempts. One |
| 70 | +training operation and one backtest may overlap, but two operations of the same |
| 71 | +type may not. This permits deliberate parallel research without allowing |
| 72 | +unbounded search. |
| 73 | + |
| 74 | +After the agent exits, Harbor collects the bundle and starts a separate |
| 75 | +verifier. The verifier checks bundle integrity, imports the submitted source in |
| 76 | +a networkless model worker, and calls `train_model` once on every public |
| 77 | +2022–2023 row. It serializes and reloads that newly fitted estimator, predicts |
| 78 | +on hidden 2024 feature matrices, audits causal invariance by perturbing future |
| 79 | +suffixes, and runs the hidden after-cost backtest. Hidden labels and |
| 80 | +scoring-only columns never enter the model worker. |
| 81 | + |
| 82 | +## Dependencies & System Requirements |
| 83 | + |
| 84 | +### Agent-facing model environment |
| 85 | + |
| 86 | +Submitted model source runs on Python 3.12 and may import only `math`, |
| 87 | +`statistics`, `numpy`, `pandas`, and `sklearn`. The model must fit and predict |
| 88 | +without file I/O, external services, or network access. Source is limited to |
| 89 | +20,000 UTF-8 bytes; each fit and prediction operation has a 1,800-second |
| 90 | +deadline. The returned estimator must serialize with Joblib and predict after |
| 91 | +reload in a fresh process. No GPU is available or required. |
| 92 | + |
| 93 | +The broader agent container includes the pinned Python scientific stack needed |
| 94 | +to inspect the Parquet data and communicate with the MCP service. Core versions |
| 95 | +are Python 3.12, NumPy 2.5.1, pandas 3.0.5, SciPy 1.18.0, scikit-learn 1.9.0, |
| 96 | +PyArrow 25.0.0, Joblib 1.5.3, and FastMCP 3.4.5. |
| 97 | + |
| 98 | +### Task deployment environment |
| 99 | + |
| 100 | +The sample targets Harbor 0.20.0 with Docker Compose and Git LFS. The task |
| 101 | +requests four CPU cores, 8 GiB of memory, 32 GiB of storage, and no GPU. The |
| 102 | +configured limits are 30 minutes for environment construction, four hours for |
| 103 | +the agent, and one hour for verification. The agent environment has public |
| 104 | +network access for general research, while submitted source executes in a |
| 105 | +networkless, read-only model worker with narrowly controlled imports and file |
| 106 | +operations. |
| 107 | + |
| 108 | +## Dataset |
| 109 | + |
| 110 | +The benchmark uses a fixed panel of historical cryptocurrency minute bars for |
| 111 | +four anonymized symbols. The canonical source universe is BTCUSDT, DOGEUSDT, |
| 112 | +ETHUSDT, and SOLUSDT, but mapped to `symbol_01` through `symbol_04`. |
| 113 | +Datetimes are replaced with synthetic UTC coordinates while preserving |
| 114 | +ordering and elapsed durations. |
| 115 | + |
| 116 | +The chronological split is: |
| 117 | + |
| 118 | +- **Public research:** 2022-01-01 through 2023-12-31, containing 4,204,792 panel |
| 119 | + rows. |
| 120 | +- **Hidden evaluation:** 2024-01-01 through 2024-12-31, containing 2,108,152 |
| 121 | + panel rows. |
| 122 | + |
| 123 | +The agent-visible public schema contains: |
| 124 | + |
| 125 | +- identity: `datetime`, `symbol`; |
| 126 | +- candidate predictors: `open`, `high`, `low`, `close`, `volume`, |
| 127 | + `quote_asset_volume`, `number_of_trades`, |
| 128 | + `taker_buy_base_asset_volume`, and `taker_buy_quote_asset_volume`; |
| 129 | +- fitting metadata: `weight_std_dollar_vol`; |
| 130 | +- supervised response: `target_horizon_1`. |
| 131 | + |
| 132 | +`target_horizon_1` is intentionally anonymized. `weight_std_dollar_vol` is fitting |
| 133 | +metadata rather than a predictor. Two additional columns, `tradable_return` and |
| 134 | +`beta_10d_fwd_1`, are retained only by the trusted backtest and verifier. They |
| 135 | +are never included in model fitting or prediction matrices. |
| 136 | + |
| 137 | +The canonical public and hidden Parquet files are approximately 330 MB and |
| 138 | +176 MB respectively. The agent image receives a compressed projection of the |
| 139 | +public artifact with only identity, predictor, weight, and target columns. |
| 140 | +Hidden data remain exclusively in the verifier environment. The dataset is |
| 141 | +bundled with the task; no external download is required at runtime. |
| 142 | + |
| 143 | +## Evaluation Strategy |
| 144 | + |
| 145 | +The benchmark uses a continuous outcome reward rather than a binary pass |
| 146 | +threshold. The primary score is annualized Sharpe on the fixed hidden 2024 |
| 147 | +after-cost strategy returns. The same value is emitted as `primary_score`, |
| 148 | +`reward`, and `sharpe`. The verifier also reports hidden CAGR, maximum drawdown, |
| 149 | +and Pearson information coefficient so that risk-adjusted performance can be |
| 150 | +interpreted alongside predictive quality and downside behavior. |
| 151 | + |
| 152 | +The portfolio backtest applies a one-basis-point linear execution fee and uses |
| 153 | +525,600 periods per year for minute-level Sharpe annualization. The submitted |
| 154 | +`max_gross_exposure` must be finite and strictly positive. Public and hidden |
| 155 | +backtests use the same task-owned portfolio construction and accounting code. |
| 156 | +The absence of a binary cutoff is intentional: the scalar reward supports |
| 157 | +ranking, reinforcement learning, and comparison across research policies. |
| 158 | + |
| 159 | +Before scoring, the verifier enforces the following safeguards: |
| 160 | + |
| 161 | +1. **Submission integrity.** Exactly one publicly accepted bundle must exist. |
| 162 | + Source and artifact hashes must match the manifest, and bundle paths may not |
| 163 | + escape the submission directory. |
| 164 | +2. **Estimator family.** `train_model` must return either an exact |
| 165 | + `sklearn.linear_model.ElasticNet` or an exact sklearn `Pipeline` whose final |
| 166 | + step is an exact `ElasticNet`. Subclasses and other final estimators are |
| 167 | + rejected. |
| 168 | +3. **Executable contract.** Submitted source is size- and import-restricted. |
| 169 | + The estimator must be fitted, expose unique ordered string |
| 170 | + `feature_names_in_`, serialize with Joblib, reload in a fresh process, and |
| 171 | + return finite predictions in the original row order. |
| 172 | +4. **Full-public refit.** Hidden scoring uses a new estimator fitted by calling |
| 173 | + the submitted `train_model` source once on the complete public 2022–2023 |
| 174 | + panel. The estimator fitted during public experimentation is not scored. |
| 175 | +5. **Causal audit.** Three hidden future-suffix probes materially perturb later |
| 176 | + feature rows. Predictions for earlier rows must remain invariant within |
| 177 | + strict numerical tolerances. |
| 178 | +6. **Hidden isolation.** The model worker receives feature matrices but never |
| 179 | + hidden targets, tradable returns, market betas, or task-owned portfolio |
| 180 | + state. |
| 181 | + |
| 182 | +A missing final submission receives `-100` for every reward field. Contract |
| 183 | +violations are rejected before hidden economic metrics are exposed. |
| 184 | + |
| 185 | +## Complexity |
| 186 | + |
| 187 | +The computational primitives are standard, but the research problem is |
| 188 | +difficult because useful minute-level signals are weak and easy to overfit. |
| 189 | +Every candidate must survive chronological separation, cross-symbol analysis, |
| 190 | +causal batch execution, regularization, portfolio normalization, turnover, and |
| 191 | +costs. Fixing the final estimator to Elastic Net removes model-family search as |
| 192 | +an escape hatch: the agent's advantage must come from better hypotheses, |
| 193 | +features, controls, and candidate selection. |
| 194 | + |
| 195 | +The public panel contains more than 4.2 million rows, and accepted training and |
| 196 | +backtesting calls share a 100-attempt budget. A strong policy must allocate that |
| 197 | +budget among data diagnostics, feature ablations, regularization comparisons, |
| 198 | +chronological confirmation, and exposure experiments while asynchronous jobs |
| 199 | +are running. |
| 200 | + |
| 201 | +Observed task runs demonstrate a broad outcome range. The current reference |
| 202 | +strategy receives hidden Sharpe `0.3723068959` with hidden Pearson IC |
| 203 | +`0.0196662380`. Recorded end-to-end and candidate-validation artifacts include |
| 204 | +missing or invalid submissions at `-100`, valid strategies with negative Sharpe, |
| 205 | +degenerate historical baselines at `0`, and positive candidates between |
| 206 | +approximately `0.15` and `0.39`. These are continuous outcomes rather than |
| 207 | +pass/fail labels. |
| 208 | + |
| 209 | +A representative Harbor run completed in 20 minutes 53 seconds end to end: |
| 210 | +14 seconds for environment setup, 28 seconds for agent setup, 16 minutes |
| 211 | +40 seconds for agent execution, and 3 minutes 11 seconds for verification. The |
| 212 | +configured limits leave room for substantially deeper research than the |
| 213 | +reference policy performs. |
| 214 | + |
| 215 | +## References & Resources |
| 216 | + |
| 217 | +The sample is self-contained. Its implementation contracts and supporting |
| 218 | +evidence are documented in: |
| 219 | + |
| 220 | +- `instruction.md` — agent-facing objective, data split, MCP workflow, and |
| 221 | + full-public refit disclosure; |
| 222 | +- `environment/agent/context/research.md` — causal research and chronological |
| 223 | + validation guidance; |
| 224 | +- `environment/agent/context/submission.md` — exact estimator, source, |
| 225 | + serialization, and prediction contract; |
| 226 | +- `environment/agent/context/backtesting.md` — public candidate-comparison and |
| 227 | + strategy-selection guidance; |
| 228 | +- `DATA_PROVENANCE.md` — fixed artifact hashes, source universe, |
| 229 | + anonymization, split construction, and license record; |
| 230 | +- `environment/mcp-server/task_config.yaml` — task-owned data, backtest, cost, |
| 231 | + execution, reward, and budget configuration; |
| 232 | +- `environment/mcp-server/feature_engineering/evaluation/verifier.py` and |
| 233 | + `scoring/official.py` — independent verification and hidden scoring flow; |
| 234 | +- `solution/solve.py` — reference MCP-only agent; |
| 235 | +- `environment/mcp-server/tests/` — causal, portfolio, runtime, release, |
| 236 | + end-to-end, and verifier behavior contracts. |
| 237 | + |
| 238 | +The complete local Python verification command is: |
| 239 | + |
| 240 | +```sh |
| 241 | +uv run --project environment/mcp-server test |
| 242 | +``` |
| 243 | + |
| 244 | +## Additional Information |
| 245 | + |
| 246 | +This document describes the OpenAI-facing sample as an outcome-scored research |
| 247 | +environment rather than a binary programming exercise. Its hidden Sharpe reward |
| 248 | +is intended for continuous evaluation, policy comparison, and reinforcement |
| 249 | +learning. The fixed Elastic Net family is an experimental control that focuses |
| 250 | +the task on feature engineering and quantitative-research decisions. |
| 251 | + |
| 252 | +## Author Information |
| 253 | + |
| 254 | +**Contributing organization:** EdotEnv |
| 255 | +**Contribution:** Task design, implementation, data packaging, verifier, |
| 256 | +reference policy, and calibration. |
0 commit comments