etoyama
diff --git a/‎skills/analysis-design/SKILL.md‎
Lines changed: 25 additions & 0 deletions b/‎skills/analysis-design/SKILL.md‎
Lines changed: 25 additions & 0 deletions
diff --git a/‎skills/batch-analysis/references/batch-prompt.md‎
Lines changed: 43 additions & 6 deletions b/‎skills/batch-analysis/references/batch-prompt.md‎
Lines changed: 43 additions & 6 deletions
@@ -99,6 +99,30 @@ If the interview did not cover methodology, ask now:
 - "What analysis method will you use? (e.g., OLS, t-test, chi-square, DID)"
 - Set `methodology = {method: "<answer>", reason: "<why this method>"}` at minimum.
 
+### Step 2.6: Enrich Methodology with Code Patterns
+
+The batch-analysis LLM (sonnet) generates notebooks from methodology content. Concrete code patterns dramatically improve generation fidelity.
+
+**When `data_source` references a registered catalog source**, auto-generate a code pattern from the catalog schema and include it in `methodology.steps`:
+
+```python
+methodology = {
+    "method": "OLS",
+    "package": "statsmodels",
+    "reason": "線形回帰で相関を検証",
+    "steps": [
+        {
+            "description": "from lib.accessor.bigquery import BigQueryAccessor\nacc = BigQueryAccessor(project_id='lmi-datau-prod')\nraw_df = acc.query_to_dataframe('''\n  SELECT col1, col2, ...\n  FROM `project.dataset.table`\n  WHERE ...\n''')"
+        }
+    ]
+}
+```
+
+**Rules:**
+- If `data_source` is a BQ table: generate `BigQueryAccessor` pattern with actual table name, key columns from schema, and filter conditions
+- If methodology uses a specific library (e.g., lingam, shap, dowhy): include the exact API call with parameter names in a subsequent step
+- If the user provides their own code pattern, use it as-is — do not overwrite
+
 ### Step 3: Create the Design
 
 ```
@@ -161,6 +185,7 @@ Only provided fields are updated; all others remain unchanged.
 | `methodology.method` | str | free text (required, non-empty) | — |
 | `methodology.package` | str | free text (optional) | `""` |
 | `methodology.reason` | str | free text (optional) | `""` |
+| `methodology.steps` | list[dict] | Each dict has `description` (str). Include concrete code patterns for batch LLM fidelity. | `[]` |
 
 **Backward compatibility**: `role`, `tier`, `intent` fields are optional in input. If omitted, defaults are applied automatically.
 
 
@@ -196,12 +196,18 @@ Execute the following steps in order. Log progress with markers for traceability
 4. If not found, use defaults:
    - `notebook_dir`: `.insight/runs/YYYYMMDD_HHmmss/{design_id}/`
    - `lib_dir`: none (disabled)
-4. Create run directory:
+5. **marimo version preflight check**:
    ```bash
-   RUN_DIR=".insight/runs/$(date +%Y%m%d_%H%M%S)"
+   uv run python -c "import re, marimo; m = re.match(r'(\d+)\.(\d+)\.(\d+)', marimo.__version__); assert m and tuple(int(x) for x in m.groups()) >= (0, 20, 3), f'marimo >= 0.20.3 required for export session (found {marimo.__version__})';"
+   ```
+   If the check fails, log the error and **stop the entire batch** — no design can be processed without `marimo export session`.
+6. Create run directory:
+   ```bash
+   RUN_DIR="${BATCH_RUN_DIR:-.insight/runs/$(date +%Y%m%d_%H%M%S)}"
    mkdir -p "$RUN_DIR"
    ```
-5. Record the `RUN_DIR` path for use throughout the session.
+   If the host project passes `BATCH_RUN_DIR` via environment variable, use it to keep session.log and batch output in the same directory.
+7. Record the `RUN_DIR` path for use throughout the session.
 
 ### Step 1: lib_dir Cataloging (if configured)
 
@@ -288,21 +294,52 @@ If `methodology.package` is specified:
 2. Create directory: `mkdir -p {notebook_dir}`
 3. If lib_dir is configured:
    - Read `{lib_dir}/CATALOG.md` for available utilities
-   - Add `sys.path.insert(0, "{lib_dir}")` to Cell 0
+   - Add lib_dir to Cell 0 using a **relative path from the notebook location**, not an absolute path:
+     ```python
+     import os as _os
+     sys.path.insert(0, _os.path.join(_os.path.dirname(__file__), _os.path.relpath("{lib_dir}", "{notebook_dir}")))
+     ```
    - Import relevant utility functions
 4. Generate notebook.py following the Cell Contract exactly:
    - Read the design's hypothesis, metrics, explanatory variables, chart specs, methodology, analysis_intent
    - Read the table schema (columns, types) from get_table_schema
    - Generate all 8 cells with appropriate content
    - Use the data source file path from catalog for CSV loading
-5. Write the notebook with the Write tool
+5. **BQ Type Coercion** (when Cell 2 uses `BigQueryAccessor.query_to_dataframe()`, not `pd.read_csv()`):
+   Add the following type coercion block in Cell 2, immediately after the query execution:
+   ```python
+   # Coerce BQ-specific types to pandas-standard types
+   for col in raw_df.select_dtypes(include=["dbdate", "dbtime"]).columns:
+       raw_df[col] = pd.to_datetime(raw_df[col])
+   # Exclude ID/key columns (suffix _id, _key, _code) to avoid precision loss
+   _numeric_cols = [c for c in raw_df.select_dtypes(include=["Int8", "Int16", "Int32", "Int64"]).columns
+                    if not re.search(r'_(id|key|code)$', c, re.IGNORECASE)]
+   for col in _numeric_cols:
+       raw_df[col] = raw_df[col].astype("float64")
+   ```
+   This prevents downstream errors: `dbdate` breaks `groupby`/`idxmax`, nullable `Int64` breaks numpy ufuncs. ID/key columns are excluded to avoid float64 precision loss on large integers.
+6. **Data Volume Strategy** (when data_source is a BigQuery table):
+   Before generating the analysis query, estimate row count as part of notebook generation:
+   1. Build a `COUNT(*)` query from the design's `data_source` + `filter_conditions`
+   2. Execute via `BigQueryAccessor`. If the COUNT fails (permissions, timeout, view complexity), skip estimation and use `direct` strategy as fallback
+   3. Include the estimated row count and chosen strategy as a comment in Cell 2: `# Estimated rows: {count} -> strategy: {strategy}`
+   4. The strategy is a **hint**, not a hard constraint — the agent chooses the final approach based on `methodology`:
+
+      | Row count | Strategy hint | Cell 2 guidance |
+      |-----------|---------------|-----------------|
+      | < 1M      | `direct`      | Pull all rows to pandas |
+      | 1M - 10M  | `sample`      | Consider TABLESAMPLE or BQ-side WHERE to reduce rows |
+      | > 10M     | `agg_first`   | Prefer BQ-side GROUP BY / PIVOT / QUALIFY before pull |
+7. Write the notebook with the Write tool
 
 #### 3f. Execute Notebook
 
 ```bash
-cd {notebook_dir} && uv run marimo export session --force-overwrite notebook.py 2>&1
+cd {notebook_dir} && perl -e 'alarm 600; exec @ARGV' -- uv run marimo export session --force-overwrite notebook.py 2>&1
 ```
 
+The 600-second (10 min) timeout prevents indefinite hangs when marimo fails to exit on cell errors. `perl -e 'alarm ...; exec @ARGV'` is used instead of `timeout` for macOS compatibility (GNU `timeout` is not available by default on macOS). A timeout-killed process enters the error repair loop (Section 5) like any other failure. **Limitation**: the alarm signal is delivered only to the exec'd process, not to its child process tree. In practice this is sufficient because `marimo export session` hangs are single-process.
+
 Check the result:
 - **Success**: session JSON exists AND cells 2, 3, 4, 6 have `text/markdown` output. Exit code alone is not sufficient (marimo may return exit code 1 with valid output on warnings).
 - **Failure**: session JSON missing OR cells 2/3/4/6 lack `text/markdown` output -> enter error repair loop (see Section 5)