feat/ GitHub pages and tutorials (#280)

* ci: add GitHub Actions workflow to deploy docs to GitHub Pages

Builds the existing mkdocs-material site and publishes it via
actions/deploy-pages on every push to main.

* Refactored and prettified documentation

* Reverte datasets dependency to >=5.0.0

* chore: sync uv.lock with datasets>=5.0.0 constraint

* Revert "chore: sync uv.lock with datasets>=5.0.0 constraint"

This reverts commit 396fa13.

* docs(tutorials): add tutorial notebooks and refresh annotation references

The annotations/ corpus is now 36 papers and every result_human.json parses
cleanly, so tutorial 5 no longer skips
9a889c1a671fd3cae48285eaa95069d189d02fe3 as corrupt. Update the counts in
tutorials 3 and 5, the tutorials index and the annotations guide, and rewrite
tutorial 5's "reading these numbers" section: on the corrected corpus three of
the four LLM judges now score harsher than the human annotator rather than
more leniently.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(tutorials): put the CLI before the guided pipeline, label tracks

Most users will run the pipeline through `lemat-synth batch`, so that notebook
now comes third and the stage-by-stage walkthrough fourth. Rows are labelled by
track (Use the data / Extract / Extend) rather than by difficulty: the notebooks
differ in what they are for, not in how hard they are.

Also repairs numbering drift left over from an earlier seven-notebook layout —
the CLI notebook was titled "Tutorial 5" and the ontology one "Tutorial 7" — and
realigns every cross-reference, the README table, the tutorials index and the
quickstart entry point.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(tutorials): run the notebooks on Colab as well as locally

Each tutorial gains an "Open in Colab" badge and one setup cell that detects the
environment: on Colab it shallow-clones the repository, installs it, and reads
API keys from Colab's secret manager; locally it walks up to pyproject.toml and
loads the repository-root .env exactly as before. Every cell below it is
unchanged in either environment.

Paths that assumed a local checkout now hang off the REPO_ROOT the setup cell
resolves - the annotations corpus, the ontology prompt files and the git-ignored
data/ scratch directories - replacing three copies of a repo_root() helper and
two hardcoded ../../.. paths.

Verified locally: tutorial 6 end to end, tutorial 5's offline half, and the
setup plus key-check cells of tutorials 1-4.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Updated the notebooks to be compatible in Colab

* Updated default Gemini models for CLI

* Updated default Claude model for CLI

* fix(tutorials): keep every model on one provider when overriding the CLI

The custom-override cell in tutorial 3 set api_base to OpenRouter but changed
only synthesis_model, leaving config/cli.yaml's `gemini/...` material and judge
models pointed at OpenRouter's URL - which answers 404, because api_base in
cli.py is one global value shared by every component LM. model_overrides() now
takes an optional synthesis model and always emits a consistent set, and the
markdown says plainly that api_base is not a per-model setting.

Also corrects the OpenRouter id for Claude Sonnet: `anthropic/claude-sonnet-4.6`
with a dot, not the dash form that names the model on the direct Anthropic route
(the dash form 404s through OpenRouter). This affected the plot/VLM slot in
tutorials 3 and 4, the tutorials index and the CLI reference.

The Colab bootstrap now clones an explicit branch, so the installed config and
the notebook cannot drift apart.

Verified by running the previously failing cell against OpenRouter end to end.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Fixed Claude name in CLI config

* Updated Gemini models for the CLI

* fix(tutorials): align direct-route fallback model with cli.yaml default

ALT_SYNTHESIS_MODEL's non-OpenRouter branch named gemini-3-flash-lite, which
config/cli.yaml no longer defaults to (it's gemini-3.5-flash-lite) - align the
two so the direct-provider path in this cell uses a model that actually
matches the rest of the notebook's defaults.

Also strips execution outputs picked up from local verification runs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* Updated Gemini models in tutorials

* fix(deps): pin anthropic exactly instead of pinning installs on Colab

anthropic>=0.54.0 had no ceiling, so a fresh `pip install -e .` (as the Colab
bootstrap cell does, since it doesn't use uv.lock) could resolve to a newer
Anthropic SDK release with a breaking Messages.create() signature - which is
what broke tutorial 4's plot extraction on Colab while working locally, where
uv.lock had anthropic pinned to 0.109.2.

Pinning to exactly that version in pyproject.toml (and syncing uv.lock to
match) fixes it at the source instead: `pip install -e .` now resolves the
same anthropic release everywhere, so the uv-export workaround added to the
Colab bootstrap cell in all six tutorials is no longer needed and is reverted.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* Reorganized notebooks and renamed contributing guidelines

* Moved case studies information from README to documentation

* Tested documentation and tutorial for developing a new domain

* Improved and standardized documentation landing page

* chore(deps): correct the transformers version-cap comment

The comment described a `transformers==4.57.1` pin that was itself commented
out, so it explained a constraint that was not in force; its claim that 4.57.1
was the highest 4.x release compatible with litellm's tokenizers pin is also
stale, since uv.lock already resolves 4.57.6 against tokenizers 0.22.2.

Rewrite it to describe the live `<5.0.0` cap and move it above the
`transformers` line it belongs to, rather than above `torch`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore: notebook execution output

* chore: ignore examples/notebooks/figures

* feat: merge with main (complete case tsudies)

---------

Co-authored-by: Stefan+o <ribes.stefano@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

Author: 3 people

.env.example

@@ -7,6 +7,10 @@ OPENAI_API_KEY=your_api_key_no_quotes      # if using OpenAI models
 MISTRAL_API_KEY=your_api_key_no_quotes     # if using Mistral models or Mistral OCR
 
 # OpenRouter (separate keys per model if desired; else set all to the same key)
+OPENROUTER_API_KEY=your_api_key_no_quotes           # generic: any openrouter/* model (used by the tutorials)
 OPENROUTER_QWEN_API_KEY=your_api_key_no_quotes      # qwen3.5-35b-a3b
 OPENROUTER_KIMI_API_KEY=your_api_key_no_quotes      # moonshotai/kimi-k2.5
-OPENROUTER_DEEPSEEK_API_KEY=your_api_key_no_quotes  # deepseek/deepseek-v3.2
+OPENROUTER_DEEPSEEK_API_KEY=your_api_key_no_quotes  # deepseek/deepseek-v3.2
+
+# HuggingFace (gated datasets: LeMat-Synth, LeMat-Synth-Papers)
+HF_TOKEN=your_token_no_quotes              # or run `hf auth login` instead

.github/workflows/docs.yml

@@ -0,0 +1,42 @@
+name: Deploy docs
+
+on:
+  push:
+    branches: [main]
+  workflow_dispatch: {}
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+concurrency:
+  group: pages
+  cancel-in-progress: false
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          python-version: "3.11"
+      - name: Install dependencies
+        run: uv sync
+      - name: Build site
+        run: uv run mkdocs build --site-dir _site
+      - uses: actions/upload-pages-artifact@v3
+        with:
+          path: _site
+
+  deploy:
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - id: deployment
+        uses: actions/deploy-pages@v4

.gitignore

@@ -52,10 +52,11 @@ annotations/annotation_guide_catalysis/annotation_guide_recording.mov
 # Documentation
 docs/_build/
 site/
+examples/notebooks/tutorials/demo_papers/
 
 # Generated figures/data (kept locally, not in git)
 results/figures/
 examples/scripts/**/*.svg
 examples/scripts/**/*.pdf
 examples/scripts/**/*.png
-examples/scripts/**/*.csv
+examples/scripts/**/*.csv