Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
d57897e
Move to resources dir in package
UnsignedArduino Aug 1, 2026
16a8844
Add skeleton for toolchain and target commands
UnsignedArduino Aug 1, 2026
c6f926c
Toolchain install command
UnsignedArduino Aug 2, 2026
00e291f
Target install command
UnsignedArduino Aug 3, 2026
3d03118
Fix build command to use toolchain/targets
UnsignedArduino Aug 4, 2026
22dee24
Toolchain uninstall command
UnsignedArduino Aug 4, 2026
1a09557
Target uninstall command
UnsignedArduino Aug 4, 2026
7158822
Remove unused install mkcd tools function, handled in toolchain setup
UnsignedArduino Aug 4, 2026
66378b1
Actually use the template npm cache
UnsignedArduino Aug 4, 2026
e24071c
Add project context doc for the rewrite
claude Oct 28, 2026
293191e
Add CLAUDE.md
claude Oct 28, 2026
00dd04e
Fix e2e test and enable redun caching
UnsignedArduino Oct 29, 2026
d7ad64a
Run lint and e2e CI on PRs into any branch
UnsignedArduino Oct 29, 2026
aca1ae1
Add MKCD2APP_HOME override and rename --clear-cache to --full-rebuild
UnsignedArduino Oct 29, 2026
60f100d
Fix small known issues: UTF-8 IO, resource path, config version error
UnsignedArduino Oct 29, 2026
4d518d9
Install website deps in their own task keyed on package.json and lock…
UnsignedArduino Oct 29, 2026
6d9ed1b
Revise PROJECT_CONTEXT.md for toolchain polish, manage Node.js and Ru…
UnsignedArduino Nov 3, 2026
f8979ba
Replace system git with dulwich for the github code source
claude Nov 3, 2026
5a70495
Generalise the github code source to git: {url, ref}
claude Nov 3, 2026
d5af806
Add file code source for PNG/UF2 with embedded project
claude Nov 3, 2026
8ba0db3
Add real PNG/UF2 editor exports as file source fixtures
claude Nov 3, 2026
fb43fd3
Fix file source e2e test on Windows CRLF checkouts
claude Nov 3, 2026
8c0bb08
Accept ZIPs of a project in the file code source
claude Nov 3, 2026
d1f88e1
Add e2e test building a GitHub-style ZIP of the empty game
claude Nov 3, 2026
d069bfb
Fix semver, title templating, and index.html filling bugs
claude Nov 3, 2026
bcb8393
Add coverage reporting and unit/e2e test markers
claude Nov 3, 2026
06130e9
Add unit tests for code/website tasks, redun patches, and utils
claude Nov 3, 2026
8925378
Split unit and e2e tests into separate CI workflows
claude Nov 4, 2026
c39140a
Merge pull request #8 from UnsignedArduino/ccr-209d5b24-ah720e
UnsignedArduino Nov 4, 2026
f1ca06d
Merge branch 'toolchain' of https://github.com/UnsignedArduino/MakeCo…
claude Nov 4, 2026
374092b
Add unit tests for semver, title rendering, and index.html filling
claude Nov 4, 2026
41cc7ac
Fix uninstall caching, relative MKCD2APP_HOME, and PNG/UF2 decoding bugs
claude Nov 4, 2026
3d8cbc0
Pass tasks only the config they use and resolve paths against the con…
UnsignedArduino Nov 4, 2026
f2f118f
Update tests and docs for per-task config and config-relative paths
UnsignedArduino Nov 4, 2026
d890eca
Add unit tests for the install tasks, CLI, config and recent fixes
UnsignedArduino Nov 4, 2026
3ec30bd
Add an integration test tier and fix the bugs it found
UnsignedArduino Nov 4, 2026
316dbc0
Add e2e tests for more sources, output contents and offline builds
UnsignedArduino Nov 4, 2026
be3196d
Let a commit message skip test tiers on a PR [skip-tests]
UnsignedArduino Nov 4, 2026
ce12c48
Use relative asset paths in the static output
UnsignedArduino Nov 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .github/workflows/check-skip.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Check for a skip tag

# Called by the test workflows, to skip their tier if the last commit of a PR says to:
# [skip-<tier>-tests], [no-<tier>-tests], [skip-<tier>] or [no-<tier>], in any case, or
# [skip-tests] or [no-tests] for every tier. Pushes to main always run everything, so
# what's merged still gets tested.
on:
workflow_call:
inputs:
tier:
description: The test tier to check for, like "unit"
required: true
type: string
outputs:
skip:
description: '"true" if this tier should be skipped'
value: ${{ jobs.check.outputs.skip }}

jobs:
check:
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
skip: ${{ steps.check.outputs.skip }}
steps:
# A pull_request event doesn't include the commit message, so look it up
- id: check
if: github.event_name == 'pull_request'
env:
GH_TOKEN: ${{ github.token }}
REPOSITORY: ${{ github.repository }}
SHA: ${{ github.event.pull_request.head.sha }}
TIER: ${{ inputs.tier }}
run: |
message=$(gh api "repos/$REPOSITORY/commits/$SHA" --jq .commit.message)
if grep -qiE "\[(skip|no)-($TIER(-tests)?|tests)\]" <<< "$message"; then
echo "skip=true" >> "$GITHUB_OUTPUT"
echo "Skipping the $TIER tests, as the commit message asks"
fi
19 changes: 15 additions & 4 deletions .github/workflows/e2e.yaml
Original file line number Diff line number Diff line change
@@ -1,13 +1,22 @@
name: CI
name: E2E tests

on:
push:
branches: [ main ]
# Any base branch, so stacked PRs get CI too
pull_request:
branches: [ main ]

jobs:
check-skip:
uses: ./.github/workflows/check-skip.yaml
permissions:
contents: read
with:
tier: e2e

e2e:
needs: check-skip
if: needs.check-skip.outputs.skip != 'true'
strategy:
matrix:
os: [ ubuntu-latest, windows-latest, macos-latest ]
Expand Down Expand Up @@ -35,5 +44,7 @@ jobs:
- name: Install dependencies
run: uv sync

- name: Run tests with pytest
run: uv run pytest
# Coverage is skipped: the tests run the mkcd2app executable in a subprocess,
# which coverage doesn't measure
- name: Run e2e tests
run: uv run pytest -m e2e --no-cov
44 changes: 44 additions & 0 deletions .github/workflows/integration.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Integration tests

on:
push:
branches: [ main ]
# Any base branch, so stacked PRs get CI too
pull_request:

jobs:
check-skip:
uses: ./.github/workflows/check-skip.yaml
permissions:
contents: read
with:
tier: integration

integration:
needs: check-skip
if: needs.check-skip.outputs.skip != 'true'
strategy:
matrix:
os: [ ubuntu-latest, windows-latest, macos-latest ]
runs-on: ${{ matrix.os }}
permissions:
contents: read
steps:
- name: Check out repo
uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
version: "latest"

- name: Set up Python
run: uv python install 3.14

- name: Install dependencies
run: uv sync

# Offline and fast; npm and mkc are faked, so no Node.js or toolchain install
- name: Run integration tests with coverage
run: uv run pytest -m integration
2 changes: 1 addition & 1 deletion .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ name: CI
on:
push:
branches: [ main ]
# Any base branch, so stacked PRs get CI too
pull_request:
branches: [ main ]

jobs:
lint:
Expand Down
44 changes: 44 additions & 0 deletions .github/workflows/unit.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Unit tests

on:
push:
branches: [ main ]
# Any base branch, so stacked PRs get CI too
pull_request:

jobs:
check-skip:
uses: ./.github/workflows/check-skip.yaml
permissions:
contents: read
with:
tier: unit

unit:
needs: check-skip
if: needs.check-skip.outputs.skip != 'true'
strategy:
matrix:
os: [ ubuntu-latest, windows-latest, macos-latest ]
runs-on: ${{ matrix.os }}
permissions:
contents: read
steps:
- name: Check out repo
uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
version: "latest"

- name: Set up Python
run: uv python install 3.14

- name: Install dependencies
run: uv sync

# Offline and fast; no Node.js or toolchain install needed
- name: Run unit tests with coverage
run: uv run pytest -m unit
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,7 @@ __pycache__/
examples/*
!examples/*.yaml
!examples/*.png
tests/fixtures/build/
.idea
.coverage
.coverage.*
112 changes: 112 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Rewrite status, branch/PR stack, roadmap, and known bugs live in the project context
doc, which is imported here. Update that doc (and its "verified as of" line) rather
than duplicating its content in this file.

@docs/PROJECT_CONTEXT.md

## Commands

Python ≥3.14 and Node.js (LTS) are required. Everything runs through uv.

```bash
uv sync # install deps + the mkcd2app entry point
uv run ruff check . # lint
uv run ruff format --check . # format check (drop --check to apply)
uv run mypy . # strict type check
uv run pytest # all tests, with a coverage report
uv run pytest -m unit # only the fast offline tests (tests/unit)
uv run pytest -m integration # only the offline scheduler tests (tests/integration)
uv run pytest -m e2e --no-cov # only the e2e tests (tests/e2e)
uv run pytest --no-cov # skip coverage (e.g. when using a debugger)
uv run pytest tests/e2e/test_statics.py::test_empty_game_build_to_statics_static
```

CI runs exactly these commands: `lint.yaml` the lint/format/mypy checks, and
`unit.yaml` (`-m unit`), `integration.yaml` (`-m integration`) and `e2e.yaml`
(`-m e2e --no-cov`) the tests, independently, on ubuntu/windows/macos. Every test must
be under `tests/unit`, `tests/integration` or `tests/e2e`, or collection fails, since a
test anywhere else would run in no workflow. Test file names must be unique across those
folders, since they have no `__init__.py`. Warnings are errors in pytest.
On a PR, `[skip-<tier>-tests]`, `[no-<tier>-tests]`, `[skip-<tier>]` or `[no-<tier>]` in
the last commit's message skips that tier (`unit`, `integration` or `e2e`; see
`check-skip.yaml`), `[skip-tests]` or `[no-tests]` skips all three, and GitHub's own
`[skip ci]` skips every workflow. Pushes to `main` ignore all but `[skip ci]`.
pre-commit runs ruff check, ruff format --check, and mypy on staged Python files.

Running the tool end to end (installs must happen first, in this order):

```bash
uv run mkcd2app toolchain install
uv run mkcd2app target install 4.0.14
uv run mkcd2app --debug build "examples/Racers.yaml"
```

Notes on the tests:
- `tests/conftest.py` marks every test with its tier by its directory, and holds the
shared fixtures, such as `mkcd2app_home` (points `MKCD2APP_HOME` at a temp dir). Keep
it the only `conftest.py`: mypy fails on two modules both named `conftest`.
- Shared helpers (`make_config_yaml`, `FakeRunCmd`, `capture_logger`, ...) are in
`tests/helpers.py`, imported as `from helpers import ...`.

Notes on the unit tests:
- Call a redun task's plain function with `.func(...)`, and monkeypatch `run_cmd` in
the module under test, so no `npm`/`mkc` runs.
- Our loggers set `propagate = False`, so `caplog` needs `capture_logger` to see them.

Notes on the integration tests:
- They run the real `main()` and redun scheduler in this process, with `run_cmd` faked,
so no Node.js is needed. Use the `project` fixture (a game with a config and an
installed target) or `cli` (runs any command), which return what ran as a `Ran`.
- `FakeToolchain` does to the disk what each `mkc` and `npm` command would, and fails the
test on any other command, so a new command needs a new case there.
- `TaskSpy` records the tasks that really ran, since a cache hit doesn't call a task's
function. What each change should rerun is the `CHANGES` table in `test_rebuilds.py`.

Notes on the e2e tests:
- Run pytest from the repo root: tests use repo-relative paths, and shell out to the
installed `mkcd2app` executable rather than importing it.
- Builds go in temp folders, each with a copy of the game and config next to each other,
since a config's relative paths are relative to it. A full build takes about a minute,
so share one (module fixtures) where a test only needs to read the output.
- `--run-network` also runs the tests marked `network`, which need a public share link.
CI doesn't pass it.
- `tests/fixtures/block_network.cjs` is a Node preload that fails any network connection
and logs it, for testing offline builds, as Node ignores `HTTP_PROXY` by default.

## Architecture

Every unit of work is a redun `@task(namespace="mkcd2app")`. `main.py` parses the CLI
and hands a task expression to `run_redun_task()` with the SQLite DB for that command.
redun builds the DAG from how task results feed into other tasks, so data dependencies
between tasks must go through task arguments/returns, not side channels.

Build pipeline (`build_project/__init__.py`), each step a task:
`fetch_code` → `build_binary_js` (runs `mkc` in its own copy of the source) +
`copy_support_files` (simulator HTML from the target dir, icon) +
`copy_website_template` → `fill_website_template`, plus `install_website_deps`
(`npm ci` from only the template's `package.json` and lockfile) → one `build_website*`
task per entry in `outputs`. A new output type means a new `case` arm in that `match`
plus a field on `BuildProjectResult`; until then `electron` and `tauri` raise
`NotImplementedError`.

Relative paths in a config are relative to the config file. `main.py` makes them
absolute with `resolve_config_paths` before any task runs.

Conventions that matter for redun hashing:
- `build_project` takes the whole config as JSON (`model_dump_json`) and passes each
task only what it uses, as strings (models dumped to JSON). Every argument is part of
a task's cache key, so an unused one causes needless reruns. Never pass pydantic
models as task arguments.
- Each stage writes to a fresh directory (named via `_website_dir` in `website.py`) and
returns a `ContentDir`/`ContentFile`; never modify a path another task returned.
- `mkcd2app/__init__.py` imports `_redun_compat` first, which monkey-patches redun and
asserts the pinned version. Changing the redun version means re-checking those patches.

Paths under the user data/state dirs (toolchain, `js_tools/node_modules/.bin` for `mkc`,
npm cache, per-version target dirs, redun DBs) all come from `utils/paths.py`; use those
helpers instead of building paths by hand. Bundled templates and fixtures under
`resources/` are reached through `utils/resources.py`.
Loading
Loading