|
| 1 | +# AGENTS.md (TLA+ specs) |
| 2 | + |
| 3 | +This file applies to every file in this folder. |
| 4 | +It tells an agent how to write and check a TLA+ spec for a change in this repo. |
| 5 | + |
| 6 | +## Policy |
| 7 | + |
| 8 | +1. This policy is a trial. A maintainer decides when it ends. While it runs, write a spec for each change you implement. |
| 9 | +2. The goal is a knowledge base of how the repo works. Cost is not a reason to skip a spec. |
| 10 | +3. Record the cost and the findings in the spec header. A maintainer uses them to cut scope later. |
| 11 | +4. This policy is a choice. It is not a claim that specs find more bugs than tests or reviews. |
| 12 | + |
| 13 | +## Stance |
| 14 | + |
| 15 | +1. Before you write a spec, write down what you expect it to find. After the run, write what it found. |
| 16 | +2. Write the first version of each invariant before you read the review comments or the fix. Add later invariants through the review ratchet. |
| 17 | +3. Report a null result. A spec that finds nothing new is data, not a failure. |
| 18 | +4. Do not tune a spec until it finds a bug you already know. |
| 19 | +5. A spec checks the model, not the code. Do not say a spec proves the code correct. |
| 20 | + |
| 21 | +## What to model |
| 22 | + |
| 23 | +Write the unit tests and do the review as usual. The spec does not replace them. |
| 24 | + |
| 25 | +| Change kind | The spec models | |
| 26 | +| -------------------------------------------- | -------------------------------------------------------------------- | |
| 27 | +| Order of events, async work, state over time | The actions and every order of them | |
| 28 | +| A value that enters a function | The input domain. Make the parser or callee an oracle. | |
| 29 | +| A setting or a config value | Each reader of the value and its default | |
| 30 | +| None of these | The source and the sink. Ask what a bad value can do at each source. | |
| 31 | + |
| 32 | +## Terms |
| 33 | + |
| 34 | +| Term | Meaning | |
| 35 | +| -------------- | ------------------------------------------------------------------------------------- | |
| 36 | +| TLC | The TLA+ model checker. It tries every state and checks each invariant. | |
| 37 | +| oracle | A function the spec does not define. TLC tries every result. Example: `parses`. | |
| 38 | +| replay | Run a counterexample on the real code. It shows if the counterexample is real. | |
| 39 | +| sabotage check | Remove a guard on purpose. The invariant must fail. If not, the invariant is vacuous. | |
| 40 | +| vacuous | True in all states because of the model, not because of the code. | |
| 41 | +| gate | The check an agent runs by hand on each commit. No script or CI step runs it yet. | |
| 42 | + |
| 43 | +## Procedure |
| 44 | + |
| 45 | +1. Search this folder for a spec of the same subsystem. If one exists, extend it. |
| 46 | +2. Write each failure that the change must prevent as one sentence. |
| 47 | +3. Write each invariant as the opposite of one failure. |
| 48 | +4. Mark the source and the sink in the code. Name the functions and events. Do not use line numbers. They go stale. |
| 49 | +5. Model only the source and the sink. Make every other part an oracle. Do not copy a parser. |
| 50 | +6. Limit the input domain to values that look real. If TLC finds an input that no user can send, add a rule to remove it. |
| 51 | +7. Run TLC on the unfixed code. If the code has no known bug, run the sabotage check. The invariant must fail. |
| 52 | +8. Replay each counterexample on the real code. Keep the counterexamples that leak. Record the false alarms. |
| 53 | +9. Run TLC on the fixed code. The invariant must hold. |
| 54 | +10. Write a property test from the same invariant sentence. Run it on the real code. |
| 55 | +11. Simplify the spec. Use the questions below. |
| 56 | +12. Run each TLC config and the property test by hand on each fix commit. Write the result in the spec header. |
| 57 | + |
| 58 | +## Review ratchet |
| 59 | + |
| 60 | +A review finding is a failure that someone saw in this repo. Use each finding to grow the spec. |
| 61 | + |
| 62 | +1. After each review round, read each finding. |
| 63 | +2. If the spec can express the failure, add an invariant or an input rule. Cite the PR and the thread in a comment. |
| 64 | +3. If the first version of the spec already covered the finding, write "covered" in the header. |
| 65 | +4. If the spec cannot express the finding, write "gap" in the header and say why. |
| 66 | +5. Run each TLC config on the next fix commit. |
| 67 | + |
| 68 | +Count the "covered", "added" and "gap" results in the header. Count the findings about the spec files too. A maintainer uses the counts to decide scope. |
| 69 | + |
| 70 | +## Simplify questions |
| 71 | + |
| 72 | +1. Is a variable always equal to a function of other variables? If yes, derive it. Do not store it. |
| 73 | +2. Do two actions have the same effect? If yes, merge them. |
| 74 | +3. Does a new variable multiply the state count? If yes, remove it or prove it is needed. |
| 75 | +4. Do two parts of the code or of this folder repeat the same rule? If yes, keep the rule in one place. |
| 76 | + |
| 77 | +## File layout |
| 78 | + |
| 79 | +1. Name each spec after its subsystem. Example: `UrlRedaction.tla`. |
| 80 | +2. Name the config that must pass `Spec.cfg`. Name each other config `Spec.Variant.cfg`, for example a sabotage run that must fail. |
| 81 | +3. Keep every module in this folder. TLC finds the modules there. |
| 82 | +4. Put shared operators in `Common.tla`. Extract an operator when a second spec needs it, not before. |
| 83 | +5. Use `EXTENDS` to split one spec. Use `INSTANCE` for a module that other specs reuse. |
| 84 | +6. Put model-checking settings in an `MC` module. Keep the spec free of them. |
| 85 | +7. Keep each spec under 300 lines. If a spec is longer, split it by subsystem. |
| 86 | +8. Start each spec with a comment. Name the functions, the failure, the invariant, the cost and the findings. |
| 87 | + |
| 88 | +Sources: [module structure](https://learning.tlapl.us/intro/module-structure/) and [MC file advice](https://discuss.tlapl.us/msg06726.html). |
| 89 | + |
| 90 | +## Run TLC |
| 91 | + |
| 92 | +1. Download `tla2tools.jar` to a folder outside the repo. Do not commit it. |
| 93 | +2. Run: `java -XX:+UseParallelGC -cp <jar> tlc2.TLC -metadir <folder outside the repo> -config Spec.Variant.cfg Spec.tla`. Without `-metadir`, TLC writes a `states` folder here. |
| 94 | +3. If the system can end, add `CHECK_DEADLOCK FALSE` to the config. Without it, TLC reports the end as a deadlock. |
| 95 | +4. Pick the smallest constants that show the interaction. Two links are enough for interleaving. |
| 96 | + |
| 97 | +## Rules for this folder |
| 98 | + |
| 99 | +1. Write prose and comments in ASD-STE100. Keep each comment to one or two lines. |
| 100 | +2. The repo has TypeScript model checks. Run `pnpm lifecycle:model-check` for lifecycle code. Read `docs/architecture/task-lifecycle-model.md` first. Do not replace these checks with a spec. |
| 101 | +3. Follow the root `AGENTS.md` for tests and lint. |
0 commit comments