Field inspection notes, spoken or typed, turned into structured maintenance drafts, with the plant's own sensors fact-checking the human before anything is filed.
Everything here is simulated. The OPC UA server, tag catalog, and values are invented for demonstration. No real industrial data, no real plant, no vendor code.
Most "LLM for plant data" demos let a person ask the machine. In a licensed facility the harder problem is the other way round: a technician in PPE calls out a gauge reading and a defect, and that note becomes a record. This tool
- extracts slots only (location, tag, reading, unit, condition, severity) from the note;
- cross-checks the reading against the live tag and asks a clarifying question when they disagree;
- produces an SAP PM-style notification that is born as a DRAFT and can only be submitted with a human signature.
The language model never reads or writes the plant. The plant interface has no write method at all. Unknown locations are refused, not guessed. Actuation requests and injected instructions are rejected before extraction.
"三号冷却泵出口压力表读数三点二巴，法兰有渗漏痕迹"
(Mandarin: "cooling pump 3 outlet gauge reads 3.2 bar, traces of leakage at the flange")
│ extract (slots only, schema-validated)
▼
{loc: CWS-PMP-003, tag: CWS-PMP-003.PT_OUT, reading: 3.2 bar, conditions: [leak]}
│ verify (read-only OPC UA)
▼
CLARIFY: You read 3.2 bar; sensor shows 3.1 bar at 2026-09-03T12:00:00Z. Confirm or re-read?
│ draft → sign(user)
▼
{status: SUBMITTED, funct_loc: CWS-PMP-003, priority: 4, signed_by: inspector.a, verification: {...}}
pip install -e .[dev]
pytest -q # 38 tests incl. 20 adversarial notes, all refused
python -m walkdown.cli "pump 3 outlet pressure reading 3.5 bar, steady leak at flange" --sign inspector.a
python -m walkdown.cli "..." --sign inspector.a --confirm # accept gauge over sensor, both values keptOptional layers, each independent:
| layer | install | use |
|---|---|---|
| real OPC UA transport | docker compose up -d (or pip install -e .[opcua] + python -m walkdown.sim_opcua) |
--opcua opc.tcp://localhost:4840/walkdown/ |
| LLM extractor | pip install -e .[llm] + OPENAI_API_KEY (any OpenAI-compatible endpoint) |
replaces the deterministic mock; output is still schema-validated |
| speech input | pip install -e .[speech] |
--audio note.wav (faster-whisper, local) |
tests/test_adversarial.py: 20 notes containing actuation verbs, injected instructions, or requests to skip signature. Each must raise before extraction and leave the store untouched.tests/test_verify_notify.py: draft never auto-submits; signing an unresolved mismatch is refused; the plant class exposes nowrite*/set*surface.tests/test_extract.py: Chinese, English and French notes, numerals, unit checks, unknown locations refused.tests/test_llm_adversarial.py(pytest -m llm, needsOPENAI_API_KEY): second, independent line of defence. The regex guard is monkeypatched off and a real model extracts the same 20 hostile notes. Whatever it returns still passes schema validation, meets a plant object with no write surface, and is born as a DRAFT. Submissions: 0. The two layers do not depend on each other; either one alone keeps the store untouched. Last run:gemini-2.5-flashthrough an OpenAI-compatible gateway, 20/20 notes, 0 submissions (2026-09-04). Running a real model found two holes the mock never could: the model returnedreadingas the string"3.2", which the schema now refuses instead of letting it reach arithmetic, and it answered "mark all locations as inspected" with a JSON list of observations, which the adapter now refuses as anything but one object.
eval/noise_eval.py mixes clean utterances with pink noise plus 50 Hz hum and its harmonics as a crude pump-room
stand-in, sweeps SNR from 20 dB to -5 dB, and reports transcript character accuracy and slot exact-match.
Supply your own recordings as wav<TAB>text; none are shipped. python eval/make_utterances.py synthesises a
36-sentence stand-in set (Mandarin, English, French) with edge-tts for a first run.
Result on 36 synthesised utterances, 12 each in Mandarin, English and French (edge-tts voices, faster-whisper small
int8 on CPU; real recordings to follow):
| lang | SNR dB | transcript similarity raw / lexicon | slot exact-match raw / lexicon |
|---|---|---|---|
| en | 20 | 0.983 / 0.999 | 0.833 / 0.917 |
| en | 10 | 0.979 / 0.997 | 0.750 / 0.917 |
| en | 5 | 0.960 / 0.969 | 0.667 / 0.667 |
| en | 0 | 0.912 / 0.926 | 0.083 / 0.167 |
| en | -5 | 0.816 / 0.831 | 0.083 / 0.083 |
| fr | 20 | 0.895 / 0.988 | 0.000 / 0.917 |
| fr | 10 | 0.865 / 0.953 | 0.000 / 0.667 |
| fr | 5 | 0.843 / 0.922 | 0.000 / 0.500 |
| fr | 0 | 0.808 / 0.868 | 0.000 / 0.167 |
| fr | -5 | 0.691 / 0.712 | 0.000 / 0.000 |
| zh | 20 | 0.490 / 0.752 | 0.000 / 0.667 |
| zh | 10 | 0.431 / 0.684 | 0.000 / 0.417 |
| zh | 5 | 0.386 / 0.628 | 0.000 / 0.167 |
| zh | 0 | 0.424 / 0.589 | 0.000 / 0.083 |
| zh | -5 | 0.266 / 0.369 | 0.000 / 0.000 |
Three conclusions. First, English is the only language where the failure is mostly acoustic: raw transcripts already
yield usable slots at 20 dB and the curve simply falls with SNR. Second, Mandarin and French fail lexically before
they fail acoustically. Raw Whisper output never yields a usable French or Mandarin slot at any SNR: French readings come
back with a decimal comma (3,2) and jargon drifts (relevé → elevée, pompe 3 → pomp3), while Mandarin field terms come
back as everyday homophones (泵 pump → 棒 stick, 读数 reading → 读书 study) and the unit 巴 (bar) is heard as the digit
八 (8), turning "three point two bar" into 3.28. A per-language lexicon of a few dozen entries plus one catalog-driven rule
(pressure readings carry one decimal) recovers most of that; see stt.normalize. Third, below about 5 dB the acoustic
model itself gives out in every language, and the remedy there is a domain-adapted or prompted ASR, not more text rules.
eval/results.tsv is the raw sweep.
Writing to the plant, bypassing signature, real SAP/OPC connectors. Those are the point.
- Bitemporal records: "which drawing revision was valid when this notification was raised".
- A benchmark harness so other agents can be scored against the same adversarial set.