Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,9 @@ Settle first: existing non-symlink files at those paths.

## 3. Hooks

Outcome, Claude Code, in `~/.claude/settings.json` under `hooks`: PreToolUse matcher `Write|Edit|NotebookEdit` runs `aai-hook claude-block-native-edit`; PreToolUse matcher `Bash` runs `aai-hook claude-bash-guard`; UserPromptSubmit runs `aai-hook claude-prompt-submit`; SessionStart runs `aai-hook claude-session-start`; UserPromptSubmit, MessageDisplay, and PostToolBatch each also run `aai-hook claude-air` (the come-up-for-air nudge: after 8 tool-call rounds with no text response of 100+ chars, it injects a reminder to surface and reassess, repeating every 5 further rounds). The air nudge is Claude-only: codex has no message-level hook event, so it cannot observe the "text happened" reset condition - the codex-shaped substitute is a sentence in AGENTS.md; revisit if codex grows one. PostToolBatch and Stop also each run `aai-hook claude-drop-sentinel`, a Python port of podlayer/message-drop-sentinel (MIT): it detects the thinking-sandwich message-drop platform bug from the transcript scar (two adjacent thinking blocks) and tells the agent its text was probably eaten: restate it in the turn-final message, or say it now and end the turn if the user needs it immediately. Retire the sentinel entries when the upstream bug is fixed (re-test recipe and issue links in that repo's README). UserPromptSubmit and MessageDisplay also each run `aai-hook claude-slop`: MessageDisplay buffers each displayed assistant message, and at the next prompt the hook scores the previous turn's final message with the `slopometer` CLI, injecting the flagged patterns as context. A prompt that is a bare `;` means the user did not understand the previous reply, and the hook injects an instruction to restate it in plain English. Bare `aai-hook` resolves because the user's shell profile puts the workspace venv on PATH; if it does not, use the absolute venv path.
Outcome, Claude Code, in `~/.claude/settings.json` under `hooks`: PreToolUse matcher `Write|Edit|NotebookEdit` runs `aai-hook claude-block-native-edit`; PreToolUse matcher `Bash` runs `aai-hook claude-bash-guard`; UserPromptSubmit runs `aai-hook claude-prompt-submit`; SessionStart runs `aai-hook claude-session-start`; UserPromptSubmit, MessageDisplay, and PostToolBatch each also run `aai-hook claude-air` (the come-up-for-air nudge: after 8 tool-call rounds with no text response of 100+ chars, it injects a reminder to surface and reassess, repeating every 5 further rounds). The air nudge is Claude-only: codex has no message-level hook event, so it cannot observe the "text happened" reset condition - the codex-shaped substitute is a sentence in AGENTS.md; revisit if codex grows one. PostToolBatch and Stop also each run `aai-hook claude-drop-sentinel`, a Python port of podlayer/message-drop-sentinel (MIT): it detects the thinking-sandwich message-drop platform bug from the transcript scar (two adjacent thinking blocks) and tells the agent its text was probably eaten: restate it in the turn-final message, or say it now and end the turn if the user needs it immediately. Retire the sentinel entries when the upstream bug is fixed (re-test recipe and issue links in that repo's README). UserPromptSubmit and MessageDisplay also each run `aai-hook claude-slop`: MessageDisplay buffers each displayed assistant message, and at the next prompt the hook scores the previous turn's final message with the `slopometer` CLI, injecting the flagged patterns as context. A bare `;` means the user did not understand the previous reply, so the hook asks the agent to restate it in plain English. A bare `'` asks the agent to say whether its closing caveat was a real issue or empty hedging. Bare `aai-hook` resolves because the user's shell profile puts the workspace venv on PATH; if it does not, use the absolute venv path.

Outcome, kernel-centric codex, in `~/.codex/hooks.json`: PostCompact, SessionStart with matcher `compact`, and PreToolUse with matcher `mcp__clikernel__execute` each run `<venv>/bin/aai-hook codex-orientation`; UserPromptSubmit runs `<venv>/bin/aai-hook codex-prompt-submit`. Hybrid codex does not install `codex-orientation`, since it does not run the dojo; it may still install `codex-prompt-submit`. codex asks the user to trust hooks on the first start after any `hooks.json` change; tell them to expect that prompt.
Outcome, kernel-centric codex, in `~/.codex/hooks.json`: PostCompact, SessionStart with matcher `compact`, and PreToolUse with matcher `mcp__clikernel__execute` each run `<venv>/bin/aai-hook codex-orientation`; UserPromptSubmit runs `<venv>/bin/aai-hook codex-prompt-submit`; Stop and UserPromptSubmit each run `<venv>/bin/aai-hook codex-slop`. Hybrid codex does not install `codex-orientation`, since it does not run the dojo; it may still install `codex-prompt-submit` and `codex-slop`. The Stop hook stores `last_assistant_message`. On the next prompt, `codex-slop` scores that text and handles the same bare `;` and `'` signals as Claude. codex asks the user to trust hooks on the first start after any `hooks.json` change; tell them to expect that prompt.

Check: `aai-hook claude-prompt-submit` fed `{"prompt": "test?"}` on stdin prints the question notice.

Expand Down
49 changes: 34 additions & 15 deletions aai_coding/harness.py
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,11 @@ def _slop_state(f):
if not isinstance(st, dict): st = {}
return {k: st.get(k, d) for k, d in _SLOP_KEYS.items()}

def _slop_save(f, st):
tmp = f.with_suffix(f'.{os.getpid()}.tmp')
tmp.write_text(json.dumps(st))
tmp.replace(f)

def _slop_report(txt):
"Zero or one scored-message notices for `txt`, applying the env-tunable thresholds"
if len(txt.split()) < int(os.environ.get('SLOP_WORDS', SLOP_WORDS)): return []
Expand All @@ -258,6 +263,19 @@ def row(f):
return [SLOP_MSG.format(d=j['density'], t=dens_min, w=j['worst'], rows=rows)]


def _slop_prompt(o, f, st):
notes = []
if (o.get('prompt') or '').strip() == ';': notes.append(SLOP_RESTATE)
if (o.get('prompt') or '').strip() == "'": notes.append(SLOP_CAVEAT)
txt, fresh = st['last'], st['lastmid'] != st['done']
if txt and fresh:
st['done'] = st['lastmid']
_slop_save(f, st)
notes += _slop_report(txt)
if notes: print(json.dumps(dict(hookSpecificOutput=dict(
hookEventName='UserPromptSubmit', additionalContext='\n'.join(notes)))))


def claude_slop(o):
"MessageDisplay/UserPromptSubmit: track the displaying message, then report the previous turn's score with the new prompt"
try:
Expand All @@ -268,22 +286,23 @@ def claude_slop(o):
if o.get('message_id') != st['mid']: st.update(mid=o.get('message_id'), buf='')
st['buf'] += o.get('delta') or ''
if o.get('final'): st['last'], st['lastmid'] = st['buf'], st['mid']
tmp = f.with_suffix(f'.{os.getpid()}.tmp')
tmp.write_text(json.dumps(st))
tmp.replace(f)
_slop_save(f, st)
return
_slop_prompt(o, f, st)
except Exception as e: print(f'[slop] fail-open: {e!r}', file=sys.stderr)


def codex_slop(o):
"Stop/UserPromptSubmit: store the final assistant message, then report its score with the next prompt"
try:
f = _state_file('slop', o.get('session_id', ''))
st = _slop_state(f)
if o['hook_event_name'] == 'Stop':
st['last'], st['lastmid'] = o.get('last_assistant_message') or '', o.get('turn_id') or ''
_slop_save(f, st)
print('{}')
return
notes = []
if (o.get('prompt') or '').strip() == ';': notes.append(SLOP_RESTATE)
if (o.get('prompt') or '').strip() == "'": notes.append(SLOP_CAVEAT)
txt, fresh = st['last'], st['lastmid'] != st['done']
if txt and fresh:
st['done'] = st['lastmid']
tmp = f.with_suffix(f'.{os.getpid()}.tmp')
tmp.write_text(json.dumps(st))
tmp.replace(f)
notes += _slop_report(txt)
if notes: print(json.dumps(dict(hookSpecificOutput=dict(
hookEventName='UserPromptSubmit', additionalContext='\n'.join(notes)))))
_slop_prompt(o, f, st)
except Exception as e: print(f'[slop] fail-open: {e!r}', file=sys.stderr)
def codex_orientation(o):
"codex PostCompact/SessionStart/PreToolUse: post-compaction doc-state reset and one-shot reorientation"
Expand Down
26 changes: 26 additions & 0 deletions tests/test_harness.py
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,32 @@ def out(): return capsys.readouterr().out
assert 'caveat' in ctx and 'did not understand' not in ctx # bare ' asks whether the closing caveat is real


@pytest.mark.skipif(not which('slopometer'), reason='slopometer not installed')
def test_codex_slop(tmp_path, monkeypatch, capsys):
"Codex Stop captures the final reply; the next prompt reports it once and handles the punctuation notices"
from aai_coding.harness import codex_slop
monkeypatch.setenv('LLMDOJO_STATE_DIR', str(tmp_path))
def stop(tid, txt): codex_slop(dict(hook_event_name='Stop', session_id='s1', turn_id=tid, last_assistant_message=txt))
def psub(prompt='hi'): codex_slop(dict(hook_event_name='UserPromptSubmit', session_id='s1', prompt=prompt))
def out(): return capsys.readouterr().out
sloppy = "This isn't just a linter - it's a comprehensive paradigm that will streamline your workflow. " * 3
stop('t1', sloppy)
assert json.loads(out()) == {}
psub()
ctx = json.loads(out())['hookSpecificOutput']['additionalContext']
assert 'previous turn' in ctx and 'splice' in ctx
psub()
assert out() == ''
stop('t2', 'The parser rejects malformed input. ' * 10)
assert json.loads(out()) == {}
psub(';')
ctx = json.loads(out())['hookSpecificOutput']['additionalContext']
assert 'did not understand' in ctx and 'previous turn' not in ctx
psub("'")
ctx = json.loads(out())['hookSpecificOutput']['additionalContext']
assert 'caveat' in ctx and 'did not understand' not in ctx


def test_synthetic_wipe_guard(tmp_path, monkeypatch):
import llmdojo.rules as lr
monkeypatch.setenv('LLMDOJO_STATE_DIR', str(tmp_path/'state'))
Expand Down