Repository navigation
Commit 2908dee
[staging] Cloud login, agent skills, SQLite-optional collector, and Cloud Jev policies (#933)
* [failproofai-config-account-login] Log in to FailproofAI Cloud from `failproofai config`, and give every agent the failproofai skill (#920)
* [failproofai-config-account-login] Log in to FailproofAI Cloud from `failproofai config`
The CONNECT step opens with a menu: log in through the browser (default),
paste an API key, or use open source. Logging in opens a link (or prints
it over SSH / with no display) where a person signs up or in, picks or
creates an org and authorizes the machine; the CLI polls with a secret
only it holds and shows who approved it, for which org. The key is
claimed at APPLY, after the daemon installs, so an abandoned run mints
nothing. A connected machine is offered "keep" first, or "log in again"
first when its saved key no longer works.
- src/hooks/cloud-login.ts: start / poll / claim against
/api/auth/cli/*, injectable fetch, no redirects followed, only http(s)
links ever handed to the opener; older clouds read as unsupported
- tui promptBrowserLogin: link printed above the live region (never cut),
esc back, ctrl-c cancel, expiry/deny offer a new link, keeps polling
through a dropped connection
- configure-wizard: Connect menu, claim at apply, re-login note, sudo
re-asked at apply when the cache expired during the login
- open-browser: canOpenBrowser (false over SSH / no display), openUrl
exported
- e2e harness __tests__/e2e/layout/cli-login.sh + pty driver
- docs: quickstart and CLI reference lead with the login
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Changelog: PR number for CLI login (#920)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [config-account-login] Match the simpler authorize page; re-ask sudo before a unit refresh
- The terminal now says "Not authorized in the browser." when the person picks
[ don't authorize ] on the page (was "The request was cancelled in the
browser."), matching the page's vocabulary; the e2e harness waits for it.
- The quickstart no longer asks people to check the key's name and
permissions: the page stopped showing them, the server sets both.
- The e2e harness approves without a `jev` flag, exactly as the page does now,
and still asserts the key gets jev:evaluate because the approver holds it.
- CodeRabbit: the sudo re-ask before apply also covers refreshing a stale
service definition; failing it there costs the scheduled audit (caution),
never the setup. The reference doc no longer says setup "never" prompts for
a password: a terminal run does, and asks again if it expired.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [config-account-login] Start the login by itself; harden what the server sends
Connect to cloud on a machine with no connection (or a refused saved key)
now starts the browser login straight away — link generated, browser opened
or link printed, countdown running. Tab or Esc there opens the other
options; Ctrl-C cancels. A machine connected with a working key still gets
the menu, keep first, so re-running config never opens a browser. An older
cloud drops straight into the options without the login entry. Leaving the
waiting screen erases the link it printed (rows counted at the current
width), so no dead link sits above the menu.
Deep-scan fixes:
- Everything the server sends is printed as text only: org name and slug,
approver, key name, errors (controls, bidi overrides and zero-width
characters stripped — an org named with escape codes cleared the screen
and retitled the terminal mid-setup), and the link as the URL parser
serialises it. Same for the introspect path's "Connected as".
- The server's poll interval and expiry are clamped (an interval past 2^31
ms made setTimeout fire every millisecond); slow_down never speeds up.
- A saved cloud address is validated like a typed one.
- A key that was created but could not be put to use is named, so it can
be revoked; a "log in again" that created no key reports the previous
connection, not open source.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [config-account-login] A failed claim on a fresh machine is not "kept"
savedConnection() returns null, not undefined, so `saved !== undefined` was
always true: a fresh machine whose login claim failed printed "This machine
keeps its previous connection." and reported mode "cloud" with no credential
on disk. Truthiness, like every other check of `saved` here. The fresh-machine
claim-failure test now asserts mode "oss" and the absence of that line
(CodeRabbit, PR #920).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [config-account-login] Release as 1.0.12-beta.1
1.0.12-beta.0 is already on npm (published from this branch on `next`, built
from a42e75f), so publish preflight would refuse it from main. Move
package.json, the Cargo workspace (+ Cargo.lock) and the Hermes plugin
manifest to 1.0.12-beta.1, and give it its own changelog section with the one
change the published beta.0 lacks (b7aaddd).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [config-account-login] A refused saved key is not kept as if it worked
When the saved key was refused and the login that followed created no key,
the run ended with "This machine keeps its previous connection." The
connection is still kept (the CONNECT step never removes one, decision D4,
same as picking "keep" with a refused key), but the line now says its key no
longer works and to run config again. The verdict from the CONNECT step is
carried to the end as `savedRefused`. Regression test fails without the fix
(CodeRabbit, PR #920).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [config-account-login] Close the remaining CodeRabbit / Hermes findings
- checkCloudKey validates the address before sending a key to it: a saved
http:// address to another host no longer receives the saved key when
config checks it (CodeRabbit, outside-diff, b7aaddd). Five older tests
used the placeholder address "u", which the guard now refuses; they use the
valid https address their neighbours already use.
- configure_applied is sent after the cloud step with the final `connected`
and mode, so a login whose claim failed is not counted as connected
(CodeRabbit + Hermes F4).
- promptBrowserLogin: a retry erases the expired/refused link (and its
heading) before printing the new one — `forgetAbove()`, shared with leaving
on Tab/Esc (CodeRabbit + Hermes F3).
- startLogin: a 200 that is not JSON (the web app serving a page in the
route's place) is `unsupported`, so the key prompt is offered (CodeRabbit).
- connectToCloud takes `org` from the caller when introspect gives none; the
wizard passes the org from the login's claim, so the saved org is not left
empty or stale (CodeRabbit).
Each new test fails without its fix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [config-account-login] Release as 1.0.12 (stable)
1.0.12-beta.1 was published to npm on `next` from another branch
(feat/cloud-jev-on-main) before this PR took that number, so publishing it
from main would be refused by preflight. Cut the stable release instead:
1.0.12 across package.json, Cargo.toml/Cargo.lock and hermes-plugin/plugin.yaml,
and the top changelog section becomes `## 1.0.12`, which the stable
release-notes check and the Discord announcement read along with every
`1.0.12-beta.*` section below it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [config-account-login] Give every agent the failproofai skill
`config` now writes one small `failproofai/SKILL.md` into each agent it finds
(and any ticked), in that agent's own user-scope skills folder. The file is a
pointer: it sends the agent to the umbrella skill and its five siblings in the
public FailproofAI/skills repo, so the content is read live and never goes
stale on disk.
Every folder was verified against the real CLI in Docker (probe skills, the
CLI's own listing or a mock-model capture), then re-checked end to end with
this code for Claude Code, Codex, Copilot, OpenCode, OpenClaw, Devin and Goose:
- one folder per agent, following CLAUDE_CONFIG_DIR, CODEX_HOME, COPILOT_HOME,
XDG_CONFIG_HOME, PI_CODING_AGENT_DIR, FACTORY_HOME_OVERRIDE and
GOOSE_PATH_ROOT; every Hermes and OpenClaw profile gets its own copy;
- never the shared ~/.agents/skills (Claude ignores it, Codex lists dupes,
Copilot drops it under COPILOT_HOME, OpenClaw lets it outrank its own);
- name = folder name, description under 200 chars and led by its subject
(Copilot drops >1024, droid shows 200, Hermes 57).
A `.failproofai-managed` marker marks folders failproofai wrote: one without it
(e.g. `npx skills add FailproofAI/skills`) is never overwritten, and
`failproofai uninstall` removes only marked folders. A folder that can't be
written is logged and skipped; setup never fails on it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [config-account-login] Add the failproofai skill silently
Drop the "Added the failproofai skill to N agents." line from config's
summary; the summary stays about enforcement and the connection.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [config-account-login] Don't give Codex the failproofai skill twice
Codex reads ~/.agents/skills as well as its own ~/.codex/skills and lists a
name found in both twice (0.164.0). `npx skills add FailproofAI/skills -g`
puts the full skill in ~/.agents/skills for Codex, so a machine with that and
`failproofai config` showed Codex two `failproofai` skills.
When ~/.agents/skills/failproofai exists, config now leaves Codex to it: it
writes no Codex copy, and removes one it wrote on an earlier run. Checked in
Docker against real Codex: npx then config lists it once; config then npx
lists it twice until config runs again, which takes ours back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [config-account-login] Never stop setup on removing an old Codex skill copy
Taking back our Codex copy (when ~/.agents/skills already serves Codex) called
rmSync outside any try, so a folder that could not be removed would throw out
of installAgentSkill and fail `config`, breaking its never-fails-setup
contract. Route it, and uninstall's removal, through one removeSkillDir that
logs and carries on.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [config-account-login] Report the kept connection when a new login's key can't be used
Logging in again on a connected machine, with the key claimed but refused
on both streams, made connectToCloud write nothing, so the previous
connection stayed on disk and working. `keptOld` tested `!token`, though, so
the summary and configure_applied reported the machine as open source.
Test `!connected` instead. A refused saved key still counts as kept (mode
cloud, decision D4), with its own message. (CodeRabbit)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* ci: test pull requests targeting staging
* fix(deps): clear staging supply-chain findings
* fix(security): scope Python 3.10 filelock exception
* [failproofai-feat-fpai-collect-optional-sqlite] fpai-collect: SQLite pollers behind a default-on `sqlite` feature (#929)
* fpai-collect: SQLite pollers behind a default-on `sqlite` feature
AgentEye's server imports uploaded Claude Code and Codex transcripts with
fpai-collect's own converters and redaction, so an imported session is shaped
exactly like a captured one. It cannot depend on the crate as it stands:
rusqlite 0.40 links `sqlite3` through libsqlite3-sys 0.38, the server's sqlx
links it through 0.30, and Cargo allows one crate per native library.
- `rusqlite` is optional behind `sqlite` (default on). Gated: `sqlitepoll`,
the goose/opencode/hermes/devin sources and `openclaw::sqlite`.
- `to_rfc3339_micros` moves from goose to `sources` (goose re-exports it), so
the cursor file source keeps it without the feature.
- The seven integration tests that build real SQLite databases declare
`required-features = ["sqlite"]`.
- CI lints the crate without the feature too; the workspace build always has
it on, so nothing else would notice that shape breaking.
failproofaid builds with the feature and is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* CHANGELOG: PR number for the optional sqlite feature (#929)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix: address combined PR review findings
* [failproofai-feat-cloud-jev-on-main] Cloud-managed Jev and hybrid policies, scoped to agents (#928)
Squash of #928 (feat/cloud-jev-on-main @ 4e679a8) onto staging so main
gets one unified PR (#933).
Conflict resolutions:
- Versions stay at staging's 1.0.12 (package.json, Cargo.toml/lock,
hermes-plugin/plugin.yaml). The unpublished 1.0.12-beta.2 fixes move
into the 1.0.12 CHANGELOG section; the published 1.0.12-beta.1 section
is kept as-is.
- cloud-connection.ts: keep #928's same-org reconnect short-circuit, and
compare on staging's `org` (introspect, else the browser-login claim).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(jev): keep captured intent with the agent profile it was typed into
Review of #933 (High): Cloud Jev's tool-call review was already skipped for
a profile no Cloud Jev check targets, but intent capture still ran for every
profile and stored prompts under the session id alone. Session ids are the
harness's, so a non-targeted profile could submit a prompt under the same id
as a targeted one, and that prompt was then sent to Cloud Jev as the targeted
profile's stated intent, able to clear a reviewable `both` policy's regex
deny. It also kept prompt text from profiles the deployment did not target.
- The intent store is keyed by agent profile + session id:
sessions/<integration>[.<instanceId>]/<sessionId>.json. The profile is the
runtime AgentIdentity the handler resolves (the same one agent targeting
matches on), or the bare integration when none is rostered. Capture
(handler -> captureIntent profile) and review (startJevReview agent ->
readIntent profile) use the same identity; jev-review's reader requires it.
- Legacy flat sessions/<id>.json entries cannot say which profile wrote them,
so the hook path never reads them; they expire under the 6h sweep, which
now also sweeps profile directories. The loose-directory check covers the
profile directory too.
- Under a Cloud Jev mode, nothing is captured for a profile with no Cloud
Jev assignment and no installed pack check (startTwoTier's own skip,
minus the pause). A deployment without the assignment list captures as
before. Local (non-Cloud) Jev captures as before, per profile.
Tests: handler-level reproduction of the attack (non-targeted profile, same
session id, cannot clear the targeted profile's deny; also when an installed
pack makes the other profile's prompt kept; also a legacy flat entry), the
capture gate, and store-level profile isolation, sweep and permissions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(fp): port Cloud CLI parity from #885 onto staging
Applies the own delta of feat/fp-cli-parity (git diff feat/tui-ux...feat/fp-cli-parity)
on top of #928: evaluator authoring (code, LLM judge, Jev: draft, dry-run, deploy,
backfill), dashboards, sentiment, session export/history/re-evaluate, assistant
artifacts and approvals, account/integration controls, alert recipients/issues,
audit agent contracts and the mint/pink human output. JSON shapes unchanged.
Dropped the stacked-branch workflow (fp-cli-parity.yml, targeted feat/tui-ux) and its
test; ci.yml already runs the fp-cloud-cli suite. Root CHANGELOG keeps staging's text.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(fp): align ported commands with staging and fill dashboard gaps
- policies backtest: every-kind backtest for jev/both (published or --semantic),
estimate by default, --run/--sample to spend Jev allowance; regex path unchanged
- policies author --kind regex|jev|both
- issues list/count: inbox filters; new issues facets
- sessions import FILE... (POST /api/events/import)
- list trigger_events / eval_envs / metric_keys
- evaluation backfill covers judge/Jev (#1183 on the server); budget warning
- usage windows: API key only (no dashboard session route)
- issues close help: exit 1, not 9; point at audits dismiss
- changelog: fold the parity entries into the open 0.0.2b0 section
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(fp): document every fp command and flag; bring in the skills-side edits
- skill: new description (the skills-repo wording; evaluator design routes to
failproofai-eval-author; no pointer to the retired evaluator skill),
openai.yaml nested policy key, an alerts section, globals
--timeout/--quiet/--no-color, corrected --file and --fields lists,
session-only/key-only lists (policies test works with a key), fleet deploy
--target/--all-agents, issues filters/facets and close = exit 1, users create
EMAIL required, users delete/defaults, sessions import, policies
author/backtest kinds, backfill covers judge/Jev
- public cloud-cli reference and fp-cloud-cli README match
- sessions import --format failproofai (public name for the server's wire value)
- test fixtures use neutral session ids
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(changelog): note fp Cloud CLI parity under 1.0.12
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(config): the managed skill points at the eval, alert and audit authoring skills
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: chhhee10 <chetanraghuvanshi85@gmail.com>1 parent a62c6bf commit 2908dee
220 files changed
Lines changed: 34946 additions & 1170 deletions
File tree
- .github/workflows
- __tests__
- actions
- audit
- e2e
- helpers
- hooks
- layout
- fixtures
- cloud-jev
- generators
- hooks
- semantic
- app
- actions
- settings
- bin
- crates
- failproofaid/src
- fpai-collect
- src
- sources
- cursor
- goose
- hooks
- openclaw
- tests
- fixtures
- fpai-ipc/src
- docs
- policies
- reference
- start
- fp-cloud-cli
- fp_cli
- commands
- skill
- agents
- references
- tests
- hermes-plugin
- sdk/python
- src
- audit
- hooks
- semantic
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| |||
266 | 266 | | |
267 | 267 | | |
268 | 268 | | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
269 | 276 | | |
270 | 277 | | |
271 | 278 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
30 | | - | |
| 30 | + | |
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| |||
Large diffs are not rendered by default.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
50 | 50 | | |
51 | 51 | | |
52 | 52 | | |
53 | | - | |
| 53 | + | |
54 | 54 | | |
55 | | - | |
| 55 | + | |
56 | 56 | | |
57 | 57 | | |
58 | 58 | | |
| |||
89 | 89 | | |
90 | 90 | | |
91 | 91 | | |
92 | | - | |
| 92 | + | |
93 | 93 | | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
94 | 105 | | |
95 | 106 | | |
96 | 107 | | |
| |||
103 | 114 | | |
104 | 115 | | |
105 | 116 | | |
106 | | - | |
| 117 | + | |
107 | 118 | | |
108 | 119 | | |
109 | 120 | | |
| |||
126 | 137 | | |
127 | 138 | | |
128 | 139 | | |
129 | | - | |
| 140 | + | |
130 | 141 | | |
131 | | - | |
| 142 | + | |
132 | 143 | | |
133 | | - | |
134 | | - | |
| 144 | + | |
| 145 | + | |
135 | 146 | | |
136 | 147 | | |
137 | 148 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
367 | 367 | | |
368 | 368 | | |
369 | 369 | | |
370 | | - | |
| 370 | + | |
371 | 371 | | |
372 | 372 | | |
373 | 373 | | |
374 | | - | |
375 | | - | |
| 374 | + | |
| 375 | + | |
376 | 376 | | |
377 | 377 | | |
378 | 378 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
43 | | - | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
44 | 49 | | |
45 | 50 | | |
46 | 51 | | |
| |||
57 | 62 | | |
58 | 63 | | |
59 | 64 | | |
| 65 | + | |
60 | 66 | | |
61 | 67 | | |
62 | 68 | | |
| 69 | + | |
63 | 70 | | |
64 | 71 | | |
65 | 72 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
0 commit comments