Repository navigation
[pull] main from DeusData:main - #132
Merged
Merged
Conversation
`daemon start` exited nonzero when the daemon came up but did not answer the UI configuration handshake. By that point the daemon is running -- the control connection above it already satisfied the startup window -- so the command reported failure for something it had already achieved. The two requests in that handshake are bounded at MAIN_CONNECT_TIMEOUT_MS, one second. On a loaded machine, especially right after an abrupt shutdown, that is thin. Windows CI hit it repeatedly: the crash-recovery section of `tests/windows/test_daemon_stability.py` hard-kills a daemon and immediately starts a fresh one, and the report that came back was that no fresh daemon had started -- when one had, and was serving. Whether the refusal is fatal now depends on what was asked for. `--port` and `--open` make the UI the point of the command and keep the existing nonzero exit. A bare `daemon start` asks for a daemon and got one, so it now warns and succeeds; it also skips the UI-warming notice, which would otherwise announce a port that nothing is serving. A seam forces the refusal, because reproducing it for real needs a machine loaded enough to miss a bounded handshake -- not a state a test can ask for. The lifecycle guard covers both halves of the contract, and with the decision reverted it reports the bare-start case as red. Signed-off-by: 刘冲 <mail@liuchong.dev>
Adds -Wl,-z,relro and -Wl,-z,now to ELF_HARDENING_FLAGS, and asserts the
resulting property on the produced artifact rather than on the compiler's
willingness to accept a flag.
The shipped Linux binaries measurably lacked the property. Built the way
.github/workflows/_build.yml builds them (STATIC=1, gcc 13.3 / GNU ld 2.42,
Ubuntu 24.04 aarch64), PT_GNU_RELRO ends at 0x11DA0000 in both cases:
without -z now: .got 0x11d9ec20+0x13c8, .got.plt 0x11d9ffe8+0x40
-> .got.plt ends at 0x11DA0028, 40 bytes PAST the window.
Those GOT slots stayed writable for the whole process
lifetime, in every Linux binary shipped to date.
with -z now: .got.plt folds into .got, the whole GOT sits inside the
window and is re-mapped read-only after startup.
Both flags are free at runtime for a static binary -- everything resolves at
link time, so there is no lazy binding left to pay for. -z relro is already
this toolchain's default and changes nothing here; it is named anyway so the
property stops depending on one distro's spec file, since the musl/portable
and glibc-floor images are different toolchains and a default is not a
guarantee.
A1c/A1d in scripts/ci/check-binary-composition.sh assert the OUTCOME on the
binary via readelf, for the same reason A1 exists beside the .note.GNU-stack
annotation: a flag the linker accepts is not evidence the artifact gained
anything. PR #1138 demonstrated that directly -- its -pie was silently
discarded under -static, and its -D_FORTIFY_SOURCE=2 silently DOWNGRADED
Ubuntu's default of 3, both while every compile probe passed.
The readers use awk rather than a short-circuiting reader on purpose: a
reader that exits at the first match leaves the upstream taking EPIPE, and
under `set -o pipefail` the satisfied case is reported as the failing one.
Deliberately NOT taken from #1138: -D_FORTIFY_SOURCE=2 (a downgrade on the
release platform), -fPIE/-pie (inert under STATIC=1), -fstack-protector-strong
(real but unmeasured cost, and #1138 injected it into SQLite, tree-sitter and
mimalloc -- the indexing hot loop), and the $(shell) probe machinery (~10
compiler spawns per Makefile parse, making the binary a function of the
builder's toolchain).
Co-authored-by: PR9000 <119280965+PR9000@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…s by name Step 0o (MCPB bundle contract) and Step 0q (release candidate derivation) hand the composition gate a stub compiled straight from a .c file with the compiler's defaults. That stub exists to exercise packaging, format detection and the needle scans; it is not linked with -z now and never ships. A1d measured eager binding on it anyway, so on a toolchain that binds lazily by default (test-diag: clang-22) Step 0o died deterministically with A1d-bind-now stage/codebase-memory-mcp: GOT section(s) .got.plt fall outside PT_GNU_RELRO [15824,16384) — they stay WRITABLE after relocation The property A1d protects is a property of the release link (-z now in Makefile.cbm), so the assertion is scoped to it: a caller that feeds the gate a packaging fixture declares so with CBM_COMPOSITION_FIXTURE=1 and A1d is reported n/a — printed on every run as a named exemption, never a silent skip. Every other assertion still runs on the fixture unchanged, and release derivation never sets the variable. Verified in the Linux arm64 container (Ubuntu 24.04, gcc 13.3): a stub linked -z lazy reproduces the CI failure text on the gate without the variable and reports n/a with it; both contracts pass under a CC wrapper that links lazily (the shape that was red), and unchanged on macOS. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
fix(build): put the whole GOT inside the RELRO window on ELF targets
…onfig fix(cli): a refused UI configuration is not a failed daemon start
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> # Conflicts: # tests/test_cli.c
Tool definitions no longer carry a separate title (the name is the MCP display fallback), so the accessor main added for the pi adapter has no field to read. The adapter already used the name when no title existed. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Fix-forward after rebasing onto main (545 commits, 26 conflicted-file resolutions). Where main had shipped a behaviour in the meantime, main wins; the PR keeps its output contract on top of it: - search_code: the scan runs under main's supervised, deadline- and output-bounded runner and feeds the branch's streaming classifier from the runner's output file; the POSIX wrappers map grep's no-match to 0, so every non-zero exit fails closed (no exit-1 special case). - trace_path: strategy/confidence precede args in header and rows (#1542), in both the tree table and the json legs. - cypher: relationship expansion is never capped (a cap falsified aggregates) and unlabeled MATCH scans every candidate; `truncated` is raised only by a real ceiling — a clamped hop range is probed one hop beyond the cap and reports truncated only when a candidate exists there. The plain BFS and the trail BFS both probe one row past max_results so saturation is observable instead of silent. - search_code full mode and the final json use main's U+FFFD sanitizer instead of '?' replacement. - search_graph json keeps an empty `groups` array for semantic-only calls; list_projects json echoes offset/limit next to next_offset and accepts main's include_details spelling for the stats projection. - discovery budget: main added get_file_outline, compare_graphs, manage_adr set_sections and the debug/include_details parameters; their descriptions are trimmed to the lean style and the ceiling moves from 15 to 18 KiB. Tests that asserted a JSON default now request format:"json"; the fail-closed search test uses an unreadable regular file (a missing or non-regular indexed path is deliberately not a scan operand on main); the non-regular fixture gives its real sources File nodes, as every indexed project has; list_projects total counts listable projects (the 0-byte ghost is never a row, so it is not a page either). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
- test_daemon_lifecycle.py: daemon coordination is account-wide and CBM_CACHE_DIR does not move the rendezvous (main's CBM_RUNTIME_DIR relocation), so a private CBM_RUNTIME_DIR per cache is what makes the --verbose call cold and lets it observe the spawn hint. - test_hook_augment.py: the CLI answers in the lean tree form by default; the guard parses the JSON projection, so it asks for it. - test_mcp.c: the fail-closed search test skipped on Windows past the teardown that names its POSIX-only response. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The hint moved into a structured info record when one-shot CLI output went quiet by default. Structured values escape spaces, so the record read `daemon_start` and a person (and the Windows lifecycle guard) could not find `daemon start` in it. Keep the quiet default and, when --verbose or CBM_LOG_LEVEL=info opts in, print the same plain stderr hint the CLI printed before. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The unreadable-operand scenario used mode-0 permission bits, which Windows cannot express and root ignores (the Colima legs run as root), so the test skipped on both — and bare SKIP() is forbidden. The scoped file list deliberately drops every missing or non-regular indexed path before grep sees it, so a refusal can only come from the scan itself. The scan seam now delivers one plausible hit and then grep's error status, on every platform, and the test asserts that neither that hit nor zero matches is ever reported as a complete page. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…efault The lean-output branch starts the raw logger at WARN so allocator setup cannot pollute protocol or CLI output before argv is classified. The index-format boundary's route record, `pipeline.route path=format_change_reindex`, was an INFO record, so a process that never applies the role policy (the test runner, an embedder) and every thin frontend at its default lost the only log-side evidence that an index was rebuilt once because its format changed (#1104) — the response's `format_migration:true` was unaffected. Log the migration route at WARN, like the sibling `invalid_existing_db` deviation: a once-per-upgrade full rebuild is exactly what explains a long index and a cursor reset. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The lean-output branch had lowered the logger's raw default from INFO to WARN so allocator setup could not pollute protocol or CLI output before argv is classified. That moved a process policy into the library: any process that never applies the role policy — the test runner, an embedder — silently lost every INFO record, and tests that observe INFO records through the log sink (the index-format boundary's `pipeline.route path=format_change_reindex`, the importance pass counters) went red for a contract that was in fact intact. Restore the INFO library default and leave the quiet default where the branch applies it, in cbm_log_init_for_process at process startup: thin frontends go quiet, detached daemons keep INFO, physical workers require it. The format-migration route record returns to INFO as on main; the previous commit's WARN promotion was the wrong layer for this fix. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…fault With the library default back at INFO, the branch's log and progress-sink tests were still handing the process a WARN level on their way out — main's `/* restore */` lines had been rewritten to WARN — so every suite that ran after them in the same process lost its INFO records (the CI shard that runs `log` before `index_format` is exactly that order). test_log.c keeps main's tests and restores verbatim; the two branch tests (the CBM_LOG_LEVEL opt-in and the startup policy) save the level on entry and hand it back. The progress-sink tests pin the quiet level they exercise explicitly and restore it, instead of assuming the process entered at WARN. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
feat: make CLI and MCP output lean by default
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )