Skip to content

Commit 9187a42

Browse files
VibeSWEclaude
andcommitted
Merge remote-tracking branch 'origin/main' into fix/apikey-role-escalation
Resolve conflicts with #60: keep both the privilege filter tests and the role escalation tests in apikeys.rs, take main's middleware test setup, and have the role escalation tests hold map::TEST_SERIAL via #60's helpers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2 parents 8fd7930 + 35d8bbb commit 9187a42

31 files changed

Lines changed: 9105 additions & 581 deletions

‎Cargo.lock‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,7 @@ tracing-actix-web = "0.7"
139139

140140
# Time and Date
141141
chrono = "0.4"
142+
chrono-tz = "0.10"
142143
chrono-humanize = "0.2"
143144
humantime = "2.1.0"
144145
humantime-serde = "1.1"

‎docs/tool-api.md‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,11 @@ Response:
3636
}
3737
```
3838

39+
A listed `inputSchema` never has a root-level `oneOf`, `anyOf` or `allOf`,
40+
because several LLM tool APIs reject tool input schemas with root combinators.
41+
Where the catalog schema has one, the server still enforces it when the tool is
42+
called, and the affected property descriptions state the constraint.
43+
3944
The response includes an `ETag` and
4045
`Cache-Control: private, max-age=300, must-revalidate`. A request with a
4146
matching `If-None-Match` receives `304 Not Modified`.
@@ -130,3 +135,144 @@ and last-seen value, and coalesces supported aliases per row. Its `total` is the
130135
number of returned groups, not an unbounded count of all matching groups. The
131136
native Errors page additionally separates issues by message, so its grouping
132137
and pagination are not identical to this summary tool.
138+
139+
## SQL tool limits
140+
141+
`query_sql`, `explain_query` and other tools that run SQL internally parse the
142+
statement with DataFusion's SQL parser and accept exactly one `SELECT` or `WITH`
143+
query. Trailing semicolons and trailing comments are allowed, including
144+
comments after the final semicolon (`SELECT a FROM t; -- note`).
145+
The outermost projection must name at most 10 explicit columns, and `*` and
146+
`table.*` are rejected there. The same limit applies to each branch of a
147+
top-level set operation such as `UNION` and to a parenthesized outer query.
148+
Projections inside CTE definitions, derived tables and subqueries are not
149+
checked. Casts to `INT64`, `UINT64` or `FLOAT64` anywhere in the statement are
150+
rejected with a hint to use `BIGINT` or `DOUBLE`. The statement is wrapped with
151+
an outer `LIMIT 500`, except in `get_trace_activity`, whose generated SQL carries
152+
its own limit (see below). `sample_events`
153+
returns at most 10 columns, choosing common observability fields such as
154+
`p_timestamp`, `service.name`, `severity_text`, `body` and `trace_id` before
155+
other fields.
156+
157+
## Dataset and time resolution
158+
159+
`list_datasets`, `resolve_dataset` and `discover_datasets` omit internal
160+
datasets.
161+
162+
`resolve_dataset` accepts an optional `telemetryType` (`logs`, `metrics` or
163+
`traces`) and `limit` (1–20, default 5). Its `status` is `resolved`,
164+
`ambiguous`, `not_found` or `error`.
165+
166+
- A `selectedDataset` or `contextDataset` resolves to the visible dataset with
167+
exactly that (case-sensitive) name. If none has that exact name, it resolves
168+
to the one visible dataset whose name matches case-insensitively; if several
169+
do, the result is `ambiguous` with those datasets as `candidates`. If no
170+
visible dataset matches, the result has `status: "error"`. If the selected
171+
dataset's telemetry type does not match `telemetryType`, the result is
172+
`not_found`.
173+
- Otherwise, if exactly one dataset is visible, it resolves to that dataset, or
174+
returns `not_found` if its telemetry type does not match.
175+
- Otherwise it scores the visible datasets that match `telemetryType` by name
176+
against `query`, ignoring case: 1000 if the name equals the query, 750 if the
177+
query contains the name as a whole word (`-` and `_` count as part of a name,
178+
so `api` is not in `rapid` or `api-gateway`), or 500 if the name contains the
179+
query, plus 25 for each query term longer than two characters that the name
180+
contains, plus 15 when `telemetryType` is given. It resolves only a unique top
181+
score of at least 500. In every other case it returns `ambiguous` with up to
182+
`limit` scored `candidates`, or `not_found` if no visible dataset matches
183+
`telemetryType`.
184+
185+
It never falls back to an arbitrary dataset.
186+
187+
`discover_datasets` counts events in the requested range for each visible
188+
dataset of the requested telemetry type that the caller can query. It returns
189+
active datasets ordered by event count as `{ name, eventsInRange }`, together
190+
with `selectedDatasets`, `activeDatasetCount`, `truncated` and up to 20
191+
`unavailableDatasets`. If no dataset is active and at least one could not be
192+
counted, the result has `status: "error"` and no datasets.
193+
194+
`resolve_time_range` accepts any IANA `timezone` (default `UTC`), and uses it
195+
for `today` and `yesterday` day boundaries. It also accepts `last <duration>`,
196+
`past <duration>`, `now-<duration>`, `<start> to <end>` and RFC3339 start
197+
times. An unknown timezone is an error.
198+
199+
`sample_events` accepts an explicit `startTime` and `endTime`, which must be
200+
provided together. They take precedence over `minutes`.
201+
202+
## Log context
203+
204+
`get_log_context` uses the same implementation as `POST /api/v1/query/context`.
205+
Arguments must include exactly one of `log`, `body` or `message`, holding the
206+
anchor row's exact value. The catalog schema enforces this with a root `oneOf`,
207+
so calls with none or more than one are rejected with `400`. The listed
208+
`inputSchema` omits that `oneOf` (see [List tools](#list-tools)) and states the
209+
rule in the property descriptions.
210+
211+
Context bounds are either `contextWindow` or both `contextStartTime` and
212+
`contextEndTime`. Giving only one explicit bound, or a window together with an
213+
explicit bound, is an error. If no bounds are given, the tool uses
214+
`contextWindow: "5 minutes"`. A window extends that duration on each side of
215+
`pTimestamp`, and both ends are rounded down to the minute; if they round to the
216+
same minute, the end is one minute after the start. Explicit bounds are used at
217+
millisecond precision without rounding, and `pTimestamp` must be at or after
218+
the start and before the end. `pageSize` is 1–500, default 100.
219+
220+
Returned records keep at most 10 fields per row, preferring `p_timestamp`,
221+
service, severity, message and trace identifiers. The response also includes
222+
the anchor index, scope and the executed queries.
223+
224+
## Trace activity
225+
226+
`get_trace_activity` returns a bucketed span and trace activity series for one
227+
traces dataset:
228+
229+
```json
230+
{
231+
"name": "get_trace_activity",
232+
"arguments": {
233+
"dataset": "otel-traces",
234+
"startTime": "2026-10-22T00:00:00Z",
235+
"endTime": "2026-10-22T01:00:00Z",
236+
"groupBy": ["service.name"],
237+
"filters": { "span_status_code": 2 },
238+
"bucketCount": 60
239+
}
240+
}
241+
```
242+
243+
`dataset`, `startTime` and `endTime` are required. `startTime` must be before
244+
`endTime`; both are resolved once and applied as exact, half-open bounds (start
245+
inclusive, end exclusive) on the dataset's time partition, so subminute ranges
246+
keep their full precision. `groupBy` takes up to 5
247+
exact field names. Each field may appear only once, and `time`, `span_count`
248+
and `trace_count` are rejected because they are output column names. `filters`
249+
maps exact field names to JSON scalars: strings, numbers and booleans test
250+
equality, and `null` tests `IS NULL`. `bucketCount`
251+
is 1–100, default 60. It approximates the number of `DATE_BIN` buckets over the
252+
dataset's time partition, or `p_timestamp` if none is configured. Buckets are
253+
whole seconds, sized from the exact range and aligned to `startTime`. The dataset
254+
must have traces telemetry and contain `span_span_id` and `span_trace_id`.
255+
Unknown fields are rejected. `span_count` is `COUNT(DISTINCT span_span_id)` and
256+
`trace_count` is `COUNT(DISTINCT span_trace_id)`, so duplicate span rows are
257+
not double counted. Rows are ordered by bucket, oldest first. The result
258+
contains `query`, `bucketInterval`, `semantics`, up to 500 `rows` and
259+
`truncated`.
260+
261+
`query` is the exact SQL the tool runs; it ends in `LIMIT 501` so the tool can
262+
detect overflow. When more than 500 rows match, the newest buckets are dropped
263+
and `truncated` is `true`; reduce `bucketCount`, `groupBy` or the time range to
264+
get the full series. `query` embeds the exact time bounds as a half-open
265+
predicate on the time field. The query API rounds a request's `startTime` and
266+
`endTime` down to the minute, so to reproduce the result, for example in a
267+
dashboard tile, run `query` through the query API with a time range that covers
268+
those bounds.
269+
270+
## Argument schema validation
271+
272+
`inputSchema` is enforced server-side for `type` (a single type or a list of types), `required`,
273+
`additionalProperties`, `enum`, `const`, `minLength`/`maxLength`,
274+
`minimum`/`maximum`, `minItems`/`maxItems`, `items`, `anyOf` and `oneOf`.
275+
`oneOf` passes only when exactly one variant matches. Combinators are checked
276+
in addition to sibling keywords; they do not replace them. Validation uses the
277+
catalog schema, including any root combinators omitted from the listed
278+
`inputSchema`.

‎frontend/docs/api-contracts.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,8 @@ Team requests share the cookie session and `credentials: 'include'`. The adminis
4949
| `getApiKey(id, signal?)` | `GET /api/prism/v1/apikeys/{encoded id}` | Full key, retrieved only on an explicit copy action |
5050
| `deleteApiKey(id)` | `DELETE /api/prism/v1/apikeys/{encoded id}` | Checked success; backend returns `{keyId, keyName, message}` |
5151

52+
API key listing accepts `?privilege=reader|ingestor|admin|writer|editor|superadmin` (alias `priviledge`), matching direct or group-inherited role privileges by kind (an admin key does not match `reader`, and a super-admin key does not match `admin`). Each entry's `roles` lists direct roles only. Unknown or differently cased values, or a repeated parameter, return 400; unknown parameter names are ignored and leave the list unfiltered.
53+
5254
API key expiry is optional: `ttlSeconds` must be an integer from 1800 (30 minutes) through 7776000 (90 days), inclusive; invalid values return 400. Omitted or null TTL means no expiration. Responses include `expiresAt`, a UTC timestamp or null; the frontend also accepts an omitted field from older servers for legacy records. Finite lifetimes are elapsed durations from creation, independent of time zones or daylight saving changes. The creation form defaults to 90 days and offers presets, custom minutes/hours/days, and No expiration. TTL cannot be edited after creation. Expired keys remain listed and deletable; their names remain reserved until deletion.
5355

5456
Both `X-API-KEY` and Bearer authentication return 401 once server time reaches `expiresAt`, and key validation reports false. Temporary API-key sessions, including those created for intra-cluster requests, cannot extend the deadline or refresh as OAuth sessions. Intra-cluster requests on behalf of an expired key return 401. Requests authorized before expiry may finish on the node that authorized them. Expiration changes credential validity, not its assigned roles or an administrator's authority to create other credentials.

‎frontend/e2e/dashboard-editor.spec.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,9 @@ async function addField(editor: Locator, label: string, value: string) {
9090
test('creating every chart type through the editor saves classic-compatible keys', async ({
9191
page,
9292
}) => {
93+
// Eleven editor flows and four full-page accessibility scans share this budget.
94+
test.setTimeout(60_000);
95+
9396
const server = await serve(page);
9497
const sql = 'SELECT time, service, requests FROM "app-logs"';
9598

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
__pycache__/
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Ingestion performance and compatibility checks
2+
3+
These scripts launch disposable local-store servers on loopback ports. They require two already-built Parseable binaries, Python 3, Linux `/proc`, and `taskset` when CPU affinity is supplied. They do not connect to an existing deployment. Each output directory must be new. Test credentials are local dummy values; telemetry and update checks are disabled.
4+
5+
Build the baseline and candidate with the same Rust toolchain, lockfile, features, allocator, profile and real UI assets. Preserve the baseline executable before building the candidate. Run builds and other experiments before timing; avoid competing activity on the test host.
6+
7+
```sh
8+
python scripts/ingest-performance/compatibility.py --baseline /abs/baseline --candidate /abs/candidate --output /abs/results/compat --cpus 2,4,6,8
9+
python scripts/ingest-performance/run.py --baseline /abs/baseline --candidate /abs/candidate --output /abs/results/burst --pairs 4 --cpus 2,4,6,8
10+
python scripts/ingest-performance/sustained.py --baseline /abs/baseline --candidate /abs/candidate --output /abs/results/sustained --pairs 2 --seconds 300 --requests-per-second 20 --cpus 2,4,6,8
11+
python scripts/ingest-performance/mixed.py --baseline /abs/baseline --candidate /abs/candidate --output /abs/results/mixed --pairs 2 --seconds 75 --requests-per-second 2 --queries-per-second 2 --cpus 2,4,6,8
12+
```
13+
14+
Select available physical cores on your machine, or pass `--cpus ''` to omit affinity. Four Actix and four Rayon workers are used in both arms. The burst runner alternates arm order across pairs; `--smoke` runs a small functional check. Sustained and mixed runners also alternate arm order with `--pairs`; the default remains one pair. The offered load starts at wall-clock minute :05 after warmup, unless phase alignment is disabled; its actual anchored start is recorded. Their fixed offered rates do not measure maximum sustainable capacity. Use separate comparisons against the original baseline and reviewed PR binary to isolate the cost of remediation; optional `--reviewed` adds a third arm but ABC/CBA ordering does not balance the middle arm across positions.
15+
16+
The mixed runner ingests 8,204-row, 127-field scalar requests while querying the same dataset. Its default body is 9,556,551 bytes, checked before starting servers against the 10 MiB HTTP limit. The server adds `p_timestamp`; production queries sort fields and exercise reordered schema adaptation during capture. Each concurrent query must include every request acknowledged before it starts and cannot include requests that start after it finishes; in-flight rows remain allowed. These timestamp bounds are checked after load and saved per query. Exact request/row identity is checked after load, drain and persisted-only restart. Mixed CPU includes ingestion and queries. Query latency is not isolated capture time or lock-hold time.
17+
18+
`--diagnostics` streams bounded 100 ms thread state/wchan/schedstat/fault counters, process I/O, host pressure and swap counters. Both arms receive the same instrumentation. Collection cost, errors, truncation and the byte cap are reported; use matched on/off runs before attributing timing changes to the server. A clean run or an off-CPU sample cannot explain a historical stall. The diagnostic does not enumerate unrelated host processes or identify their I/O. Client lag, workload duration, live payload bytes and sampled staging bytes are bounded; hitting a safety bound invalidates the run rather than demonstrating a performance threshold.
19+
20+
Every timed request must succeed. Request IDs are unique, including warmup requests. Verification checks each request's row count, distinct row IDs and bounds. Burst checks repeat after SIGKILL and restart, polling for eventual query visibility for up to 120 iterations. This tests process-crash recovery, not power-loss durability or immediate recovery readiness.
21+
22+
The sustained runner samples server CPU/RSS and file sizes each second. After load stops, it polls the target dataset’s data-bearing staging files for up to 240 seconds, requires three consecutive empty observations, and verifies rows again. It then kills its server, preserves the old staging directory and starts with empty staging to check rows from the persisted local store. Internal `pstats` work may continue after the target dataset drains. CPU is reported separately for the ACK window and through observed drain (the latter also includes the first verification query). Sampled peak RSS includes verification queries and drain; it is not an ingestion-only peak or allocator high-water mark. Staged file size is not a total in-memory backlog measure.
23+
24+
Burst payloads are prepared before timing and capped at 768 MiB of client payload data. The current app/wide workloads retain about 413/528 MB in the client; reported server RSS excludes the client, kernel page cache and other processes. Request identities add one string field to each row. Payloads, binary hashes, timing phase, configuration and host load are recorded. Short burst results may overlap periodic background work differently; inspect all repetitions rather than a single ratio.
25+
26+
The compatibility corpus compares HTTP status/error responses, schemas, ingestion accounting and logical row multisets. It deliberately ignores generated `p_timestamp` and unspecified SQL result order. Every request's expected status is asserted independently, including intended rejections and raw OTLP cases; `partitiontypes` pins one named inherited 500 with its known one-row partial outcome. Successful setup/readback statuses, row counts and accounting are asserted, and the static case additionally checks schema types, IDs and decoded values. Equal errors cannot satisfy these success checks. Exact non-static content/error bodies still rely on differential comparison. Ordered flattening, exact RecordBatch/schema behavior and failure boundaries are covered separately by the Rust differential tests.
27+
28+
Run the server-free oracle and harness tests with `python3 -B -m unittest discover -s scripts/ingest-performance -p 'test_*.py' -v`. `compatibility-baseline.json` is the real successful normalized transcript from commit `6aaee1d61c261c6b71c0dad30b6bee75ac654cab`. Replay tests reject equal 503s on ingests/readbacks and incorrect static values, schemas, or accounting; a replay pass does not replace live validation on the final binaries.
29+
30+
To render PNG/SVG charts, install matplotlib and numpy in an isolated environment, then:
31+
32+
```sh
33+
python scripts/ingest-performance/plot.py --burst /abs/results/burst --sustained /abs/results/sustained --output /abs/results/figures
34+
```
35+
36+
The local-store results do not establish S3 ingestion capacity. Multipart memory and failure behavior require separate storage probes. Keep exact build provenance and limitations with any reported figures.

0 commit comments

Comments
 (0)