Skip to content

Commit 61f6089

Browse files
committed
Release 0.18.2: bound log reports and correct diagnostic graph scoring
1 parent 7a659a6 commit 61f6089

31 files changed

Lines changed: 1731 additions & 191 deletions

‎README.md‎

Lines changed: 46 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1199,7 +1199,7 @@ Flag semantics:
11991199

12001200
- flag absent — logs are not collected and every `server_log` item is skipped;
12011201
- `--log-depth-time-min` without a value — the last 10 minutes;
1202-
- `--log-depth-time-min N` — the last `N` minutes (`0` disables, maximum 1440);
1202+
- `--log-depth-time-min N` — the last `N` minutes (`0` disables, maximum 10080 / 7 days);
12031203
- works with both `one-shot` and `snapshots`; the window ends when the log
12041204
phase starts, which is after all other items (and after the sampling window
12051205
in snapshots mode).
@@ -1278,6 +1278,30 @@ already collected events and marks coverage incomplete; it does not discard the
12781278
whole item. Larger limits allow detailed analytical plans while keeping resource
12791279
use bounded.
12801280

1281+
The string values retained in the report's log tables, chart points and shared
1282+
SQL/plan references have a **soft 128 MiB memory budget**. Admission happens after
1283+
item-level ranking and eviction. `sys.getsizeof()` measures each retained Python
1284+
string object without JSON serialization or UTF-8 copies. Aliases of the same
1285+
object count once; separate equal strings count separately. Keys, container
1286+
storage, presentation metadata, parser buffers and non-log sections are excluded.
1287+
Unused SQL and plan payloads are not added to the report catalog.
1288+
1289+
A whole output row or chart point and its references can cross the threshold.
1290+
Further log output stops, remaining log items are marked skipped with an explicit
1291+
budget reason, and the report finishes normally. Truncated tables/charts have
1292+
updated displayed/omitted counts. Coverage includes `report_value_limit_hit`,
1293+
`estimated_value_bytes`, `value_budget_bytes` and
1294+
`value_budget_method: python_string_objects`. The estimate describes retained
1295+
output; `parsed_records` and the covered timestamps still describe the scan.
1296+
Admission updates a shared counter incrementally; a final traversal accounts for
1297+
presentation-normalized values without serializing the dictionary.
1298+
1299+
HTML shows a red warning immediately below the header:
1300+
“В ходе наполнения отчёта был достигнут лимит; отчёт содержит неполные данные.”
1301+
The warning is also shown when saved JSON is rendered later. This budget applies
1302+
to log data in `logs`, `one-shot` and `snapshots`. It measures neither process RSS
1303+
nor file size: intermediate buffers, metadata and HTML resources have other costs.
1304+
12811305
The artifact records the phase outcome in `runtime.log_collection`
12821306
(`{status, reason, coverage, source}`); `coverage` states the requested and
12831307
actually covered window, scanned bytes, and truncation reasons, so an incomplete
@@ -1322,7 +1346,7 @@ How the command differs from `--log-depth-time-min` on `one-shot`/`snapshots`:
13221346
| --- | --- | --- |
13231347
| File discovery | `pg_ls_logdir()` through the database connection | `*.csv` entries of `--log-dir`; at most 1024 files are probed (newest by modification time first) and symlinks are refused. Modification times never stand in for record times: a truncated listing, an unreadable file, or a file whose last record could not be located within 64 MiB marks the window incomplete |
13241348
| Record boundaries | Server-formatted window, collector parses records | CSV boundaries by quote parity, never physical lines: a date planted inside a quoted message cannot start or end a record. Tail probes are accepted only when parity closes, and the scan start chosen by binary search inside the oldest candidate is validated by record structure (falling back to the whole file); the file that anchors the window is re-checked with exact parity counted from its first byte (`source.anchor_verified`), and a file is excluded from the window only on an exact or trusted timestamp, otherwise it is verified the same way within a 512 MiB budget (`source.files_verified`; past the budget the window is incomplete) |
1325-
| Window end | The server clock when the log phase starts | The newest complete record found in the directory, so `--log-depth-time-min` counts back from the last logged event (default 10, maximum 1440). With a known zone the newest record and the window are chosen on absolute time: across a DST transition both wall-clock scan bounds are widened, the absolute filter decides afterwards, and every record is placed by its own offset |
1349+
| Window end | The server clock when the log phase starts | The newest complete record found in the directory, so `--log-depth-time-min` counts back from the last logged event (default 10, maximum 10080 / 7 days). With a known zone the newest record and the window are chosen on absolute time: across a DST transition both wall-clock scan bounds are widened, the absolute filter decides afterwards, and every record is placed by its own offset |
13261350
| csvlog layout | Known from `server_version_num` | Detected per file from the column count of its first complete record (the head probe grows up to 64 MiB): 23 columns for PostgreSQL 10-12, 24 for 13, 26 for 14 and later; a directory that spans a major upgrade is parsed file by file |
13271351
| Log clock zone | `log_timezone` and its UTC offset | Numeric suffixes (`+03`) and UTC resolve on their own; a bare abbreviation such as `MSK` has no known offset, chart items then carry a `log_timezone_unknown` warning and show the log clock as if UTC unless `--log-timezone Europe/Moscow` names the IANA zone, which also resolves the repeated hour of a fall-back transition through the suffix (`CEST`/`CET`) |
13281352
| Logging settings, rotation findings, active file, block size | From the server GUCs and `pg_current_logfile()` | Unknown: `log_files_overview` lists the files without rotation findings and marks no file as current; `maintenance_events` qualifies page volumes against the largest supported block size (32 KiB) and reports `block_size_unknown` |
@@ -1342,6 +1366,26 @@ window incomplete and turns counts into lower bounds. A missing or empty directo
13421366
items are `unsupported` with the reason in `runtime.log_collection.reason`;
13431367
the command still exits with status 0 because the report was written.
13441368

1369+
The `logs` report has a **300-second overall deadline**, including directory
1370+
discovery, collection, parsing, item construction, JSON serialization and HTML
1371+
rendering. Collection uses the remaining time with 30 seconds reserved for
1372+
finishing the report; discovery, the local scanner and remote harvester share
1373+
that deadline. Blocking validation, planning, artifact creation, local reads,
1374+
parsing, query catalog construction and final rendering run in killable worker
1375+
processes. Cancellation stops and reaps the active worker and terminates an
1376+
active SSH command; shutdown time is reserved within the overall budget.
1377+
Interrupted JSON/HTML writes have their temporary files removed by a supervised
1378+
worker, while existing reports and other runs' temporary files are preserved.
1379+
Cleanup shares the deadline, so a stalled filesystem cannot block shutdown.
1380+
If collection times out first, the remaining time can be used to write an explicit
1381+
collection-error report. If later stages exhaust the overall budget, the command
1382+
exits with a deadline error. The log phase on `one-shot`/`snapshots` keeps its
1383+
existing 60-second bound.
1384+
1385+
Local and SSH scanners use the same wire framing budget and stop at the same
1386+
record boundary when it is exhausted. `scanned_bytes` counts examined input,
1387+
excluding transport read-ahead, so the count does not depend on buffering.
1388+
13451389
## Collection Timing and Metric Evaluation
13461390

13471391
### Collection Timing Scopes

0 commit comments

Comments
 (0)