Repository navigation
fix: normalize Rust-emitted log record tags in openjd.sessions._v1 - #377
AlexTranAmz wants to merge 2 commits into
Conversation
The Rust session runtime tags every record it emits on the "openjd.sessions" logger with session_id, openjd_log_content (a LogContent bitmask) and openjd_timestamp_usec. Those records reach Python through pyo3-log, which renders each structured value with its Rust Display implementation before passing it to Logger.makeRecord(..., extra=...), so openjd_log_content arrives as a decimal string such as "8" instead of the LogContent this logger documents. pyo3-log has no hook to preserve the types, and consumers that route records by type-checking openjd_log_content against LogContent dropped every Rust-emitted record. Install one logging.Filter on the "openjd.sessions" logger when openjd.sessions._v1 is imported. It converts openjd_log_content back to a LogContent and openjd_timestamp_usec back to an int. It sits on the logger rather than a handler because pyo3-log delivers records with Logger.handle(), which applies the logger's filters before any handler or handler-level filter sees the record. Values this LogContent cannot represent are left unchanged; the filter never drops a record. Also re-export LogContent from openjd.sessions._logging instead of declaring a second, identically-shaped Flag enum here: enum.Flag comparisons are class-scoped, so a record tagged with the duplicate class would fail the same type check consumers use. The end-to-end test runs a real Rust-backed session and is skipped until the installed openjd-model build forwards the structured fields. Signed-off-by: AlexTranAmz <167144297+AlexTranAmz@users.noreply.github.com>
|
aws-deadline/deadline-cloud-worker-agent#1130 -> will need to fix the actions here too before merging |
Signed-off-by: AlexTranAmz <167144297+AlexTranAmz@users.noreply.github.com>
| if isinstance(content, str) and content.isdigit(): | ||
| try: | ||
| setattr(record, "openjd_log_content", LogContent(int(content))) | ||
| except ValueError: |
|
Claude review · advisory Reviewed Open: 0 blocking · 0 should-fix · 0 nit. ✅ Nothing blocking. Fix or reply to each thread; the next revision's review re-checks open threads and resolves those it agrees are handled. Resolving a thread yourself also closes it. Later revisions review only what changed. |
| @@ -2,33 +2,16 @@ | |||
| from __future__ import annotations | |||
There was a problem hiding this comment.
Can we add a spec (follow up) documenting the Python and Rust behaviors so it never drifts?
|
Before/after evidence for reviewers. One real Rust-runtime task ( Before: released openjd-model 0.14.0 + openjd-sessions 0.14.0 No tags at all: every Rust line passes every session's filter (cross-session bleed), nothing reaches the worker log, and the After openjd-model-for-python#384 only (pyo3-log
After #384 + this PR Same records, now typed; the HOST_INFO / PROCESS_CONTROL lines reach the worker log, which matches what the Python runtime has always produced. The gated test in |
Fixes: N/A
What was the problem/requirement? (What/Why)
The Rust session runtime tags every record it emits on the
openjd.sessionslogger withsession_id,openjd_log_content(aLogContentbitmask) andopenjd_timestamp_usec. Those records reach Python through pyo3-log, which renders each key-value with its RustDisplaybefore passing it toLogger.makeRecord(..., extra=...). Soopenjd_log_contentarrives as a decimal string such as"8"rather than theLogContentthis logger's docstring promises, andopenjd_timestamp_usecas a string. Consumers route these records by type-checkingopenjd_log_contentagainstopenjd.sessions.LogContent(the Deadline Cloud Worker Agent doesisinstance(record.openjd_log_content, LogContent)), so every Rust-emitted record fails the check and is dropped from the worker log.This only becomes observable with an openjd-model build that enables pyo3-log's
kvfeature (companion PR in openjd-model-for-python); today the fields are missing entirely.A related trap:
openjd/sessions/_v1/_logging.pydeclared its ownLogContent(Flag)with the same members and bit values asopenjd/sessions/_logging.py.enum.Flagcomparisons are class-scoped, soopenjd.sessions._v1.LogContent.BANNERfailed the sameisinstancecheck against the public class. Nothing in_v1tags records with that copy today, but the name is re-exported.What was the solution? (How)
openjd/sessions/_v1/_logging.py:openjd.sessions._v1installs onelogging.Filteron theopenjd.sessionslogger. Ifopenjd_log_contentis a digit string it becomesLogContent(int(value)); ifopenjd_timestamp_usecis a digit string it becomes anint. Anything else (absent, already typed, a bit thisLogContentdoes not define, malformed) is left as it arrived. The filter never drops a record.Logger.handle(), which applies the logger's filters before any handler or handler-level filter sees the record, which is where consumers type-check.LogContentis now re-exported fromopenjd.sessions._logginginstead of redefined, soopenjd.sessions._v1.LogContent is openjd.sessions.LogContent.What is the impact of this change?
Consumers attaching handlers to
openjd.sessionsreceive the Rust runtime's records with the documented types once an openjd-model release forwards the fields. With the current openjd-model 0.14.0 the fields are absent and the filter is a no-op. Either release order is safe: this package first is inert; openjd-model first restoressession_id(correct routing) whileopenjd_log_contentstays a string that typed consumers reject exactly as today.How was this change tested?
See DEVELOPMENT.md for information on running tests.
Have you run the unit tests? Yes.
test/openjd/sessions_v1/test_logging.py: filter unit tests withlogging.makeLogRecord(eachLogContentmember and a composite from strings; already-typed, absent, malformed and undefined-bit values preserved; timestamp conversion;session_iduntouched; idempotent install; records passed throughLogger.handle()reach handlers and handler-level filters normalized;_v1.LogContent is openjd.sessions.LogContentwith unchanged bit values).test/openjd/sessions_v1/test_rust_log_tags.py: runs a realopenjd.sessions._v1.Sessiontask (sys.executable -c, so Linux/macOS/Windows) and asserts the captured records carry aLogContent, this session'ssession_id, aninttimestamp, and that the task's stdout is taggedCOMMAND_OUTPUT. The four assertions that need the fields are gated on a probe (any record carryingsession_id) so they skip on openjd-model 0.14.0 and run automatically on a build that forwards the fields; the probe has its own ungated tests.Local results (Linux, Python 3.12):
hatch run testwith openjd-model 0.14.0: 1046 passed, 50 skipped, 16 xfailed (gated tests skip as designed)hatch run testwith a wheel built from the companion openjd-model branch: 1050 passed, 46 skipped, 16 xfailed (gated tests run and pass)hatch run lint(ruff, black, mypy): cleanWas this change documented?
LOGdocstring documents the three attributes and the normalization; the filter and install helper carry docstrings.Is this a breaking change?
No.
openjd.sessions._v1is private and exports the same names.openjd.sessions._v1.LogContentis now the same class object asopenjd.sessions.LogContent; member names and bit values are unchanged. Cross-class comparisons that previously returnedFalseor raisedTypeErrornow behave as same-class comparisons, which is the intended fix.Does this change impact security?
EXCEPTION_INFO,PROCESS_CONTROLorHOST_INFObecome visible to consumers that allowlist those categories, which is the same routing the Python runtime's records have always had. No new data is produced.Cross-port to openjd-rs
This package is being migrated to Rust in
openjd-rs/crates/openjd-sessions.Behavioral changes made here should be replicated there to keep the two
implementations in sync until the migration is complete.
openjd-rs(link the PR here): , oropenjd-rsto port this change (link here): , oropenjd-rsalready emits these fields as typed key-values (session_log!sendsopenjd_log_contentas au32andopenjd_timestamp_usecas au64).By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.