Skip to content

fix: normalize Rust-emitted log record tags in openjd.sessions._v1 - #377

Open
AlexTranAmz wants to merge 1 commit into
OpenJobDescription:mainlinefrom
AlexTranAmz:fix/v1-normalize-rust-log-tags
Open

AlexTranAmz wants to merge 1 commit into
OpenJobDescription:mainlinefrom
AlexTranAmz:fix/v1-normalize-rust-log-tags

Conversation

@AlexTranAmz

Copy link
Copy Markdown

Fixes: N/A

What was the problem/requirement? (What/Why)

The Rust session runtime tags every record it emits on the openjd.sessions logger with session_id, openjd_log_content (a LogContent bitmask) and openjd_timestamp_usec. Those records reach Python through pyo3-log, which renders each key-value with its Rust Display before passing it to Logger.makeRecord(..., extra=...). So openjd_log_content arrives as a decimal string such as "8" rather than the LogContent this logger's docstring promises, and openjd_timestamp_usec as a string. Consumers route these records by type-checking openjd_log_content against openjd.sessions.LogContent (the Deadline Cloud Worker Agent does isinstance(record.openjd_log_content, LogContent)), so every Rust-emitted record fails the check and is dropped from the worker log.

This only becomes observable with an openjd-model build that enables pyo3-log's kv feature (companion PR in openjd-model-for-python); today the fields are missing entirely.

A related trap: openjd/sessions/_v1/_logging.py declared its own LogContent(Flag) with the same members and bit values as openjd/sessions/_logging.py. enum.Flag comparisons are class-scoped, so openjd.sessions._v1.LogContent.BANNER failed the same isinstance check against the public class. Nothing in _v1 tags records with that copy today, but the name is re-exported.

What was the solution? (How)

openjd/sessions/_v1/_logging.py:

  • Importing openjd.sessions._v1 installs one logging.Filter on the openjd.sessions logger. If openjd_log_content is a digit string it becomes LogContent(int(value)); if openjd_timestamp_usec is a digit string it becomes an int. Anything else (absent, already typed, a bit this LogContent does not define, malformed) is left as it arrived. The filter never drops a record.
  • It is a logger filter, not a handler filter: pyo3-log delivers records with Logger.handle(), which applies the logger's filters before any handler or handler-level filter sees the record, which is where consumers type-check.
  • Installation is idempotent (repeated imports do not stack filters).
  • LogContent is now re-exported from openjd.sessions._logging instead of redefined, so openjd.sessions._v1.LogContent is openjd.sessions.LogContent.

What is the impact of this change?

Consumers attaching handlers to openjd.sessions receive the Rust runtime's records with the documented types once an openjd-model release forwards the fields. With the current openjd-model 0.14.0 the fields are absent and the filter is a no-op. Either release order is safe: this package first is inert; openjd-model first restores session_id (correct routing) while openjd_log_content stays a string that typed consumers reject exactly as today.

How was this change tested?

See DEVELOPMENT.md for information on running tests.

  • Have you run the unit tests? Yes.

  • test/openjd/sessions_v1/test_logging.py: filter unit tests with logging.makeLogRecord (each LogContent member and a composite from strings; already-typed, absent, malformed and undefined-bit values preserved; timestamp conversion; session_id untouched; idempotent install; records passed through Logger.handle() reach handlers and handler-level filters normalized; _v1.LogContent is openjd.sessions.LogContent with unchanged bit values).

  • test/openjd/sessions_v1/test_rust_log_tags.py: runs a real openjd.sessions._v1.Session task (sys.executable -c, so Linux/macOS/Windows) and asserts the captured records carry a LogContent, this session's session_id, an int timestamp, and that the task's stdout is tagged COMMAND_OUTPUT. The four assertions that need the fields are gated on a probe (any record carrying session_id) so they skip on openjd-model 0.14.0 and run automatically on a build that forwards the fields; the probe has its own ungated tests.

Local results (Linux, Python 3.12):

  • hatch run test with openjd-model 0.14.0: 1046 passed, 50 skipped, 16 xfailed (gated tests skip as designed)
  • hatch run test with a wheel built from the companion openjd-model branch: 1050 passed, 46 skipped, 16 xfailed (gated tests run and pass)
  • hatch run lint (ruff, black, mypy): clean
  • Not run locally: Python 3.9-3.11, 3.13, 3.14, Windows, macOS (CI matrix); cross-user suites (not touched).

Was this change documented?

  • Are relevant docstrings in the code base updated? Yes: the LOG docstring documents the three attributes and the normalization; the filter and install helper carry docstrings.

Is this a breaking change?

No. openjd.sessions._v1 is private and exports the same names. openjd.sessions._v1.LogContent is now the same class object as openjd.sessions.LogContent; member names and bit values are unchanged. Cross-class comparisons that previously returned False or raised TypeError now behave as same-class comparisons, which is the intended fix.

Does this change impact security?

  • Does the change need to be threat modeled? No. Rust records already classified as EXCEPTION_INFO, PROCESS_CONTROL or HOST_INFO become visible to consumers that allowlist those categories, which is the same routing the Python runtime's records have always had. No new data is produced.

Cross-port to openjd-rs

This package is being migrated to Rust in openjd-rs/crates/openjd-sessions.
Behavioral changes made here should be replicated there to keep the two
implementations in sync until the migration is complete.

  • This change does not affect runtime behavior (docs / tests / tooling only), or
  • A matching change has been opened in openjd-rs (link the PR here): , or
  • A tracking issue has been filed in openjd-rs to port this change (link here): , or
  • Cross-porting is not applicable for this change because: it is Python-side normalization of values that pyo3-log stringifies on the way into Python; openjd-rs already emits these fields as typed key-values (session_log! sends openjd_log_content as a u32 and openjd_timestamp_usec as a u64).

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

The Rust session runtime tags every record it emits on the "openjd.sessions"
logger with session_id, openjd_log_content (a LogContent bitmask) and
openjd_timestamp_usec. Those records reach Python through pyo3-log, which
renders each structured value with its Rust Display implementation before
passing it to Logger.makeRecord(..., extra=...), so openjd_log_content arrives
as a decimal string such as "8" instead of the LogContent this logger
documents. pyo3-log has no hook to preserve the types, and consumers that
route records by type-checking openjd_log_content against LogContent dropped
every Rust-emitted record.

Install one logging.Filter on the "openjd.sessions" logger when
openjd.sessions._v1 is imported. It converts openjd_log_content back to a
LogContent and openjd_timestamp_usec back to an int. It sits on the logger
rather than a handler because pyo3-log delivers records with Logger.handle(),
which applies the logger's filters before any handler or handler-level filter
sees the record. Values this LogContent cannot represent are left unchanged;
the filter never drops a record.

Also re-export LogContent from openjd.sessions._logging instead of declaring a
second, identically-shaped Flag enum here: enum.Flag comparisons are
class-scoped, so a record tagged with the duplicate class would fail the same
type check consumers use.

The end-to-end test runs a real Rust-backed session and is skipped until the
installed openjd-model build forwards the structured fields.

Signed-off-by: AlexTranAmz <167144297+AlexTranAmz@users.noreply.github.com>
@jericht

jericht commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

aws-deadline/deadline-cloud-worker-agent#1130 -> will need to fix the actions here too before merging

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants