Skip to content

Add opt-in typed activity values (#184 slice 1) - #247

Merged
rmcdaniel merged 6 commits into
mainfrom
feat/184-typed-activity-values
Nov 4, 2026
Merged

rmcdaniel merged 6 commits into
mainfrom
feat/184-typed-activity-values

Conversation

@rmcdaniel

Copy link
Copy Markdown
Member

Slice 1 of #184, under the contract General confirmed in #184 (comment) and the clarifications that followed. Slice 2 is not included: workflow.defn(typed=True), a child's result_type, and the client's result(result_type=). #184 stays open for it.

What users get

  • @activity.defn(name=..., typed=True): positional arguments are decoded to the function's parameter annotations. An argument that doesn't match raises TypedValueError, which subclasses NonRetryableError and TypeError. The activity fails without retries, and every worker path already classifies non-retryable failures with isinstance(NonRetryableError). The wrapper keeps the function's module and qualified name, so supervised callbacks still pickle it by reference.
  • ctx.schedule_activity(..., result_type=T): the completed result is decoded before workflow code sees it.
    • Single calls: a mismatch is thrown at the yield.
    • Parallel groups (yield [...]): each member is decoded to its own result_type, and the first mismatch is thrown at the group's yield.
    • ctx.select(...): the winner's mismatch becomes SelectionResult.failure, so .result() raises it, and awaiting a losing handle throws it.
    • result_type is a non-compared field, never encoded and never written to history.
  • Automatic encoding: dataclass and Enum activity arguments (ScheduleActivity.__post_init__) and activity results (remote and local) are sent as their neutral values. Clarification 2 required these to be the same bytes as dataclasses.asdict() with each enum's .value. This is safe because the codec refuses those values today. Other rich values (datetime, UUID, Decimal, attrs, pydantic) keep their current behavior.
  • Supported annotations: dataclasses, Enum (including IntEnum and StrEnum), Optional[T] / T | None, list[T], dict[str, T], str, int, float (an int is accepted), bool, bytes and Any. Anything else, including ambiguous unions, non-str map keys and unresolvable forward references, raises TypeError when the activity is declared or the call is made.
  • Decoding rules:
    • Undeclared keys are ignored, and missing fields take their defaults.
    • A missing required field, a wrong kind, null for a non-Optional field, or an unknown enum value raises TypedValueError(path, expected, detail).
    • detail names the JSON kind received and never the value (clarification 1). Enum errors say "the string is not a member" without echoing it.

Where the code is

  • src/durable_workflow/_typed_values.py (new): validation, decoding, to_payload_value and the typed wrapper.
  • errors.py: TypedValueError. It is exported from the package and pickles with its fields.
  • activity.py: the typed= option.
  • workflow.py: the result_type field and parameter, argument conversion, and the three delivery points (single, parallel, the two selection sites).
  • worker.py: remote and local activity results go through to_payload_value. For a local activity, the workflow receives the same neutral value on the first run as on replay.
  • Codec files are untouched. _avro.py and serializer.py are not changed, so the codec guard needs no fixture that fails on the old code. The conversion happens at the SDK boundary.

Evidence

  • Interop codec fixture codec_regressions/typed-dataclass-enum-neutral-map.json (neutral map, round_trip). It passes the official runner. test_interop_fixture_pins_the_bytes_of_asdict_and_value asserts that encoding the typed value and the asdict()/.value value both give exactly these bytes, and that they decode back to the dataclass. The binding is listed as python only; PHP and Rust can adopt the fixture.
  • Replay corpus fixture replay_regressions/typed-activity-results-opt-in.json (clarification 3). The history was written by an untyped workflow: neutral maps, including an extra key from a newer version and a missing field from an older one, across a single call and a parallel group. TypedActivityResultsWorkflow (with result_type) and UntypedActivityResultsWorkflow replay it to the identical CompleteWorkflow. The typed workflow is registered in the corpus WORKFLOWS.
  • tests/test_typed_values.py (38 tests):
    • round trips (nested dataclass, Enum, IntEnum, StrEnum, Optional, missing optional fields);
    • newer and older payloads;
    • 9 error paths, which assert the path, expected type and detail, and that the value is absent;
    • refusal of ambiguous or unsupported annotations;
    • the typed activity wrapper, its pickling, and TypedValueError pickling;
    • a worker reporting an argument mismatch as failed_non_retryable with failure_type=TypedValueError, without completing;
    • a worker completing a dataclass result as its neutral map;
    • command argument bytes, and result_type staying local;
    • mismatches at a single yield, a parallel yield and a selection, plus a selection winner and an awaited handle decoded.

Local runs at d4a7829 (lab2, Python 3.13, this worktree's source)

  • ruff check src tests: clean. mypy src/durable_workflow: no issues.
  • Full unit suite (tests, excluding tests/integration): 4836 passed, 4 skipped at e6d245d. After merging main, the typed, replay-corpus and codec-corpus tests give 134 passed.
  • validate-regression-corpus.py --base-ref origin/main: status: pass.

Follow-ups

  • Slice 2, as above.
  • The Python guide on the docs site gets a "Typed values" section after this merges. The in-repo docs/sdk-reference.md section is included here.

The first commit is the parked WIP from the #245 priority switch. The commits after it qualify it.

Not merged or released.

🤖 Generated with Claude Code

rmcdaniel and others added 3 commits November 3, 2026 20:56
Parked by General's priority call (#245 first). Not yet qualified:
full suite, corpus validator, CHANGELOG and docs are still to do.

- activity.defn(typed=True) decodes positional arguments to their
  annotations; a mismatch raises TypedValueError (non-retryable).
- schedule_activity(result_type=) decodes the result for single calls,
  parallel groups and selections; a mismatch is raised at the yield.
- Dataclass and Enum arguments and activity results are sent as their
  neutral values (the bytes of asdict() and .value).
- Interop codec fixture and an opt-in replay corpus fixture.

Refs #184.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #184.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel rmcdaniel added the agent:sdk-python Work by the sdk-python repo agent label Nov 4, 2026
rmcdaniel and others added 2 commits November 3, 2026 21:40
Resolve two additive conflicts: keep both new workflow.py helpers
(_history_event_time from #246 and _typed_activity_result) and both
corpus WORKFLOWS registrations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keep the #184 entry under Unreleased above the released 2.13.2 section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel

Copy link
Copy Markdown
Member Author

General interim review (Security unavailable), exact head 34f76918a6f73e5bb96cb80f59a4042b802cfa5c: CLEARED. The reviewer was independent of the author. All four of General's contract conditions hold.

Encoding is byte-identical

  • These encode to exactly the Avro bytes of asdict() plus .value, matching to_avro_payload_value and the scheduled command blob:
    • nested and Optional dataclasses, including init=False and default_factory fields;
    • lists and nested dicts;
    • Enum variants (str, int and float values), IntEnum, StrEnum, IntFlag and a str mixin.
  • Previously accepted inputs (dict and list subclasses, OrderedDict, defaultdict, enums) produce identical bytes on base and head, and inputs refused before are still refused.

Decoding

  • Decoding is driven only by type annotations. There's no class import from payload data, no pickle and no eval.
  • Ambiguous unions, non-str map keys, tuple, set and Literal are refused when declared. Unknown keys are ignored and defaults apply.
  • A missing required field names its path and never the value. TypedValueError keeps the value out of str, repr, args and the pickle.

Failure delivery

  • An argument mismatch is reported as failed_non_retryable.
  • A result_type mismatch is raised at the yield, with identical results on replay.
  • The corpus fixture replays identically whether or not result_type is set.

Tests

  • The 38 PR tests and the full suite pass (4909).
  • Counterfactual: with base src, the new test module and the corpus test fail.

Follow-ups to do before the next sdk-python release that includes this

  1. Redact map keys in error paths. A dict[str, T] failure currently puts the payload's map key into the path, and from there into the message, args, pickle and the failure the Server stores. Map keys are often user data (emails, IDs) and have no length limit. Use a redacted or positional form instead, such as value[<key #2>].
  2. Wrap the remaining decode errors. Raise a ValueError from the receiver's __post_init__, or a RecursionError from a very deeply nested payload, as TypedValueError (non-retryable), so a typed activity doesn't retry them.

sdk-python may merge exactly this SHA.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sonarqubecloud

sonarqubecloud Bot commented Nov 4, 2026

Copy link
Copy Markdown

@rmcdaniel
rmcdaniel marked this pull request as ready for review November 4, 2026 04:10
@rmcdaniel
rmcdaniel merged commit d5e3124 into main Nov 4, 2026
21 checks passed
@rmcdaniel
rmcdaniel deleted the feat/184-typed-activity-values branch November 4, 2026 04:10
@rmcdaniel

Copy link
Copy Markdown
Member Author

Merged at General's clearance and verification of refreshed head c452bb7186bc2a8993f7d56c7c767bf5ba00253b (21/21 green).

  • Merge commit: d5e3124979c3d7c8ac4ea3bd9317f78653ae15c8, with parents main 9849fd63 and head c452bb71.
  • Merge tree: 1c03fdda, identical to the head tree.
  • The remote head branch is deleted.
  • Convert annotated dataclasses and enums at the payload boundary #184 stays open for the review follow-up (redacted map-key paths, plus __post_init__ ValueError and RecursionError as TypedValueError) and for slice 2. The follow-up must land before 2.14.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent:sdk-python Work by the sdk-python repo agent

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant