Skip to content

Avoid redundant continuations for completed gRPC client responses - #840

Merged
Bernd Verst (berndverst) merged 1 commit into
mainfrom
berndverst-activity-execution-performance
Nov 12, 2026
Merged

Bernd Verst (berndverst) merged 1 commit into
mainfrom
berndverst-activity-execution-performance

Conversation

@berndverst

Copy link
Copy Markdown
Member

Summary

What changed?

  • Record already-successful unary responses directly in ChannelRecreatingCallInvoker.ObserveOutcome, avoiding a continuation task and optional boxed cancellation state.
  • Add allocation regression tests and full-invoker coverage for completed success, pending success, and pending transport failure.
  • Add an Unreleased entry in CHANGELOG.md.

Why is this change needed?

Response at observation time Caller token Before After
Already successful Non-cancellable 112 B/call 0 B/call
Already successful Cancellable 144 B/call 0 B/call
Still pending Non-cancellable 112 B/call 112 B/call
Still pending Cancellable 144 B/call 144 B/call
  • Completed-success observation measured approximately 168–173 ns before and 7 ns after (median of five 1,000,000-call samples, tiered compilation disabled). These isolated observer measurements are not an end-to-end RPC throughput or latency claim. The optimization applies only when the response is already successful when observed.
  • Preserve Harden gRPC worker and client against silent disconnects #708's hardening: successes still reset the failure count. Pending and faulted responses keep the existing continuation, cancellation classification, failure threshold, cooldown, and background recreation behavior. Worker reconnect handling is unchanged.

Issues / work items


Project checklist

  • Release notes are not required for the next release
    • Otherwise: Notes added to release_notes.md (this repository uses CHANGELOG.md; the entry is there).
  • Backport is not required
    • Otherwise: Backport tracked by issue/PR N/A.
  • All required tests have been added/updated (unit tests, E2E tests)
  • Breaking change?
    • If yes:
      • Impact: N/A; no public API, serialization, or wire-protocol changes.
      • Migration guidance: N/A.

AI-assisted code disclosure (required)

Was an AI tool used? (select one)

  • No
  • Yes, AI helped write parts of this PR (e.g., GitHub Copilot)
  • Yes, an AI agent generated most of this PR

If AI was used:

  • Tool(s): GitHub Copilot.
  • AI-assisted areas/files: Investigation, src/Client/Grpc/ChannelRecreatingCallInvoker.cs, test/Client/Grpc.Tests/GrpcDurableTaskClientChannelRecreationTests.cs, and CHANGELOG.md.
  • What you changed after AI output: No human edits are claimed. The agent verified allocation failures before the fix, checked the corrected behavior, and reviewed the final diff.

AI verification (required if AI was used):

  • I understand the code and can explain it
  • I verified referenced APIs/types exist and are correct
  • I reviewed edge cases/failure paths (timeouts, retries, cancellation, exceptions)
  • I reviewed concurrency/async behavior
  • I checked for unintended breaking or behavior changes

Testing

Automated tests

  • Dependency setup: passed:
    dotnet restore test\Client\Grpc.Tests\Client.Grpc.Tests.csproj --configfile nuget.config --verbosity quiet
  • Before applying the production fix, ran the five new test cases: 2 expected allocation failures, 3 behavior cases passed:
    dotnet test test\Client\Grpc.Tests\Client.Grpc.Tests.csproj --configuration Release --no-restore --filter 'FullyQualifiedName~ObserveOutcome_CompletedSuccess_DoesNotAllocatePerCall|FullyQualifiedName~AsyncUnaryCall_CompletedOrPendingSuccess_ResetsFailureCount|FullyQualifiedName~AsyncUnaryCall_PendingTransportFailure_StillTriggersRecreate' --verbosity quiet -clp:ErrorsOnly
  • After the fix, full client gRPC test project: passed, 86 tests:
    dotnet test test\Client\Grpc.Tests\Client.Grpc.Tests.csproj --configuration Release --no-restore --verbosity quiet -clp:ErrorsOnly
  • Client gRPC build: passed for netstandard2.0, net6.0, net8.0, and net10.0; 0 errors, 66 existing warnings:
    dotnet build src\Client\Grpc\Client.Grpc.csproj --configuration Release --no-restore --verbosity quiet -clp:ErrorsOnly
  • Result: Passed for the changed component. Full solution and live DTS/emulator tests were not run.

Manual validation (only if runtime/behavior changed)

  • Environment (OS, .NET version, components): Windows; .NET SDK 10.0.403 / runtime 10.0.14; actual production Client.Grpc observer, with no service/network dependency.
  • Steps + observed results:
    1. Compare Harden gRPC worker and client against silent disconnects #708 and subsequent changes with fetched origin/main at ae1464cb; confirm Reduce per-call allocations in ChannelRecreatingCallInvoker outcome tracking #782 leaves unconditional continuation allocation in place.
    2. Bind a delegate to the production ObserveOutcome before measurement; pre-create completed responses and cancellation sources, warm up, and measure elapsed time and thread allocations with DOTNET_TieredCompilation=0. Measure completed and pending responses, with and without cancellable tokens.
    3. Re-run the same probe after the fix: completed-success observer allocations fall to zero, while pending-response allocation costs stay unchanged. The committed tests independently assert zero bytes over 10,000 warmed observations for each token case.
  • Evidence (optional): Before/after measurements above; both allocation assertions failed against unchanged production code and passed after the fast path was added.

Notes for reviewers

  • The production correction is six lines. It deliberately does not replace asynchronous observation or weaken Harden gRPC worker and client against silent disconnects #708's disconnect recovery.
  • A task cannot transition out of RanToCompletion, so the fast path is safe during concurrent completion. A response finishing after the check follows the unchanged ContinueWith path.
  • This fixes avoidable allocation overhead on completed successes; it does not claim that all necessary asynchronous observation overhead can be removed.

Record completed unary successes directly instead of allocating a continuation task and optional cancellation-state box. Preserve existing observation and channel-recreation behavior for pending and failed responses.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 6, 2026 19:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the optimization that avoids continuation allocations on already-successful gRPC responses.

No open issues 🎉

What changed in this PR

Optimizes gRPC client outcome tracking by skipping continuation allocation for responses that have already succeeded. This builds on the earlier channel-recovery and allocation improvements while preserving transport-failure handling.

Changes:

  • Record completed successes directly, resetting the failure count without allocating a continuation.
  • Add allocation regression tests and behavior coverage for completed success, pending success, and pending transport failure.
  • Document the optimization in the Unreleased changelog.
File Description
test/​Client/​Grpc.Tests/​GrpcDurableTaskClientChannelRecreationTests.cs Adds allocation and channel-recovery regression coverage.
src/​Client/​Grpc/​ChannelRecreatingCallInvoker.cs Adds a fast path for already-successful unary responses.
CHANGELOG.md Records the optimization under Unreleased.

Review effort: Balanced

@berndverst
Bernd Verst (berndverst) merged commit f598311 into main Nov 12, 2026
11 checks passed
@berndverst
Bernd Verst (berndverst) deleted the berndverst-activity-execution-performance branch November 12, 2026 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants