Skip to content

FIX: Share one Azure Blob container client across concurrent operations - #4410

Open
Richard Lundeen (richlundeen) wants to merge 1 commit into
microsoft:mainfrom
richlundeen:richlundeen-azure-blob-shared-client
Open

Richard Lundeen (richlundeen) wants to merge 1 commit into
microsoft:mainfrom
richlundeen:richlundeen-azure-blob-shared-client

Conversation

@richlundeen

Copy link
Copy Markdown
Contributor

Description

Operations that overlap on one AzureBlobStorageIO instance can fail. Each operation closes the shared container client when it finishes. Other operations that still use that client then fail on a closed client. This PR keeps the client open until the last operation finishes.

AzureSQLMemory uses one AzureBlobStorageIO instance as results_storage_io. Serializers write and read media through it. When two writes overlap, for example during concurrent attacks, the slower write fails.

  • Count the operations that use the shared container client. Each operation acquires the client before it starts and releases it in finally.
  • Close the client and its credential only when the last operation releases it.
  • Client creation does not await, so concurrent first operations create one client. The docstring of _create_container_client_async states this rule.
  • Apply this to all AzureBlobStorageIO operations: read, write, stream write, delete, path exists, and is file.

This change comes from #4133. It is split out so that it can merge on its own.

Each AzureBlobStorageIO operation closed the shared container client in its
finally block and reset it to None. When operations overlapped on one
instance, such as concurrent serializer writes through AzureSQLMemory's
results_storage_io, the first operation to finish closed the client that the
others still used, and they failed on a closed session.

Count the operations that use the client. Each operation acquires the client
and releases it when it finishes. The last release closes the client.
Client creation does not await, so concurrent first operations share one
client.

Split out of microsoft#4133.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 18, 2026 23:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the Azure Blob client lifecycle changes.

No open issues 🎉

What changed in this PR

This PR fixes premature client closure during overlapping Azure Blob storage operations, including concurrent media reads and writes through AzureSQLMemory. It isolates the storage fix from #4133.

Changes:

  • Track active operations and close the shared container client and credential only after the last operation releases them.
  • Apply acquire/release handling across all six blob operations.
  • Add regression tests for overlapping reads, writes, streaming uploads, and concurrent initial client acquisition.
File Description
tests/​unit/​memory/​storage/​test_storage.py Adds concurrency tests for client sharing and cleanup.
pyrit/​memory/​storage/​storage.py Adds reference-counted client lifecycle management across blob operations.

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants