Repository navigation
FIX: Share one Azure Blob container client across concurrent operations - #4410
Open
Richard Lundeen (richlundeen) wants to merge 1 commit into
Open
Richard Lundeen (richlundeen) wants to merge 1 commit into
Richard Lundeen (richlundeen) wants to merge 1 commit into
Conversation
Each AzureBlobStorageIO operation closed the shared container client in its finally block and reset it to None. When operations overlapped on one instance, such as concurrent serializer writes through AzureSQLMemory's results_storage_io, the first operation to finish closed the client that the others still used, and they failed on a closed session. Count the operations that use the client. Each operation acquires the client and releases it when it finishes. The last release closes the client. Client creation does not await, so concurrent first operations share one client. Split out of microsoft#4133. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain for the Azure Blob client lifecycle changes.
No open issues 🎉
What changed in this PR
This PR fixes premature client closure during overlapping Azure Blob storage operations, including concurrent media reads and writes through AzureSQLMemory. It isolates the storage fix from #4133.
Changes:
- Track active operations and close the shared container client and credential only after the last operation releases them.
- Apply acquire/release handling across all six blob operations.
- Add regression tests for overlapping reads, writes, streaming uploads, and concurrent initial client acquisition.
| File | Description |
|---|---|
tests/unit/memory/storage/test_storage.py |
Adds concurrency tests for client sharing and cleanup. |
pyrit/memory/storage/storage.py |
Adds reference-counted client lifecycle management across blob operations. |
Review effort: Balanced
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Operations that overlap on one
AzureBlobStorageIOinstance can fail. Each operation closes the shared container client when it finishes. Other operations that still use that client then fail on a closed client. This PR keeps the client open until the last operation finishes.AzureSQLMemoryuses oneAzureBlobStorageIOinstance asresults_storage_io. Serializers write and read media through it. When two writes overlap, for example during concurrent attacks, the slower write fails.finally._create_container_client_asyncstates this rule.AzureBlobStorageIOoperations: read, write, stream write, delete, path exists, and is file.This change comes from #4133. It is split out so that it can merge on its own.