Repository navigation
Conversation
Motivation: Transaction._stage deduplicated requirements by type only, so a transaction writing two branches sent AssertRefSnapshotId for the first branch only. A concurrent change to the second branch was not detected and was overwritten. Approach: deduplicate AssertRefSnapshotId by (type, ref); other requirement types keep per-type deduplication. Duplicates within a single _stage call are also dropped. Validation: added test_stage_keeps_ref_requirement_per_ref, which fails without the change and passes with it (uv run pytest tests/table/test_init.py -k stage_keeps). The reproduction script from the issue prints [1, 2, 3] before and [1, 3] after. make lint passes. Four other tests in tests/table/test_init.py fail identically on the base commit in this environment (missing optional modules). The retry path with two branches is not covered by a dedicated test. Report: apache#4183 Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com> Assisted-by: claude-sonnet-5-5 (via Claude Code)
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain for the per-ref transaction requirement deduplication change.
No open issues 🎉
What changed in this PR
Fixes requirement deduplication in PyIceberg transactions so each written branch retains its own snapshot assertion, preventing concurrent changes to a second branch from being silently overwritten.
Changes:
- Deduplicates ref requirements by
(type, ref), while keeping type-only deduplication for other requirements. - Drops duplicates within a single
_stagecall and adds a regression test for distinct refs and repeated requirements.
| File | Description |
|---|---|
tests/table/test_init.py |
Adds regression coverage for per-ref requirement deduplication. |
pyiceberg/table/__init__.py |
Preserves snapshot assertions for every distinct ref during transaction staging. |
Review effort: Balanced
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
Transaction._stagededuplicated requirements by type only, so in a transaction writing two branches only the firstAssertRefSnapshotIdwas sent. A concurrent change to the second branch was not detected and got overwritten. Requirements are now deduplicated by(type, ref)forAssertRefSnapshotId; all other types are unchanged. Duplicates within a single_stagecall are also dropped now.Are these changes tested?
Yes. Added
test_stage_keeps_ref_requirement_per_ref(fails before the fix, passes after). The issue's repro script prints[1, 2, 3]before and[1, 3]after.make lintpasses. Intests/table/test_init.py, 4 tests fail identically onmain(missing optional modules in my env);test_partitioning.pyalso has failures onmain.Are there any user-facing changes?
Multi-branch transactions now assert every written branch, so a concurrent change to any of them raises a conflict and retries instead of being silently overwritten.
Fixes #4183