Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions stovepipe/controller/issuetrack/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,12 @@ go_library(
importpath = "github.com/uber/submitqueue/stovepipe/controller/issuetrack",
visibility = ["//visibility:public"],
deps = [
"//platform/base/id:go_default_library",
"//platform/base/messagequeue:go_default_library",
"//platform/consumer:go_default_library",
"//platform/metrics:go_default_library",
"//stovepipe/core/loader:go_default_library",
"//stovepipe/core/messagequeue:go_default_library",
"//stovepipe/entity:go_default_library",
"//stovepipe/extension/storage:go_default_library",
"@com_github_uber_go_tally//:go_default_library",
Expand All @@ -18,11 +22,14 @@ go_library(
go_test(
name = "go_default_test",
srcs = [
"delivery_test.go",
"fixture_test.go",
"issuetrack_test.go",
],
embed = [":go_default_library"],
deps = [
"//platform/base/messagequeue:go_default_library",
"//platform/consumer/mock:go_default_library",
"//stovepipe/core/messagequeue:go_default_library",
"//stovepipe/entity:go_default_library",
"//stovepipe/extension/storage:go_default_library",
Expand Down
16 changes: 15 additions & 1 deletion stovepipe/controller/issuetrack/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Issue tracking

Issue tracking maintains current issue state and opening/closing history from persisted build issues. This layer applies one issue at a time; notification handling and bounded passes follow separately. No consumer is registered yet.
Issue tracking maintains current issue state and opening/closing history from persisted build issues. Each delivery handles a bounded batch of issues and requests immediate redelivery when more work remains. No consumer is registered yet.

## Applying one issue

Expand All @@ -17,3 +17,17 @@ Closed issue rows are retained, so their versions cannot reset on reopening. A r
The caller must finish the current cursor and its history before moving on or admitting another build. If a replacement worker advances, completing that cursor changes the issue version first. A paused worker's earlier issue write then fails its row guard. A worker that reads later is rejected by source ordering or the conditional cursor write.

The post-read cursor write and processed-build updates, including unchanged statuses, are required for this ordering. A progress version alone is not a lock around issue writes. These operations use individual K-V reads, conditional writes, and immutable history creates; no cross-record transaction or stored future-change payload is needed.

## Bounded passes

Every delivery processes at most 100 task/unit keys, including a revisited cursor and retained closed rows. Refreshes open reported issues without closing missing ones. Finalization also scans issue states and closes issues absent from that build's accumulated issues. A canceled build may not have performed validation, so it can open reported issues but cannot close existing issues by absence.

The cursor is inclusive: revisit it to finish the issue and its history, then continue the ordered range after it. A full batch retains the active phase and requests redelivery. Exhausting the range conditionally marks the pass refreshed or finalized.

An unfinished pass completes before another source is admitted. A notification that helps finish a different pass requests redelivery before handling its own build. Completed notifications and older sources are ignored, as are refreshes after the same Request has finalized. A final notification requires a terminal build chosen by its Request.

Build IDs are opaque; source order comes from Request IDs. All scans are bounded reads under leading primary-key prefixes. There is no transaction spanning progress, issue state, and history.

## Recovery and activation

An error returns to normal consumer retry. The next delivery resumes the cursor before advancing. Permanently failing cursor work blocks issue tracking for that queue; terminal recovery and consumer registration remain follow-up work. This PR does not enable consumption or publish repair requests.
Loading
Loading