Deadline conflicts by CfP platform, block-based PR body, changedetection routing

.claude/skills/critic/SKILL.md

@@ -19,6 +19,7 @@ For each entry in `changes`, check:
 - A change to an existing entry's `conference`, or one that replaces or clears its `alt_name`, fails (AGENTS.md "Names").
 - A replaced value: `old` is what the data holds, and the source shows it wrong for its own edition, not only for another one (AGENTS.md "Complete data").
 - A set that moves an entry to another year loses no value: every field the entry held is in the moved entry or stays on the edition it belongs to.
+- A change that carries `anchor` (AGENTS.md "Deadline conflicts"): the field is `cfp`, `cfp_ext` or `timezone`; the anchor holds, meaning the platform page is the edition's `cfp_link` as the data holds it now (not one this set changes), or you fetched the anchor page on the conference's `link` host and it links to the platform page with `anchor_quote`; and a `timezone` from a Sessionize offset is a zone of the event's location with that offset on the deadline date. A converted `cfp` names both zones in `reason`; the desk computes its value, so check the zones, not the arithmetic. An anchor that does not hold fails the change.
 
 For each entry in `titles`, check:
 
@@ -54,7 +55,7 @@ A change set flagged `denylist-review` gets `needs-maintainer`, whatever its che
 
 - `accept`: the changes pass and, for a new series, no signal against was found and at least one signal for was. A failing change or variation goes in `drop` with its reason and the rest stand, as long as a new entry keeps every field `utils/tidy_conf/schema.py` requires.
 - `reject`: the domain is on the denylist's `reject` list; the find shows pay-to-present fees, paid publication or no Python content; or nothing usable passes. Give the reason.
-- `needs-maintainer`: everything in between, such as a page that did not render, the denylist's `review` list, mixed signals, or sources that disagree beyond the precedence rule in AGENTS.md.
+- `needs-maintainer`: everything in between, such as a page that did not render, the denylist's `review` list, mixed signals, or sources that disagree beyond the precedence rule in AGENTS.md. A deadline conflict an anchored platform settles (AGENTS.md "Deadline conflicts") is not one of these: when its checks pass, accept it and keep the `sources-disagree` flag for the PR.
 
 Return:
 

.claude/skills/enricher/SKILL.md

@@ -14,7 +14,7 @@ The desk hands you one worklist item about an edition the data holds in any of t
 2. **Fetch** `link`, `cfp_link`, the CfP platform page, every URL in the queue items, and the pages the site links for sponsors, financial aid and its Code of Conduct.
    Done when: every URL has an outcome: fetched, redirected (to where) or failed (how).
 3. **Check every field** against what the sources state:
-   - `cfp` and `cfp_ext`: look for an extension, a second extension included (it replaces `cfp_ext`). A CfP that stays open past its deadline without a stated new date gives no value.
+   - `cfp` and `cfp_ext`: look for an extension, a second extension included (it replaces `cfp_ext`). A CfP that stays open past its deadline without a stated new date gives no value. When the site and the CfP platform state different deadlines or zones, settle it by AGENTS.md "Deadline conflicts" rather than returning `needs-maintainer`.
    - `start`, `end` and `place`, and whether the edition is cancelled (`cfp: Cancelled`).
    - `cfp_link`, `sponsor`, `finaid` and `coc`: each lands on the page it names.
    - `twitter`, `mastodon`, `bluesky` and `youtube`: the conference's own accounts, as its site links them.
@@ -28,7 +28,7 @@ The desk hands you one worklist item about an edition the data holds in any of t
    - `signal`: the payload `diff` shows what changed on a watched page; account for every change in it.
    - `question`: the canonical import asks whether `yaml_value` or `remote_value` is right for `field`. The live site decides: a change when the remote value is right, otherwise `no-change` with the quote that supports the data. `default_answer` is the importer's guess, not evidence.
      - When `field` is `conference`, the question is instead whether the two names are one series, and the import has already written the remote row as its own entry under `remote_value`. When the sources show one series, return `needs-maintainer` naming that entry (`remote_value`, `year`) and the `titles.yml` variation to add (`remote_value` under the `yaml_value` key); otherwise `no-change`.
-   - `drift`: `diffs` maps each field to its value in the data (`data`) and the value the pretalx CfP page at the queue item's `url` enforces (`source`). For `cfp`, `data` is `TBA` or the effective deadline, marked `(cfp_ext)` when it is the extension. pretalx is the platform that enforces the deadline, so under AGENTS.md precedence a later pretalx date wins over the homepage; flag `sources-disagree` when the conference's own prose states another deadline. For `cfp`:
+   - `drift`: `diffs` maps each field to its value in the data (`data`) and the value the pretalx CfP page at the queue item's `url` enforces (`source`). For `cfp`, `data` is `TBA` or the effective deadline, marked `(cfp_ext)` when it is the extension. pretalx is the platform that enforces the deadline, so when it is anchored a later pretalx date wins over the homepage (AGENTS.md "Deadline conflicts", which also covers its zone and converting `cfp`); flag `sources-disagree` when the conference's own prose states another deadline. An unanchored pretalx page is a lead. For `cfp`:
      - `data` is `TBA`: the pretalx deadline is `cfp`.
      - pretalx is later than `data`: its deadline goes in `cfp_ext`.
      - pretalx is earlier: it corrects `cfp` only when a source states it as the original deadline. Otherwise leave `cfp` as it is and flag `sources-disagree`.

.claude/skills/scouter/SKILL.md

@@ -14,6 +14,7 @@ The desk hands you the daily `scout` item, or an item about a series no data fil
      - `lead-only`: an aggregator lead. Copy no value from it; take every value from a primary source.
      - `no-website`: the shared sheet has this edition but no website. Search for one.
      - `denylist-review: <domain>: <reason>`: the domain is on the denylist's `review` list. Copy the flag into the `flags` of the find's change set; the critic returns needs-maintainer.
+   - A `registry` item whose payload `registry` is `listing`: a watched page that lists many events changed. Each event name or link the payload's `diff` adds is a find, with the item's `url` as where it came from; fetch that page when the diff is cut short or names an event without a link. A listing page is a lead, never the evidence (step 4).
    - A `maintainer-request` item: the series and URLs the comment names are the find, and the request settles its scope (step 2).
    - The `scout` item: search each of these, with `<year>` standing for this year and next (the year of the worklist's `today`, which the desk gives with the item).
      - Targeted web searches: "PyCon <year>", "PyData <year>", "Python conference call for proposals", "PyDay", "Python Pizza", framework and scientific library names with "conference", and the same in other languages ("conferencia Python", "Python Konferenz").

.github/workflows/archive-new-urls.yml

@@ -14,11 +14,7 @@ on:
   workflow_dispatch:
     inputs:
       skip_archive:
-        description: 'Skip Web Archive (only add to changedetection)'
-        type: boolean
-        default: false
-      skip_changedetection:
-        description: 'Skip changedetection.io (only archive)'
+        description: 'Skip the Web Archive (only report the new URLs)'
         type: boolean
         default: false
 

.github/workflows/canonical-source-update.yml

@@ -313,80 +313,43 @@ jobs:
           echo "commit_sha=$COMMIT_SHA" >> "$GITHUB_OUTPUT"
           echo "commit_msg=$COMMIT_MSG" >> "$GITHUB_OUTPUT"
 
+      # The rolling PR body has one block per writer (utils/desk_pr_body.py).
+      # This job owns COMMITTED:canonical and adds one line per commit there,
+      # naming the editions the commit changed. A commit that changed none,
+      # such as a re-quote or the sort moving entries between files, adds no
+      # line. The script comes from the commit this run started on: the
+      # accumulator branch checked out here may not have it yet.
       - name: Create or update PR
         if: steps.check_changes.outputs.changed == 'true' && inputs.dry_run != true
-        uses: actions/github-script@v9
         env:
-          UPDATE_BRANCH: ${{ env.UPDATE_BRANCH }}
+          GH_TOKEN: ${{ github.token }}
+          REPO: ${{ github.repository }}
           SOURCE: ${{ steps.source.outputs.source }}
           COMMIT_SHA: ${{ steps.commit.outputs.commit_sha }}
-          COMMIT_MSG: ${{ steps.commit.outputs.commit_msg }}
-        with:
-          script: |
-            const branch = process.env.UPDATE_BRANCH;
-            const source = process.env.SOURCE;
-            const sha = process.env.COMMIT_SHA.substring(0, 7);
-            const date = new Date().toISOString().split('T')[0];
-
-            // Source labels for the "Conference" column (matches existing workflow format)
-            // Using 📊 emoji to distinguish canonical source updates from website scrapes
-            const sourceLabels = {
-              'ics': '📊 PSF Calendar',
-              'csv': '📊 python-organizers',
-              'all': '📊 Canonical sources'
-            };
-            const sourceLabel = sourceLabels[source] || `📊 ${source}`;
-
-            // Check for existing PR from the update branch
-            const { data: prs } = await github.rest.pulls.list({
-              owner: context.repo.owner,
-              repo: context.repo.repo,
-              head: `${context.repo.owner}:${branch}`,
-              state: 'open'
-            });
-
-            // Use same table format as check-conference-update.yml (Conference | Commit | Date)
-            const entry = `| ${sourceLabel} | [${sha}](${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/commit/${process.env.COMMIT_SHA}) | ${date} |`;
-
-            if (prs.length > 0) {
-              // Update existing PR - append to table
-              const pr = prs[0];
-              let body = pr.body || '';
-
-              // Add entry before the END_UPDATES marker
-              body = body.replace(/(<!-- END_UPDATES -->)/, `${entry}\n$1`);
-
-              await github.rest.pulls.update({
-                owner: context.repo.owner,
-                repo: context.repo.repo,
-                pull_number: pr.number,
-                body
-              });
-
-              console.log(`Updated PR #${pr.number}`);
-              core.notice(`Updated PR #${pr.number} with canonical source merge`);
-            } else {
-              // Create new PR - use EXACT same format as check-conference-update.yml
-              const { data: newPr } = await github.rest.pulls.create({
-                owner: context.repo.owner,
-                repo: context.repo.repo,
-                title: '🐍 Conference updates',
-                head: branch,
-                base: 'main',
-                body: `## 🐍 Automated Conference Updates\n\n| Conference | Commit | Date |\n|------------|--------|------|\n${entry}\n<!-- END_UPDATES -->`
-              });
-
-              // Add labels
-              await github.rest.issues.addLabels({
-                owner: context.repo.owner,
-                repo: context.repo.repo,
-                issue_number: newPr.number,
-                labels: ['automated', 'conference-update']
-              });
-
-              console.log(`Created PR #${newPr.number}`);
-              core.notice(`Created PR #${newPr.number} for canonical source merge`);
-            }
+        run: |
+          SCRIPT="$RUNNER_TEMP/desk_pr_body.py"
+          git show "$GITHUB_SHA:utils/desk_pr_body.py" > "$SCRIPT"
+          case "$SOURCE" in
+            ics) LABEL="📊 PSF Calendar" ;;
+            csv) LABEL="📊 python-organizers" ;;
+            *) LABEL="📊 Canonical sources" ;;
+          esac
+
+          PR=$(gh api "repos/$REPO/pulls?head=${REPO%%/*}:$UPDATE_BRANCH&state=open" --jq '.[0].number // empty')
+          if [ -z "$PR" ]; then
+            pixi run python "$SCRIPT" skeleton > "$RUNNER_TEMP/body.md"
+            PR=$(gh api "repos/$REPO/pulls" -f title='🐍 Conference updates' -f head="$UPDATE_BRANCH" -f base=main \
+              -F body=@"$RUNNER_TEMP/body.md" --jq .number)
+            gh api "repos/$REPO/issues/$PR/labels" -f 'labels[]=automated' -f 'labels[]=conference-update' --silent
+            echo "::notice title=PR Created::Opened #$PR for the canonical source merge"
+          fi
+
+          pixi run python "$SCRIPT" --repo "$REPO" commit-line --sha "$COMMIT_SHA" --label "$LABEL" > "$RUNNER_TEMP/line.md"
+          if [ -s "$RUNNER_TEMP/line.md" ]; then
+            pixi run python "$SCRIPT" --repo "$REPO" --pr "$PR" append COMMITTED:canonical "$RUNNER_TEMP/line.md"
+          else
+            echo "::notice title=No Edition Changed::$COMMIT_SHA changed no edition, so #$PR gets no line for it"
+          fi
 
       - name: Trigger TBA enrichment
         if: steps.check_changes.outputs.changed == 'true' && inputs.dry_run != true

.github/workflows/check-conference-update.yml

@@ -3,11 +3,17 @@ name: Check Conference Update
 # A changedetection.io watch saw a conference page change. The change goes to
 # the desk queue as it is: the desk checks it against the data and the live
 # site on its next run, so nothing is filtered or edited here.
+#
+# conference-change comes from a watch on one conference's pages and queues a
+# signal for that series. listing-change comes from a watch on a page that
+# lists many events (pydata.org/upcoming-events, barcamps.eu and the like) and
+# queues a registry item, which the worklist always gives the scouter: the
+# names and links the diff adds are the finds.
 
 on:
   # Triggered by changedetection.io webhook
   repository_dispatch:
-    types: [conference-change]
+    types: [conference-change, listing-change]
 
   # Manual trigger for testing
   workflow_dispatch:
@@ -52,17 +58,20 @@ jobs:
 
           sys.path.insert(0, "utils")
           from desk_queue import clean_watch_title
+          from desk_queue import resolve_watch_url
 
           event_name = os.environ["GITHUB_EVENT_NAME"]
           with open(os.environ["GITHUB_EVENT_PATH"], encoding="utf-8") as f:
               event = json.load(f)
 
+          listing = False
           if event_name == "repository_dispatch":
               payload = event.get("client_payload") or {}
               url, title = payload.get("url"), payload.get("title")
               diff = payload.get("diff")
               watch_uuid = payload.get("watch_uuid") or None
               trigger = "website_change"
+              listing = event.get("action") == "listing-change"
           else:
               inputs = event.get("inputs") or {}
               url, title = inputs.get("url"), inputs.get("conference_name")
@@ -93,23 +102,26 @@ jobs:
           name, year = clean_watch_title(title)
           name = " ".join(name.split())  # one line, for the step output
           payload = {"diff": diff, "watch_uuid": watch_uuid, "trigger": trigger}
+          if listing:
+              payload["registry"] = "listing"
           if event_name == "workflow_dispatch":
               # The run id gives each manual request its own queue id, so
               # asking again with the same inputs is not taken as handled.
               payload["requested"] = os.environ["GITHUB_RUN_ID"]
           record = {
               "source": "changedetection",
-              "kind": "signal",
+              "kind": "registry" if listing else "signal",
               "title": title.strip(),
-              "url": url.strip(),
+              # A watch that follows a series sends its URL with the year template in it
+              "url": resolve_watch_url(url.strip()),
               "year": year,
               "payload": payload,
           }
           with open(os.environ["RECORDS"], "w", encoding="utf-8") as f:
               f.write(json.dumps(record, ensure_ascii=False) + "\n")
           with open(os.environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as out:
               out.write(f"series={name}\n")
-          print(f"Queueing a {trigger} signal for {name or title}")
+          print(f"Queueing a {trigger} {record['kind']} item for {name or title}")
           BUILD
 
       - name: Write to the desk queue