Skip to content

Deadline conflicts by CfP platform, block-based PR body, changedetection routing - #520

Merged
JesperDramsch merged 11 commits into
mainfrom
desk/deadline-rule-pr-body-cd-sync
Oct 28, 2026
Merged

JesperDramsch merged 11 commits into
mainfrom
desk/deadline-rule-pr-body-cd-sync

Conversation

@JesperDramsch

@JesperDramsch JesperDramsch commented Oct 28, 2026 •

Copy link
Copy Markdown
Owner

Three fixes to the conference-data automation: deadline conflicts no longer wait on the maintainer, the rolling PR body says what needs attention instead of repeating evidence, and changedetection signals carry the right URL.

Why

  • Deadlines. When a conference's site and its CfP platform disagree, the critic sent the change to the maintainer, so the site kept showing the old deadline. PyCascades 2027 (extension to 9 November) is waiting on exactly that. The platform that takes the submissions decides when the CfP closes, so it should settle the conflict. Anyone can open an event on pretalx or Sessionize under any name, though, so the rule must not become a way to inject data.
  • PR body. One desk run wrote about 15,000 characters, and two runs reached 39,000 of GitHub's 65,536. Most of it was an evidence table that repeats what every commit body already holds. The Actions writers edited the body with JavaScript string replacement, and the canonical import added a row even for commits that only re-quoted values or moved entries between files.
  • changedetection. The notification replaced each watch's year template with a literal 2026, so a watch on a 2027 page sent the desk the 2026 URL. The sync skipped any new URL on a domain it already watched, so on pydata.org or events.linuxfoundation.org only the first series ever got a watch. It also printed the first characters of the Cloudflare Access secret to the public Actions log.

What changes

  • AGENTS.md "Deadline conflicts", with matching checks in the critic and enricher skills. An anchored pretalx or Sessionize page wins for cfp, cfp_ext and timezone only. Anchored means it is the edition's existing cfp_link, or the conference's own domain links to it. A cfp_link the same change sets doesn't count. When the new zone differs from the zone the existing cfp was stated in, cfp is re-expressed at the same instant by utils/convert_deadline.py, never by hand.
  • utils/desk_pr_body.py now manages blocks. Each writer owns marker-delimited blocks under "Needs you" and "Committed"; a write replaces or extends one block and renders the layout again, keeping everyone else's blocks and ticked boxes. The first write moves an old-format body whole into a folded block. desk.md describes the desk's four blocks. The canonical and enrichment workflows call the script instead of editing the body in JavaScript, name the editions each commit changed, and skip commits that changed none.
  • changedetection intake and sync. The queue builder resolves a watch's year template itself (resolve_watch_url). The sync matches only by path on hosts that carry many series. The secret-prefix logging goes, and so does a dead input in archive-new-urls.yml.
  • Listing pages and the trademark list. Watches on pages that list many events (PyData and SciPy upcoming, barcamps.eu, fwdays) used to notify only the maintainer. They now send listing-change, which queues a registry item, and the worklist always gives that kind to the scouter, with the diff's new names and links as the finds. The PSF PyCon trademark list is read in full whenever its watch sends registry-change, instead of weekly. desk_leads.py gains psf-trademarks and pycon-org as their own sources, and pycon.org stays weekly. The queue builder inside check-conference-update.yml gets tests that run it straight from the workflow file.
  • Desk leads on change. Each lead source now runs when changedetection sees it change, and the schedule is only the fallback for a change a watch missed. registry-change names its source (developers-events, psf-trademarks or pycon-org, checked against that list). Watches on the Python rows of developers.events' TAGS.csv, on the trademark page and on the pycon.org registry's latest commit send it. pretalx-change names one CfP page, and desk_leads.py --url checks only the entries on that page. pretalx moves from daily to weekly, and the other three from weekly to monthly.
  • Archived links. When the data swaps a link for its Internet Archive copy, the sync pauses the watch that fetches that URL today. A year-templated series watch is compared by the page it gives today, so archiving an old edition's page leaves it on. A URL another entry still uses live keeps its watch too. The sync also stops adding watches on web.archive.org links, which never change.

After merging

  1. Switch the "Conference" tag's notification body from watch_url|regex_replace('{%.*?%}', '2026') to plain watch_url, so the year template reaches the queue builder unresolved. Until then, signals keep the hard-coded year.
  2. The open rolling PR converts to the new layout on its first write. Its old body stays readable in the folded block.

The changedetection side is already set up. The conference-helper tag sends listing-change. A Registry tag on the trademark, pycon.org and TAGS.csv watches sends registry-change, with the source taken from the watch title. A pretalx tag on the 20 pretalx.com watches sends pretalx-change. Until this merges, those dispatches find no listener and the current schedule still runs, so nothing is lost.

Rollover of year-templated watches

The sync used to set a watch's year-template offset to the conference's start month. That flipped some conferences to next year months before they ran and others months after. A watch now moves on 14 days after the effective CfP deadline (a deadline can still move in those two weeks), and at the latest when the conference starts; without a deadline, at the start. The template changes at the first month start on or after that day. utils/watch_templates.py holds the rule. The existing 79 templated watches on the instance were set the same way, and 16 of them now fetch a different edition's page than before.

Checks

  • pytest tests/ (pinned uv line from REVIEW.md): 2557 passed, 5 skipped, 3 xfailed, 2 xpassed.
  • ruff@0.4.4 check utils tests and black --check: clean.
  • The changed workflows parse as YAML. actionlint could not run in the session sandbox.
  • desk_pr_body.py --print against the live 🐍 Conference updates #519 body: renders, round-trips through parse, and keeps the old body intact.
  • commit-line on real commits: f5db91a gives "PyCon Togo 2026 (cfp_link, coc, finaid, timezone)". 873c697 and 08b0acf (re-quote and file moves only) give no line.

https://claude.ai/code/session_016a2XGjzRsX9nEb4swsKYms


Generated by Claude Code

A pretalx or Sessionize CfP page the conference itself points to (its cfp_link, or a link from its own domain) now wins for cfp, cfp_ext and timezone, and the critic accepts such a change instead of sending it to the maintainer. The rule covers deadlines only, and an unanchored platform page stays a lead, so a look-alike event on a CfP platform cannot use it. When the new zone differs from the zone the existing cfp was stated in, cfp is re-expressed at the same instant with utils/convert_deadline.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016a2XGjzRsX9nEb4swsKYms
The body now leads with what needs the maintainer, then what was committed, with the evidence left in the commit bodies the Committed lines link. desk_pr_body.py reads and writes one block at a time and renders the whole layout again, so the desk, the canonical import and the TBA enrichment keep each other's blocks and the maintainer's ticked boxes. A body from before the layout is kept whole in a folded block. The two Actions writers drop their JavaScript string edits, name the editions each commit changed, and add no line for a commit that changed none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016a2XGjzRsX9nEb4swsKYms
Signals now carry the URL the watch fetched: the queue builder fills in the watch's year template instead of trusting the notification. The sync no longer matches new URLs by domain on hosts that carry many series, so a second PyData or Linux Foundation event gets its own watch, and it stops printing the start of the Cloudflare Access secret. archive-new-urls drops an input that no longer did anything.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016a2XGjzRsX9nEb4swsKYms

Copy link
Copy Markdown
Owner Author

Routine review

1. Request

No linked issue, so the PR body is the request. It asks for three things:

  • Deadline conflicts. An anchored pretalx or Sessionize page settles cfp, cfp_ext and timezone, and a re-zoned cfp is converted by utils/convert_deadline.py. Covered in AGENTS.md, the critic and enricher skills, and desk.md step 6.3.
  • Block-based rolling PR body. utils/desk_pr_body.py (skeleton/get/set/append/commit-line), used by desk.md, canonical-source-update.yml and enrich-tba-conferences.yml.
  • changedetection fixes. resolve_watch_url in utils/desk_queue.py and check-conference-update.yml, MULTI_SERIES_HOSTS in sync-changedetection.yaml, the secret-prefix logging removed, and the dead skip_changedetection input in archive-new-urls.yml dropped.

2. Checked against

  • Date: Wed Oct 28 12:14:08 UTC 2026.
  • unverified: GitHub's PR-body limit of 65,536 characters (BODY_LIMIT). Taken from the PR and code, not fetched.
  • unverified: the jinja2-time syntax {% now 'utc' + 'months=N', '%Y' %} and the arrow shift units that WATCH_URL_TEMPLATE accepts. Not fetched.
  • In-repo, not external: the site treats a missing timezone as fixed UTC-12 (_includes/index_conf_title_row.html:39, static/js/timezone-utils.js:10). That matches AOE = timezone(timedelta(hours=-12)) in utils/convert_deadline.py.

3. Needs your decision

  1. Trust rule for deadlines. The PR makes an anchored CfP platform authoritative for cfp/cfp_ext/timezone without a maintainer step (AGENTS.md "Deadline conflicts"). This is a policy change to what the desk may publish unattended. The anchor requirement and the scope limited to these fields keep it narrow. Recommendation: keep it.
  2. One PR or three. The PR bundles three independent changes (deadline rule, PR-body layout, changedetection). Recommendation: keep them together. Each is small, and the PR-body and deadline changes both touch desk.md, so splitting would mostly create merge churn.
  3. Manual step after merging. The changedetection "Conference" tag notification body has to change from regex_replace('{%.*?%}', '2026') to plain watch_url after merge. Until it does, resolve_watch_url gets nothing to resolve. Recommendation: do it right after merging, before the year changes in January.

4. Findings

Important: None.

Nits

  • Possible missed match (judgement call). .github/workflows/sync-changedetection.yaml:523-524: on a MULTI_SERIES_HOSTS host, url_exists now matches only exactly or by the /YYYY/ path prefix (YEAR_PATTERN). Say a watch is templated inside a path segment, such as https://pydata.org/london{% now … %}/. A data URL like https://pydata.org/london2026/ then matches neither, and the sync adds a second, dated watch. Before this change, the base-URL match hid that (along with every other series on the host). Worth a look only if templated watches exist on those hosts.
  • .github/workflows/canonical-source-update.yml / enrich-tba-conferences.yml: COMMITTED:canonical and COMMITTED:enrich only ever grow. Once the body would pass BODY_LIMIT, append exits 2, so the "Create or update PR" step fails after the data commit is already pushed. The desk prunes its own MORE:desk, but nothing prunes the Actions writers' blocks.
  • enrich-tba-conferences.yml:282-287: NEEDS:enrich is replaced only when a run finds differing links. A later run that finds none leaves the earlier list, including any items it no longer reproduces. The step comment documents this, so it may be intended.

5. Tests

  • Invariant coverage. tests/test_convert_deadline.py checks the conversion invariant, same instant across zones (round trip, AoE in both directions, DST on both sides of the November change). tests/test_resolve_watch_url.py covers each template unit (years/months/days, %Y/%y, several templates in one URL, an unknown shape left as it is). tests/test_desk_pr_body.py has 38 tests. All three modules under test are new, so these tests cannot pass on the base revision.
  • Not run here. The session's permission classifier refused the pytest run on the PR head, so I could not run any test, on the base or on the change. See Bounds.
  • Lint, change: uvx ruff@0.4.4 check utils tests → All checks passed!
  • pre-commit.ci on the head: pre-commit.ci - pr: success — checks completed successfully
  • Other CI on 08aaea0 at review time: CodeQL, Data Validation, Pre-commit Checks and the Analyze jobs succeeded. Python Tests was in_progress and had not finished when this review posted.

6. Bounds

  • Route. Cloned the repository, fetched refs/pull/520/head and diffed against origin/main. The PR, comments and check reads went through REST gh api.
  • Head moved. The trigger named efe6f3a, but the PR head is now 08aaea0: a pre-commit.ci auto-fix commit that touches only utils/desk_pr_body.py (+13/-9) sits on top of efe6f3a. I reviewed 08aaea0.
  • Not run. pytest was blocked by the session's permission classifier, so there was no base-vs-change run. The REVIEW.md data check (sort_yaml.py --skip_links) was not run either, since no _data/ file changes. actionlint was not run.
  • Read in full: utils/desk_pr_body.py, utils/convert_deadline.py, the utils/desk_queue.py diff, both new test files, the AGENTS.md / critic / enricher / desk.md diffs, the two Actions "Create or update PR" steps, and url_exists with its helpers in sync-changedetection.yaml.
  • Skimmed: tests/test_desk_pr_body.py, and the parts of .claude/hooks/desk_guard.py that concern desk_pr_body.py. The guard still blocks subagents from running it, which is fine because only the desk writes the body.
  • Pre-existing:
    • url_exists lowercases the candidate but existing_urls stores the raw URL (sync-changedetection.yaml:507), so the exact match misses any watch URL with uppercase letters.
    • The read-modify-write of the PR body can still race between writers. desk_pr_body.py narrows the window but does not close it.

Generated by Claude Code

A changedetection watch on a page that lists many events (PyData and SciPy upcoming, barcamps.eu, fwdays) now sends listing-change, which queues a registry item for the scouter with the diff as the finds. The PSF PyCon trademark list is read in full whenever its watch sends registry-change, instead of weekly; pycon.org stays on the weekly run. desk_leads.py gains psf-trademarks and pycon-org as sources of their own, and registry still reads both. The queue builder inside check-conference-update.yml gets tests that run it straight from the workflow file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016a2XGjzRsX9nEb4swsKYms
A link swapped for its Internet Archive copy is dead or on a domain someone else now owns, so its watch only reports that. The sync now pauses the watch that fetches it today (a year template is compared by the page it gives today, so a series watch that moved on to a later edition stays on), unless another entry still uses the URL live. It also stops adding watches on web.archive.org links, which never change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016a2XGjzRsX9nEb4swsKYms
@JesperDramsch JesperDramsch changed the title Deadline conflicts by CfP platform, block-based PR body, changedetection fixes Deadline conflicts by CfP platform, block-based PR body, changedetection routing Oct 28, 2026
claude and others added 5 commits October 28, 2026 15:51
registry-change now names its source (developers-events, psf-trademarks or pycon-org, checked against that list), and pretalx-change names one CfP page, for which desk_leads.py --url checks only the entries on that page. The schedule becomes the fallback for a change a watch missed: pretalx weekly instead of daily, the other three monthly instead of weekly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016a2XGjzRsX9nEb4swsKYms
A watch on a pretalx page sends its URL with the year template unresolved, and the template has spaces, so refusing all whitespace refused every templated watch. Only a line break could add step outputs, so only that is refused now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016a2XGjzRsX9nEb4swsKYms
… closes

The sync set a watch's year-template offset to the conference's start month, which rolled some series over months before their conference and others months after it. The rollover is now 14 days after the effective CfP deadline, when the deadline can no longer move, and at the latest the start, so the watch still sees some of the site's updates; with no deadline it is the start. The template changes at the first month start on or after that day. utils/watch_templates.py holds the rule and its tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016a2XGjzRsX9nEb4swsKYms
@JesperDramsch
JesperDramsch merged commit c9b48e1 into main Oct 28, 2026
11 checks passed
@JesperDramsch
JesperDramsch deleted the desk/deadline-rule-pr-body-cd-sync branch October 28, 2026 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants