Skip to content

feat(reputation): Google Postmaster Tools and SNDS on the dashboard - #100

Merged
clbotdev merged 1 commit into
mainfrom
issue-76-reputation
Oct 19, 2026
Merged

clbotdev merged 1 commit into
mainfrom
issue-76-reputation

Conversation

@clbotdev

Copy link
Copy Markdown
Contributor

Part of #79. Closes #76.

Changed premises (see ADR 0004)

  • Google retired domain and IP reputation in Postmaster Tools v2. This PR uses the v2 API: SPAM_RATE, AUTH_SUCCESS_RATE for SPF/DKIM/DMARC, DELIVERY_ERROR_RATE and getComplianceStatus. Google's compliance verdict replaces the "domain reputation Low/Bad" alert.
  • SNDS's new REST API is undocumented, and its OAuth tokens last 8h with no refresh token. This PR uses the automated data access link, which now expires every 30 days. Expiry is detected and the user is asked for a new link.
  • Uses the narrowest Google scope, postmaster.traffic.readonly. SendLit checks the team's ESP From domains instead of listing Postmaster domains, which would need a domain-management scope. Domains not verified in Postmaster Tools are flagged.

What

  • DB (migration 0020):

    • reputation_connections: per team and provider; encrypted refresh token or SNDS link, status, last error, next_sync_at, alert state, per-domain state.
    • reputation_daily_stats: subject, day and metrics, unique per connection, subject and day.
  • API /reputation:

    • GET
    • POST google/connect (returns the sign-in URL)
    • DELETE google
    • PUT snds (validates by fetching)
    • DELETE snds
    • POST sync (once per 5 min)
    • Public OAuth callback /integrations/google-postmaster/callback with a signed 15-minute state; redirects back to the dashboard.
    • Writes require a team admin or a team key.
  • Sync job: a loop every 10 minutes (REPUTATION_SYNC_CHECK_SECONDS) atomically claims due connections and syncs each daily. Outages retry after 1h; revoked Google access or an expired SNDS link waits for the user.

  • Alerts: Gmail spam ≥0.1% is a warning and ≥0.3% critical; compliance needs work; SNDS yellow or red. They show on the card and email org owners only when new or worse.

  • SNDS safety: links are fetched only from substrate.office.com and sendersupport.olc.protection.outlook.com over https with redirect: manual (plus SNDS_ALLOWED_HOSTS for tests). The parser handles the header-less export, a header row, and HTML error pages.

  • MCP: get_reputation, sync_reputation, connect_google_postmaster, disconnect_google_postmaster, connect_snds, disconnect_snds (settings scopes; disconnects marked destructive).

  • Web: a "Sender reputation" card on the dashboard. It has:

    • a Gmail spam-rate line chart with 0.1% and 0.3% lines;
    • pass rates and compliance badges;
    • SNDS daily filter strips;
    • alert banners and empty states that explain setup.

    shadcn chart was added via the CLI. Fixed the CLI's bogus cn import again, and added biome-ignores on upstream markup as for the other generated components.

  • Docs: email-marketing/reputation, MCP docs, .env.example (GOOGLE_POSTMASTER_CLIENT_ID/_SECRET), ADR 0004.

Verification

  • check.sh (725 API tests incl. 10 new: parser, host allowlist, Google stats/compliance, alert dedupe, reauth, SNDS expiry, single claim, OAuth callback/state), check:drift, test:conformance, docs build.

  • Browser E2E (8/8) against a local stand-in for Google OAuth, Postmaster v2 and an SNDS link:

    1. Empty states.
    2. Connect → consent page → back with success; the first sync shows the chart, "Meets Google's requirements", and "Not in Postmaster Tools" for an unverified domain.
    3. SNDS rejects a non-Microsoft link and connects the right one, showing per-IP strips and a red alert.
    4. A spam spike gives critical and compliance alerts, an owner email (logged outside production), and a second Refresh is rate-limited.
    5. The background job marks an expired SNDS link and keeps the history.
    6. Secrets are not stored in plaintext.
    7. Disconnecting deletes the data.

    This run also caught an invisible chart line (the theme has no --chart-1; now uses --primary and no animation).

Before launch on sendlit.app

  • Create a Google OAuth client with redirect https://api.sendlit.app/integrations/google-postmaster/callback, enable the Gmail Postmaster Tools API, and get the consent screen verified for the Postmaster scope.
  • Check rate units and the SNDS CSV layout against a live account (they're taken from the API reference and the SNDS export format).

A Sender reputation card on the dashboard shows 30 days of Gmail data per
sending domain (Postmaster Tools v2: user-reported spam rate, SPF/DKIM/DMARC
pass rates, delivery errors, compliance with Google's sender requirements)
and Outlook data per IP (SNDS filter result, complaint rate, trap hits).

Google connects over OAuth with the read-only traffic scope; SNDS through
its automated data access link, fetched only from Microsoft's hosts. Tokens
and links are stored encrypted. A background job syncs each connection
daily, and alerts (Gmail spam >= 0.1% / 0.3%, compliance needs work, SNDS
yellow/red) show on the card and email the owners when they start or get
worse. REST, MCP, docs and ADR 0004.

Google retired domain/IP reputation grades in Postmaster Tools v2, so the
compliance verdict replaces the Low/Bad reputation alert.

Closes #76
@clbotdev
clbotdev merged commit efe6554 into main Oct 19, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Show Google Postmaster Tools and Microsoft SNDS reputation data on the dashboard

2 participants