Skip to content

Coverage reaches Codecov: every upload signs in with OIDC - #2162

Merged
Rafael-SOWNet merged 2 commits into
masterfrom
coverage-uploads-are-authenticated-by-oidc
Oct 27, 2026
Merged

Rafael-SOWNet merged 2 commits into
masterfrom
coverage-uploads-are-authenticated-by-oidc

Conversation

@Rafael-SOWNet

@Rafael-SOWNet Rafael-SOWNet commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Codecov has kept no coverage from this repository's branches for a long time. Its newest master commit is from 2022-11-28 (90c00a8, 80.99%), which is the figure the README badge still shows. The newest pull request upload it holds came from a fork on 2026-08-04.

Why: Codecov takes an upload without a token only from a fork's branch. From a branch of this repository it answers:

error - Upload queued for processing failed: {"message":"Token required because branch is protected"}

codecov-action logs that error and the step still passes. None of the four legs has a token. Today's master runs of C# Test, F# Test, Interactive Test and C++ Test all log that line, and so does #2161's C++ Test. So everything #798 and #1038 set up for #397 collected coverage that Codecov rejected.

The change:

  • Every upload step sets use_oidc: true, in a job granted id-token: write and contents: read.
    • The job asks GitHub for a token that lasts for the run, so no secret is stored, and there is nothing for a maintainer to set up.
    • On a fork's pull request the action skips OIDC (CC_FORK, action.yml line 237 at the v5 SHA these runs use). GitHub gives forks no id-token, and Codecov takes their upload without one, as before.
    • The block grants only those two permissions. The jobs use nothing else: checkout, setup-dotnet, upload-artifact and codecov.
  • fail_ci_if_error is set for pushes to master only. A refused upload then turns master red instead of passing silently, while an outage at Codecov cannot fail a pull request.
  • cpp is declared in codecov.yml. Cover the fourth test leg, and watch the SymPy snapshot for drift (#746 items 35, 73) #1038 uploads it but never declared it, so it had no path scope and no carry-forward, which the file's own comment says every flag needs. It is now scoped to Sources/Wrappers/AngouriMath.CPP.Importing/.
  • Coverage.md has a section on how a report reaches Codecov, and how to check that it did.

Verification on this PR:

  • Every upload step on b662b2844 logged a token (Token length: 2031, for example) and then Upload queued for processing complete with no error after it. That covers all four legs, including C# from Windows. Codecov then had four sessions for the commit.
  • codecov.yml now validates (second commit). Codecov reads it: codecov/patch, which the file turns off, is no longer posted.
  • The master case can only be checked on the first push after merge. I'll read those four runs and Codecov's commit list then.

Found once the uploads landed: the C# report is empty. Codecov's per-flag totals on ed6657252:

flag lines hits
cpp 135 51
fsharp 185 61
interactive 442 180
csharp 75,114 0

The report itself says the same. The Report artifact's summary reads visitedSequencePoints="0" of 81,664, and coverlet's table on the Linux and macOS legs prints AngouriMath | 0% | 0% | 0%. Locally, coverlet 3.0.3 records 11.43% from one test class, so the cause lies in the full run, not in the coverlet version alone. I'm bisecting that separately. Until it is fixed, master's project figure will read near 0%, which is true of what has been measured. codecov/project fails on this PR for that reason, and because its base is the 2022 commit. It is not a required check.

Part of #397. Its C++ box was done in #1038, but none of the four flags has reached Codecov since.

🤖 Generated with Claude Code

https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura

Codecov takes an upload without a token only from a fork's branch. From a
branch of this repository it answers "Token required because branch is
protected", and codecov-action logs that and finishes green. None of the
four legs had a token, so all four collected coverage and Codecov kept
none of it. The newest master commit it holds is 90c00a8, from
2022-11-28, and the newest pull request upload came from a fork on
2026-08-04. On today's master runs of C# Test, F# Test, Interactive Test
and C++ Test, and on #2161's C++ Test, every upload step logs that error.

Each upload step now sets use_oidc in a job granted id-token: write. That
gets a token for the run, so no secret is stored. On a fork's pull request
the action skips it and Codecov still takes the upload without a token.
fail_ci_if_error is set for pushes to master, so a refused upload turns
master red and an outage at Codecov cannot fail a pull request.

The cpp flag from #1038 was uploaded but never declared in codecov.yml,
so it had no path scope and no carry-forward. It is declared now, scoped
to the C++ wrapper that Coverage.md says it measures.

Part of #397

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura
@Rafael-SOWNet Rafael-SOWNet added this to the 2.7.0 milestone Oct 27, 2026
@Rafael-SOWNet Rafael-SOWNet moved this from Todo to In review in AngouriMath Oct 27, 2026
@codecov

codecov Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 0.38%. Comparing base (90c00a8) to head (ed66572).
⚠️ Report is 1231 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (90c00a8) and HEAD (ed66572). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (90c00a8) HEAD (ed66572)
1 0
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #2162       +/-   ##
==========================================
- Coverage   80.99%   0.38%   -80.62%     
==========================================
  Files         155     295      +140     
  Lines       13687   75876    +62189     
  Branches     1957   19088    +17131     
==========================================
- Hits        11086     292    -10794     
- Misses       1990   75486    +73496     
+ Partials      611      98      -513     
Flag Coverage Δ
cpp 37.77% <ø> (?)
csharp 0.00% <ø> (?)
fsharp 32.97% <ø> (?)
interactive 40.72% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 249 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Codecov's validator rejected the file with "Error at ['coverage',
'changes']: unknown field". Since it was added in 2021, `patch: no` and
`changes: no` had sat directly under `coverage`, while Codecov reads them
under `coverage.status`. For an invalid file Codecov "keep[s] using the
last valid YAML provided" (https://docs.codecov.com/docs/codecov-yaml),
and this one had no valid version. So the flags #798 declared never
applied, and the patch status they meant to turn off has been posted
anyway: it appears on #2162.

Moved under status, the validator answers Valid! and reads back all four
flags with their paths and carry-forward. The comment and Coverage.md
give the command to check the next edit.

Part of #397

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura
@Rafael-SOWNet
Rafael-SOWNet merged commit b043d06 into master Oct 27, 2026
34 of 35 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in AngouriMath Oct 27, 2026
Rafael-SOWNet added a commit that referenced this pull request Oct 27, 2026
… it does not (#2165)

C# Test's Windows leg ran `dotnet test` under PowerShell, in a multi-line
step whose result is its last command's, CopyCovReport. On most master
runs since 2026-09-15 the test host crashed with a stack overflow, and the
step was green each time. Sampled from master:

  09-15  7,889 tests run, crashed  (RationalIntegralsTest)
  09-25  2,094, crashed            (RationalFunctionBesideTheRootOfAQuadraticTest)
  10-05  15,075, complete
  10-15  5,265, crashed            (APowerOfALinearOverAPowerOfAQuadraticIntegralTest)
  10-22  8,507, crashed
  10-27  5,681, crashed            (RationalIntegralsTest)

It was the only leg testing a Debug build, since its `dotnet test` named
no configuration. On a thread of a given stack size, the six rows of
RationalIntegralsTest.ALinearNumeratorOverASymbolicQuadratic, as written in
the test: Debug overflows 1 MB on the cubic row (654 repeats of the
InnerSimplified cycle), and Release passes all six on 512 KB. The library
has no Debug.Assert, so a Debug build tests nothing that Release does not.

The leg now tests Release, as the other two do, under bash, which fails
the step at the first failing command. The coverage report from the
crashed host recorded 0 of 81,664 sequence points, which is what Codecov
received after #2162. A new step fails on a report with no visited point,
as the C++ leg does for an empty one. Locally, the full suite under the
same collector records 90.57% of lines.

Part of #397


Claude-Session: https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Rafael-SOWNet added a commit that referenced this pull request Oct 27, 2026
…2172)

With no threshold, codecov/project fails on any fall at all from the base
commit's figure. Now that reports reach Codecov (#2162, #2165), that turns
ordinary changes red: #2171 was at -0.02%, five lines that a new guard
returns before. A threshold of 1% lets that movement through and still
fails what the status is for, such as a test leg whose report goes empty,
which moved the figure from 86.41% to 0.38%. Codecov's validator accepts
the file and reads the threshold back as 1.0.


Claude-Session: https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Rafael-SOWNet
Rafael-SOWNet deleted the coverage-uploads-are-authenticated-by-oidc branch October 28, 2026 10:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant