Skip to content

Log skipped TUH recordings and accept float rates in EEGAbnormalTUAB - #1474

Open
AxelNoun wants to merge 4 commits into
sunlabuiuc:masterfrom
AxelNoun:fix/tuh-task-robustness
Open

AxelNoun wants to merge 4 commits into
sunlabuiuc:masterfrom
AxelNoun:fix/tuh-task-robustness

Conversation

@AxelNoun

@AxelNoun AxelNoun commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Contributor: Axel Coiffard-Donaty (@AxelNoun)

Contribution Type: Bug fix

Description:

Part of #1473.

  • Skipped recordings are now reported. EEGEventsTUEV and EEGAbnormalTUAB used to skip a recording they could not read or convert without any message. TUEV also contains linked-ear recordings, which have no EEG XX-REF channels, so they were dropped silently. The skip itself is unchanged, as in the BIOT preprocessing, so the samples produced are the same. A logger.warning now gives the file and the reason.
  • A TUAB recording without one of the 16 electrodes of the bipolar montage is skipped. convert_to_bipolar ran outside the try block, so the missing channel raised a ValueError that stopped set_task. It is now skipped with a warning, like an unreadable recording (suggestion of the AI review).
  • Float sampling rates work. EEGAbnormalTUAB used fs * 10 as a slice index, so a float resample_rate (the parameter is typed float) raised TypeError: slice indices must be integers. The window length is now an int.
  • Docstrings:
    • TUEV resamples to resample_rate (200 Hz by default, not 256 Hz);
    • both tasks return torch.FloatTensor signals;
    • offending_channel is the channel number from the .rec file, which indexes the 22-channel TCP montage (0-21), not a row of signal.

No public API change. The samples produced from the recordings that were already processed are unchanged.

Note: #1293 also edits the EEGEventsTUEV docstring. The overlap is a few lines and easy to resolve, whichever PR lands first.

Files to Review:

  • pyhealth/tasks/temple_university_EEG_tasks.py: warnings for skipped recordings, int window length in EEGAbnormalTUAB, docstrings.
  • tests/core/test_tuab.py: float resample_rate (200.0, 256.0), and the warnings for an unreadable file and for a recording missing an electrode.
  • tests/core/test_tuev.py: a -REF recording is used, and a -LE recording is skipped with a warning.

Testing:

  • pytest tests/core/test_tuab.py tests/core/test_tuev.py on Linux: 33 passed, 2 subtests passed.
  • Without the change in temple_university_EEG_tasks.py, the new tests fail: 4 failures, including both float-rate subtests. The test for a missing electrode fails with ValueError: 'EEG T5-REF' is not in list without the last commit.
  • The EEG/signal test subset (19 files) on Linux: 219 passed, 1 skipped (it needs a real download), 0 failed.
  • python tools/check_pr_rules.py --base upstream/master --head HEAD: passed.

Claude helped me with this PR.

EEGEventsTUEV and EEGAbnormalTUAB skipped a recording they could not
read or convert without any message. TUEV also contains linked-ear
recordings, which have no EEG XX-REF channels, so the task drops them
without telling the user. Keep the skip, as the BIOT preprocessing does,
but log a warning with the file and the reason.

EEGAbnormalTUAB used fs * 10 as a slice index, so a float resample_rate
(the parameter is typed float) raised TypeError. Compute the window
length as an int.

Fix the docstrings: TUEV resamples to resample_rate (200 Hz by default,
not 256 Hz), both tasks return torch tensors, and offending_channel
indexes the 22-channel TCP montage of the .rec file, not the 16 rows of
signal.
@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Test Coverage

Total: 84.95% (39734 / 46772 lines)

Modified file Coverage Lines
pyhealth/autoimprove/__init__.py 100.00% 10 / 10
pyhealth/autoimprove/llm.py 97.09% 100 / 103
pyhealth/calib/predictionset/__init__.py 100.00% 10 / 10
pyhealth/calib/predictionset/group_conditional.py 95.04% 115 / 121
pyhealth/datasets/__init__.py 100.00% 64 / 64
pyhealth/datasets/helsinki_neonatal.py 97.06% 66 / 68
pyhealth/datasets/hup_ieeg.py 96.00% 96 / 100
pyhealth/datasets/pbc.py 96.77% 30 / 31
pyhealth/metrics/__init__.py 100.00% 16 / 16
pyhealth/metrics/competing_risks.py 97.22% 70 / 72
pyhealth/models/__init__.py 100.00% 78 / 78
pyhealth/models/behrt.py 93.37% 183 / 196
pyhealth/models/daft.py 93.64% 221 / 236
pyhealth/models/deepsleepnet.py 99.22% 127 / 128
pyhealth/models/neural_fine_gray.py 92.59% 125 / 135
pyhealth/models/sleepkd.py 98.80% 82 / 83
pyhealth/tasks/__init__.py 97.26% 71 / 73
pyhealth/tasks/competing_risks.py 88.46% 46 / 52
pyhealth/tasks/competing_risks_pbc.py 100.00% 56 / 56
pyhealth/tasks/medmod_decompensation_mimic4.py 95.40% 83 / 87
pyhealth/tasks/seizure_detection_helsinki.py 100.00% 149 / 149
pyhealth/tasks/sleep_staging_v2.py 98.75% 79 / 80
pyhealth/tasks/soz_localization_hup.py 98.96% 190 / 192
pyhealth/tasks/temple_university_EEG_tasks.py 82.39% 145 / 176

Workflow run · 1af4ac3

Copy link
Copy Markdown
Collaborator

/review


Generated by Claude Code

@solarsys solarsys added the status: automatic The PR will go through automatic review process. label Nov 16, 2026
@solarsys

Copy link
Copy Markdown
Collaborator

/review

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previous review

Reviewed commit 560d5c0

github-actions[bot]
github-actions Bot previously approved these changes Nov 16, 2026

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previous review

AI Review

suggestion: approve
decision: approve

Thank you for this contribution. It makes skipped TUEV and TUAB recordings visible through logged warnings and fixes the TypeError that a float resample_rate caused in EEGAbnormalTUAB. The docstring corrections are also useful.

The change stays inside the two task classes and doesn't touch shared code. The description matches the diff. The new tests cover float rates, the unreadable-file warning, a -REF recording being used, and a -LE recording being skipped with a warning. Coverage of temple_university_EEG_tasks.py is 81.98%, which meets the 80% bar. Whole-repo coverage is 83.31%, which is above the 60% bar. It looks ready to merge.

Optional suggestions:

  • round(fs * 10) shortens the window slightly for non-integer rates such as 200.05. You could note in the docstring that the shape 10 * resample_rate assumes the product is a whole number of samples.
  • In EEGAbnormalTUAB, convert_to_bipolar is outside the try block. A recording missing an electrode would raise instead of being skipped with a warning. This was already the case before your change, so it is fine to leave for a follow-up.

Reviewed commit 560d5c0

Resolve conflicts in the TUAB and TUEV tests by keeping both the new robustness tests and the metadata cache tests from sunlabuiuc#1470.
The window is round(10 * resample_rate) samples, so it is exactly 10 s only when that product is a whole number.
convert_to_bipolar ran outside the try block, so a recording without one
of the 16 EEG XX-REF electrodes raised a ValueError and stopped set_task.
It is now skipped with a warning, like an unreadable recording.
@AxelNoun

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed commit 1af4ac3

Each returned sample contains:
- "signal": np.ndarray, shape (16, 256*5)
- "offending_channel": int
- "signal": torch.FloatTensor, shape (16, 5 * resample_rate)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] TUEV shape docstring is wrong for non-integer resample_rate

The docstring says the signal shape is (16, 5 * resample_rate), but BuildEvents allocates int(fs) * 5 samples and slices with int(fs), so for example resample_rate=200.5 gives 1000 samples, not 1002.5. Either write the shape as (16, 5 * int(resample_rate)), as the TUAB docstring does with round(), or note that the shape holds for integer rates.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review

suggestion: approve
decision: approve

Thank you for this contribution, Axel. It makes skipped TUEV and TUAB recordings visible through warnings. It also skips TUAB recordings that lack one of the 16 electrodes instead of crashing, lets EEGAbnormalTUAB accept float resample rates, and fixes several docstrings. The change stays inside the two self-contained task classes. It adds no new links between components, and the tests match the described behavior. It looks ready to merge.

Coverage: temple_university_EEG_tasks.py is at 82.39%, above the 80% bar. The whole repository is at 84.95%, above the 60% bar.

Optional suggestions:

  • The TUEV docstring says the signal shape is (16, 5 * resample_rate). That isn't exact for non-integer rates, because BuildEvents uses int(fs). You could add a short note like the one in the TUAB docstring.
  • The missing-electrode warning shows Python's raw "'EEG T5-REF' is not in list" text. A message that names the missing channel directly would read more clearly.

Reviewed commit 1af4ac3

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: automatic The PR will go through automatic review process.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants