Repository navigation
Check whether a dose series supports tests in each cell type - #80
Merged
Merged
Conversation
DoseAxis checked its dose column, the finiteness and sign of each dose, and the rule that refuses two doses within 1e-8 of each other, in private helpers of its own module. A check of dose designs needs the same reading, and a second copy would let the two drift apart. The reading moves unchanged into cytepool/_doses.py: real_vector, checked_doses, column_doses and distinct_doses, with their messages. DoseAxis keeps its refusal of fewer than two distinct doses on the axis. The docstring addresses of dose_axes.py point to the new chapter's section on dose axes.
A dose series can fail per cell type in ways nothing in CytePool reports before fitting: filtering can leave a dose group without a sample in a depleted cell type, a dose processed in its own lane or plate cannot be told apart from that batch, which breaks the control of false positives (Lun and Marioni 2017), and 57 of 130 single-cell dose studies had no biological replicate per dose. check_dose_design(cube, design, dose=axis) counts the samples, cells and counts each dose group keeps in every cell type after the filter limma_voom applies, fits the dose-group model of each cell type by R's qr() rule, and calls a dose group estimable when its column, placed last, lies outside the span of the others, as limma's nonEstimable reports it. A dose group is confounded with a batch when the batches' indicators, those of a single sample pooled into one, absorb it beside the design's other terms. Each cell type is inferential, descriptive (no estimable treated group keeps a second sample, no residual df, or an unmodelled batch confounds a group) or refused, with a reason that says what to do. With rules="etap_2024" the report lists the design minimums of EPA's Transcriptomic Assessment Products (US EPA 2024, Sec. 3.4.1.3, 3.4.2 and 3.4.5.2) that the design misses, with what ETAP does when each is unmet: a dose level ETAP excludes leaves the status unchanged. validation/dose_design.py compares the counts with scanpy 1.12.4 and pandas, the residual df, estimable and confounded groups with R 4.4.3's qr() and limma 3.62.1's nonEstimable on 20 designs, the statuses and ETAP's minimums with the same rules coded in R, and runs GSE208161, one participant per dose, where every cluster is descriptive: 41 checks, all identical.
…heck Dose axes were explained inside "Designs, linear models and contrasts", and nothing said what a number of dose groups supports or how to check a dose series before fitting it. A new chapter, "Dose response", after "Composition tests", takes the section on dose axes and the vehicle and adds what K dose groups support (the degrees of freedom for dose, the largest spline, curves that leave a spare group, and what MCP-Mod and ETAP ask), how check_dose_design decides each status, with an example that runs in the tests, and ETAP's design minimums with what ETAP does when each is unmet. The Validation page gains a "Dose response" group with the dose axes section and the new dose designs section, its summary rows, totals and command; dose_axes.py ran again with its table unchanged. "Choosing a method", the glossary (dose group, descriptive, rule set), the bibliography (US EPA 2024, EMA 2014, Sonigra 2022), the API reference and THIRD_PARTY_NOTICES.md (GEO series GSE208161) follow.
The module docstring named check_dose_design and DoseDesignReport without their top-level path, and the first lines of four properties held a colon, which the API reference reads as a type. Both now build without warnings.
The dose design check counted samples as replicates even when the design named the animal or participant as a random intercept, so one animal sampled twice at each dose looked replicated. A random intercept of the batch excused every dose it confounded, also when the vehicle sat alone in one lane, where a review simulation put 17 to 19% of null genes below p = 0.05. A batch holding a single sample, when it was the only one, absorbed that sample and marked its dose as confounded. With random intercepts, the replicates of a dose group are now the levels of each grouping among its samples. A random intercept of the batch, found by the grouping it makes rather than by its name, compares the doses it confounds with the vehicle only when the vehicle and each such dose span two or more batches. A sample alone in its batch joins no batch. In the report, n_dose_groups counts the dose groups that keep a sample and n_dose_groups_estimable those the design can compare with the vehicle; minimums names its consequence column when_unmet; and the printout names absorbed doses and the levels ETAP excludes. The sample filter is shared with limma_voom's, relations come from one QR, and the chapter shows how to act on the report, how to check one compound of a screen, and that the batch rule finds exact confounding only. validation/dose_design.py now checks 46 cases, five of them new, against R's qr(), limma's nonEstimable and the new rules coded again in R, and all match. The tests catch 27 deliberate bugs, among them the 12 mutants that survived the review.
The section on simulated dose series followed the dose axes in "Designs, linear models and contrasts", and the dose axes now open the "Dose response" chapter. The section, its limits, its checks and its further reading move there after ETAP's design minimums, and its Validation section and summary row join the chapter's group on the Validation page. Every link to the section points at its new address: the docstrings of simulate_dose_series and DoseTruth, the API page, the glossary's benchmark dose, coverage and EC50, "Choosing a method", the Validation page and the chapter test. The chapter now opens with what simulated series show, and the bibliography page lists the works each chapter cites.
The comment on the field batch_modelled of the check's inputs ended mid-sentence; it now says that a random intercept groups the samples as the batch does.
The Dose response chapter now cites the agency's 2024 standard methods and its 2026 BMDS guide, whose bibliography entries named it as US EPA and as the US Environmental Protection Agency. Both entries now spell it out, as the entry of the European Medicines Agency does, and the chapter's reading list says "the agency's standard methods" so that the name is not repeated.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
A dose analysis compares each dose with the vehicle, the samples given no dose, and judges every difference against the variation between replicates given the same dose. In a single-cell dose series that comparison can fail in one cell type and not another, and nothing in CytePool says so before a fit:
The solution
cp.check_dose_designreports, for every cell type of a cube, what its dose series supports, before anything is fitted:limma_voomwould use (min_cells,min_counts).dosesgives, per cell type and dose group, the samples used and left out and the fewest cells and counts;samplesgives each sample's cells, counts and why it was left out, inlimma_voom's words.qr()rule, and calls a dose group estimable when its column, placed last, lies outside the span of the others, as limma'snonEstimablereports it.(1 | animal), are one replicate.batch, a dose group is confounded when it is estimable and the batches absorb it beside the design's other terms; batches of a single sample are pooled into one, and a sample alone in its batch, when it is the only one, joins none. A lane per sample confounds nothing; the vehicle alone in its lane confounds every dose. The rule finds exact confounding, and the documentation advises adding a known batch to the design whenever it is unbalanced across the doses.rules="etap_2024",minimumslists each of ETAP's minimums that samples show, met or not, with what ETAP does when it is unmet (when_unmet): a deviation from the design, a dose level excluded, or a dose response not modelled.rules_statusis "deviation" when a minimum of the design or a vehicle minimum is unmet; a level that ETAP excludes leaves it "met", since ETAP's procedure goes on without it, and the printout names the level.A new chapter, "Dose response", after "Composition tests", takes the sections on dose axes and on simulated dose series (#79) from "Designs, linear models and contrasts" and adds what a number of dose groups supports (from one comparison at 2 groups to ETAP's 9), the check with an example that the tests run, and ETAP's minimums.
Why it matters
PR 7's
dose_responseruns a cell type only where the check allows, on the samples it marks used, and gives descriptive cell types no p-values; PR 10 counts its spare dose groups fromn_dose_groups_estimable; PR 14 applies ETAP's exclusion of levels, which the report already lists; PR 16's records carry the unmet minimums. On GSE208161, one participant per dose level sampled before and after dosing (Sonigra et al. 2022), 13 of 18 clusters keep one residual degree of freedom, the placebo participant's change from baseline, andlimma_voomwould fit them; the check calls all 18 descriptive. The PR builds on PR 3'sDoseAxis(#78) and on PR 1's recorded runs (#77), and shares the reading of dose columns with PR 6's simulator (#79).Validation
validation/dose_design.py, run on Modal: 46 checks, 44 pass and 2 information, none failed; second of two runs in one container 6.8 s and 520 MiB.get.aggregate, pandas 3.1.0min_counts0 and 10,678, on 7,277 simulated cells of a TCDD-like seriesqr(), limma 3.62.1nonEstimable; statuses by the rules coded again in R~ patient~ 1, 18 descriptive; with the day-29 samples alone, 16 descriptive and clusters 14 and 17 refusedvalidation/dose_axes.pyran again, since its checked code changed (the reading of dose columns and the axis's column names moved intocytepool/_doses.py), and printed PR 3's table line for line (234 checks; 15.9 s, 350 MiB). Modal runs, all ephemeral incytearc_profiling: fourrun --scripts dose_design.py --write-fixtureson 2 CPU and 4 GiB (the first stopped in R on a zero-length name, fixed; the others passed and wrote the fixtures, the last in 21 s after the review's changes), and three containers on 4 CPU and 8 GiB that ran the scripts twice each (77 s and 65 s for both scripts, then 32 s fordose_design.pyalone after a last change to its checked code; its numbers above are from that run).The helper
docs/changed_since_run.pyalso lists 30 scripts. Master lists 29 of them too, whose checked code changed before this PR: activity.py, combined_p_values_and_replication.py, composition_tests.py, cube_collection.py, deseq2_tests.py, design.py, differential_expression.py, differential_expression_options.py, dream.py, empirical_bayes.py, gene_set_files_and_ora.py, gene_set_tests.py, gsea.py, interferon_stimulated_pbmcs.py, kenward_roger_and_f_tests.py, linear_models.py, meta_analysis.py, mixed_models.py, multiple_testing.py, negative_binomial_fits.py, pathway_scores.py, pca_and_dot_plots.py, plotting.py, quasi_likelihood.py, rotation_gene_set_tests.py, sample_quality_weights.py, sumrank.py, variance_partitioning.py and voom.py. The 30th, simulate_dose_series.py, checks the reading of dose columns andDoseAxis, which this PR extends;tests/test_simulate_dose_series.pypasses against its fixtures unchanged. This PR changes none of their results. Master also lists dose_axes.py, whose checked code #79 moved intocytepool/_doses.py; this PR's run of it is current.Tests
tests/test_dose_designs.py(40 tests) checks the report against the fixtures of the validation script (scanpy's sums and pandas' tables, R's aliasing, ETAP's minimums in R, GSE208161), against rules derived by hand (statuses of small designs, replicates counted after filtering and only in groups that can be compared, the units a random intercept names, batch confounding on a dozen lane layouts with a sample alone in its lane among them, a random intercept of the batch over one or two lanes of the vehicle, other terms that absorb dose groups, a unit nested one to one in the dose groups and its pooled cube, ETAP's steps on both sides of the tolerance, a level ETAP excludes), againstlimma_voom's own choice of samples, and against invariants (sample order, design rows in another order than the cube, dose units, labels). Every refusal is tested with its message, each column of the axis by the nameDoseAxis.design_columnsgives it, and the chapter's example runs, with the fit it leads to, the printout it shows and the values it states. 51 deliberate bugs were caught, 24 in the build and 27 after the review, among them one pivoted QR in place of the column-last rule, a lone single-sample batch given a column of its own, replicates counted as samples under a random intercept, a random intercept of the batch excusing every confounded group, the design's rows left out of cube order, the vehicle counted in the replication rule, animals counted after filtering, and an excluded level counted as a deviation; each rule's mutation is caught by a test derived by hand, not only by the fixtures. Coverage ofcytepool/dose_designs.py,cytepool/_doses.pyandcytepool/dose_axes.pyis 100% of statements and branches withNUMBA_DISABLE_JIT=1.tests/test_dose_axes.pyandtests/test_simulate_dose_series.pypass unchanged but for the chapter they read.Decisions
doses, while residual df alone would call 13 of GSE208161's clusters inferential on the placebo participant's change from baselineminimumslists the levels ETAP excludes, for PR 14dose_response; stripping by name would miss columns computed from the dose under other namesqr()rulenonEstimable. A batch unbalanced across the doses belongs in the design, as the documentation saysDesign.check's message, which advises dropping the random effectn_dose_groupscounts the dose groups that keep a sample, andn_dose_groups_estimablethe treated groups the design can compare with the vehiclen_dose_groupsfor that number; PR 10's spare groups need the second count, with the vehicledose_designs.py, as functions on plain inputs in a section of their owncytepool/_doses.py, shared byDoseAxis, the check and the simulator of #79tests/test_dose_axes.pypinsDesign.check's numeric-codes heuristic unchangedTrimmed from the plan, with where it goes: spare dose groups per model set (PR 10, whose fits define the models; the chapter gives the arithmetic);
rules="ntp_2018"(PR 14); ETAP's study conditions, such as species, route and duration, which samples do not show (PR 16's records); ETAP's sequencing quality metrics, which are bulk TempO-Seq metrics (not applied; the cell type's filter stands in); designs with a dose measured per sample (PR 21); a provenance record on the report (PR 16). No guide holds a dose series before PR 7, so the worked example lives in the chapter, and PR 7's guide should open with the check.