Repository navigation
[SYSTEMDS-3971] Add Scuro evaluation layer tests - #2639
Open
shieru1214 wants to merge 1 commit into
Open
shieru1214 wants to merge 1 commit into
shieru1214 wants to merge 1 commit into
Conversation
Assisted-by: AI
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change adds tests for the Scuro evaluation layer in two new files.
test_split_aware_evaluator.pychecks split-aware evaluation, andtest_task.pychecks cross-validation inTask.Split-Aware Evaluation
EvaluationSplit: The test checks three invalid splits: repeated fit IDs, repeated evaluation IDs, and an ID shared by both parts. Each case should raise aValueError.fit: A supervisedTfIdfsubclass records the inputs to eachfitcall. The test checks that the instance IDs and labels come only from the fit part of that fold.TfIdf → MLPAveragingDAG and checks that the node cache stays empty when fitted results are not shared by default. A fitted result depends on the current fold, so it should not be reused across folds by default._leaf_input: The tests check that a leaf selects the right input by modality ID and representation index, including when the modality ID is a string. An unknown modality or an index out of range should raise an error._align_partition_shapes: The tests check that shorter partitions are padded with zeros to the longest length, while the longest partition stays unchanged. Data should also stay unchanged when the shapes are already the same or the ranks are different.materialize: The test checks that a modality with neither data nor a loader raises an error.Cross-Validation in
Task