Repository navigation
WIP, TST: yield array API infra from sklearn - #1064
tylerjereddy wants to merge 1 commit into
Conversation
* Related to data-apisgh-17. I believe recent "virtual discussions" have taken a different turn than some of the views expressed there, such that consideration may be given to "upstreaming" common array API testing/support utilities that are used in sklearn, but currently private there. At least, that was my impression for recent array API standard meetings. * This branch starts adding a few of the `yield..` array API utilities that are currently private in `sklearn`, but that our team uses as if they were public in our machine learning/estimator research work. I verified that substitution of `yield_namespace_device_dtype_combinations` from this branch in place of the `sklearn` private version produces the same testsuite results in one of our test branches (lanl/GFDL#129), whether `SCIPY_ARRAY_API` is set or not. * The pixi `lint` checks passed except for some apparently unrelated `pyrefly` complaints. `PYTHONPATH=src pixi run tests` did pass locally (why do I need to set my path manually like that?).
| str | ||
| The name of the Array API namespace. | ||
| """ | ||
| _NUMPY_NAMESPACE_NAMES = {"numpy", "sklearn.externals.array_api_compat.numpy"} |
There was a problem hiding this comment.
can we safely do this outside of sklearn?
| """Yield supported namespace, device_name, dtype_name tuples for testing. | ||
|
|
||
| Use this to test that an estimator works with all combinations. | ||
| Pass the yielded values to `_array_api_for_tests` which returns (xp, device) |
There was a problem hiding this comment.
This is sklearn-specific nomenclature. There may be other such cases--I've not yet cleaned that up until there's a more firm decision on potentially moving forward...
|
This thrust is related to work being done by @eiviani-lanl and @sdtemple on the LANL side. |
|
sorry, I really ought to fix that, please try setting |
okay, the problem is that since Python 3.15 editable installs use https://docs.python.org/3/library/site.html#site-start-files, which causes older Python versions to be unable to find the installation at runtime. The solution is to avoid building array-api-extra with 3.15 when we want to use it with an older Python. I'll send a PR |
Related to ENH: add testing support utilities #17. I believe recent "virtual discussions" have taken a different turn than some of the views expressed there, such that consideration may be given to "upstreaming" common array API testing/support utilities that are used in sklearn, but currently private there. At least, that was my impression for recent array API standard meetings.
This branch starts adding a few of the
yield..array API utilities that are currently private insklearn, but that our team uses as if they were public in our machine learning/estimator research work. I verified that substitution ofyield_namespace_device_dtype_combinationsfrom this branch in place of thesklearnprivate version produces the same testsuite results in one of our test branches (ENH: Array API forGFDLRegressorlanl/GFDL#129), whetherSCIPY_ARRAY_APIis set or not.The pixi
lintchecks passed except for some apparently unrelatedpyreflycomplaints.PYTHONPATH=src pixi run testsdid pass locally (why do I need to set my path manually like that?).A brief broader context is that we're working on adding some estimators to sklearn (scikit-learn/scikit-learn#33215), but even if those do eventually get in, we still have "research" versions of them and other related estimators we'd like to maintain separately for experiments that support array API at https://github.com/lanl/GFDL. So, that's where the desire to make this kind of infrastructure "non-private" is coming from.
TODOs here might be:
sklearn/utils/tests/test_array_api.py? Probably, if we decide to move forward.sklearnspecific? (currently, this infra has only been slightly modified from copy-pasting to appease the linter, etc.)AI Generation Disclosure
So far I've only asked Claude Opus
5.5how to run the tests here, which is where the confusingPYTHONPATH=src pixi run testsincantation above comes from. We should probably get an LLM to check things more carefully if we decide to move forward. For now, the rest was done by hand.