Repository navigation
task: agent ready prep #280
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
jharlow-intel
wants to merge
7
commits into
main
Choose a base branch
from
task/agent-ready
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+436
−260
Open
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
823e526
task: agent ready prep
jharlow-intel 9cefba6
task: clean up CONTRIBUTING.md
jharlow-intel 3f8573f
task: address review on agent docs
jharlow-intel 67f65c0
Merge remote-tracking branch 'origin/main' into task/agent-ready
jharlow-intel 0d3f956
task: address second review round on agent docs
jharlow-intel ec65659
Merge branch 'main' into task/agent-ready
jharlow-intel 27d9109
Merge branch 'main' into task/agent-ready
jharlow-intel File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| # Keep shell scripts LF on checkout so shellcheck and conda build scripts work | ||
| # the same on Windows and WSL checkouts with core.autocrlf enabled. | ||
| *.sh text eol=lf |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| name: Bug report | ||
| description: Report incorrect results, a crash, or a build failure | ||
| labels: ["bug"] | ||
| body: | ||
| - type: markdown | ||
| attributes: | ||
| value: | | ||
|
jharlow-intel marked this conversation as resolved.
|
||
| For security vulnerabilities, do not open an issue — follow [SECURITY.md](https://github.com/IntelPython/mkl_umath/blob/main/SECURITY.md). | ||
|
|
||
| - type: textarea | ||
| id: description | ||
| attributes: | ||
| label: Description | ||
| description: What happened, and what did you expect instead? | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: reproducer | ||
| attributes: | ||
| label: Reproducer | ||
| description: A minimal, self-contained snippet. Include the ufunc, input values, and dtype. | ||
| render: python | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: dropdown | ||
| id: install-source | ||
| attributes: | ||
| label: How was `mkl_umath` installed? | ||
| options: | ||
| - Intel conda channel (software.repos.intel.com) | ||
| - conda-forge | ||
| - pip, Intel index (software.repos.intel.com) | ||
| - pip / PyPI | ||
| - Built from source | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: versions | ||
| attributes: | ||
| label: Versions | ||
| description: | | ||
| Output of: | ||
| ``` | ||
| python -c "import mkl_umath, numpy; print(mkl_umath.__version__); print(numpy.__version__)" | ||
|
jharlow-intel marked this conversation as resolved.
|
||
| ``` | ||
| Add the MKL version (`conda list mkl` or `pip list | grep -i mkl`), your OS, and your Python version too. | ||
| render: shell | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: notes | ||
| attributes: | ||
| label: Anything else | ||
| description: | | ||
| Optional. Whether NumPy patching was active (`mkl_umath.is_patched()`), whether the result differs from stock NumPy, or a non-default `MKL_NUM_THREADS`. | ||
| validations: | ||
| required: false | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| blank_issues_enabled: true | ||
| contact_links: | ||
| - name: Security vulnerability | ||
| url: https://www.intel.com/content/www/us/en/security-center/vulnerability-handling-guidelines.html | ||
| about: Report security vulnerabilities through Intel's process, not a public issue. | ||
| - name: Question about usage | ||
| url: https://github.com/IntelPython/mkl_umath/blob/main/README.md | ||
| about: Check the README first, including the patching section. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| name: Feature request | ||
| description: Propose a new ufunc loop, dtype, or capability | ||
| labels: ["enhancement"] | ||
| body: | ||
| - type: textarea | ||
| id: problem | ||
| attributes: | ||
| label: What problem does this solve? | ||
| description: The use case, not the implementation. | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: proposal | ||
| attributes: | ||
| label: Proposal | ||
| description: | | ||
| What you would like `mkl_umath` to do. If it covers a NumPy ufunc, name it — NumPy's semantics are the contract for patched loops. | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: input | ||
| id: upstream | ||
| attributes: | ||
| label: Upstream equivalent | ||
| description: Link to the NumPy docs for the ufunc, if there is one. | ||
| validations: | ||
| required: false | ||
|
|
||
| - type: textarea | ||
| id: alternatives | ||
| attributes: | ||
| label: Alternatives considered | ||
| description: Optional. Workarounds you are using today. | ||
| validations: | ||
| required: false |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| # Description | ||
|
|
||
| <!-- What changed and why. Link any related issues. --> | ||
|
|
||
| ## Verification | ||
|
|
||
| <!-- The commands you ran, and the platform and versions you ran them on. --> | ||
|
|
||
| - Tests: <!-- e.g. `pytest mkl_umath/tests`, Python 3.12 / NumPy 2.x, Linux --> | ||
| - Lint: <!-- `pre-commit run --all-files` --> | ||
|
|
||
| ## Not verified | ||
|
|
||
| <!-- | ||
| Anything skipped or left to CI, and why. Examples: Windows, the Intel-channel | ||
| conda build, the benchmarks. Write "none" if you ran everything relevant. | ||
| --> | ||
|
|
||
| ## Checklist | ||
|
|
||
| - [ ] Results match stock NumPy, or the difference is intentional and called out above. | ||
| - [ ] Behavior changes have tests in `mkl_umath/tests/`; bug fixes have a regression test. | ||
| - [ ] `CHANGELOG.md` updated under `## [dev]` with a `[gh-NNN](https://github.com/IntelPython/mkl_umath/pull/NNN)` link, or the change isn't user-visible. | ||
|
|
||
| <!-- See CONTRIBUTING.md for the build and test workflow, and AGENTS.md for the module map. --> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,109 +1,69 @@ | ||
| # AGENTS.md | ||
| # AGENTS.md — mkl_umath | ||
|
|
||
| Entry point for agent context in this repo. | ||
|
|
||
| ## What this repository is | ||
| `mkl_umath` exposes Intel® OneMKL-powered universal function loops for NumPy, originally part of Intel® Distribution for Python* and factored out per NEP-36 (Fair Play). | ||
|
|
||
| It provides: | ||
| - `mkl_umath._ufuncs` — OneMKL-backed NumPy ufunc loops | ||
| - `mkl_umath._patch_numpy` — runtime patching interface (`patch_numpy_umath` `restore_numpy_umath`, `is_patched()`) | ||
| - Performance-optimized math operations (sin, cos, exp, log, etc.) using Intel MKL VM | ||
| ## What this project is | ||
| `mkl_umath` provides NumPy ufunc loops backed by Intel® oneMKL Vector Math, and | ||
| can patch them into NumPy at runtime. It was factored out of Intel® Distribution | ||
| for Python* per NEP-36 (Fair Play). | ||
|
|
||
| ## Key components | ||
| - **Python interface:** `mkl_umath/__init__.py`, `_init_helper.py` | ||
| - **Core C implementation:** `mkl_umath/src/` (ufuncsmodule.c, mkl_umath_loops.c.src) | ||
| - **Cython patch layer:** `mkl_umath/src/_patch_numpy.pyx` | ||
| - **Code generation:** `generate_umath.py`, `generate_umath_doc.py` | ||
| - **Build system:** meson-python + Cython | ||
|
|
||
| ## Build dependencies | ||
| **Required:** | ||
| - Compiler toolchain: Intel `icx` or `clang` (with Intel-only flags gated when using clang) | ||
| - Intel® oneMKL (`mkl-devel`) | ||
| - meson-python, CMake, Ninja, Cython>=3.1.0, NumPy | ||
|
|
||
| **Build against an existing `mkl` installation:** | ||
|
|
||
| Install the build dependencies via Conda: | ||
| ```bash | ||
| conda install -c https://software.repos.intel.com/python/conda \ | ||
| mkl-devel dpcpp_linux-64 "cython>=3.1.0" meson-python cmake ninja numpy | ||
| ``` | ||
| or via pip: | ||
| ```bash | ||
| pip install mkl-devel "cython>=3.1.0" meson-python cmake ninja numpy | ||
| ``` | ||
| then build: | ||
| ```bash | ||
| CC=icx pip install --no-deps --no-build-isolation . # clang is also supported in CI | ||
| ``` | ||
|
|
||
| ## CI/CD | ||
| - **Platforms:** Linux, Windows | ||
|
jharlow-intel marked this conversation as resolved.
|
||
| - **Python versions:** 3.10, 3.11, 3.12, 3.13, 3.14, 3.15 | ||
| - **Workflows:** `.github/workflows/` | ||
| - `conda-package.yml` — main conda build/test pipeline | ||
| - `conda-package-cf.yml` — conda-forge-oriented build/test pipeline | ||
| - `build_pip.yml` — validates pip build with pre-release NumPy | ||
| - `build-with-clang.yml` — Intel clang compatibility check | ||
| - `build-with-standard-clang.yml` — standard clang compatibility check | ||
| - `openssf-scorecard.yml` — security scorecard | ||
|
|
||
| ## Distribution | ||
| - **Conda:** `https://software.repos.intel.com/python/conda` | ||
| - **PyPI:** `https://software.repos.intel.com/python/pypi` | ||
| - Requires Intel-optimized NumPy from Intel channels | ||
|
|
||
| ## Usage | ||
| ```python | ||
| import mkl_umath | ||
| mkl_umath.patch_numpy_umath() # Patch NumPy to use MKL loops | ||
| # ... perform NumPy operations (now accelerated) ... | ||
| mkl_umath.restore_numpy_umath() # Restore original NumPy loops | ||
| ``` | ||
|
|
||
| ## How to work in this repo | ||
| - **Performance:** Changes should maintain or improve MKL VM utilization | ||
| - **Compatibility:** Must work with upstream NumPy APIs (NEP-36 compliance) | ||
| - **Testing:** Add tests to `mkl_umath/tests/test_basic.py` | ||
| - **Build hygiene:** `meson.build` is the source of truth for build config — verify Linux + Windows | ||
| - **Docs:** Update docstrings via `ufunc_docstrings_numpy2.py` | ||
|
|
||
| ## Code structure | ||
| - **Generated code:** `*.src` files are templates (conv_template.py processes them) | ||
| - **Precision flags:** fp:precise, fimf-precision=high, fprotect-parens (non-negotiable) | ||
| - **Security:** Stack protection, FORTIFY_SOURCE, NX/DEP enforced in `meson.build` | ||
| - **Build options:** `opt_report` and `mkl_threading` are exposed via `meson.options` | ||
|
|
||
|
|
||
| ## Common pitfalls | ||
| - **NumPy source:** Requires Intel-optimized NumPy from Intel channel (`software.repos.intel.com/python/conda`). PyPI NumPy may cause runtime failures or incorrect results. | ||
| - **Precision flags:** `fp:precise`, `fimf-precision=high` enforce IEEE 754 compliance. Removing them breaks numerical correctness in scientific computing. | ||
| - **Patching order:** If using multiple Intel patches (e.g., `mkl_random` + `mkl_umath`), apply `mkl_umath` last. Verify with `is_patched()` after each. | ||
| - **Compiler/toolchain:** `icx` and `clang` are both supported; when using clang, keep Intel-only flags behind compiler guards. | ||
| - **Build validation:** | ||
| - After setup: `which ${CC:-icx}` → should resolve to the intended compiler toolchain | ||
| - Check: `python -c "import numpy; print(numpy.__version__)"` → confirm NumPy is available | ||
|
|
||
| ## Notes | ||
| - `_vendored/` contains vendored NumPy code generation utilities | ||
| - Version in `mkl_umath/_version.py` (read dynamically by `meson.build`) | ||
| - Patching is runtime-only; no NumPy source modification | ||
| - **Package and public API:** `mkl_umath/`, `mkl_umath/__init__.py` | ||
| - **Ufunc extension:** `mkl_umath/src/ufuncsmodule.c`, plus `__umath_generated.c` | ||
| from `mkl_umath/generate_umath.py` | ||
| - **Loop templates:** `mkl_umath/src/mkl_umath_loops.{c,h}.src` | ||
| - **Patching:** `mkl_umath/src/_patch_numpy.pyx`; persistent and one-shot | ||
| patching in `mkl_umath/patch.py`, `mkl_umath/with_patch.py`, | ||
| `mkl_umath/_patch_startup.py`, and the `mkl_umath/__main__.py` CLI | ||
| - **Tests:** `mkl_umath/tests/` | ||
| - **Vendored helpers:** `_vendored/` | ||
| - **Packaging:** `conda-recipe/`, `conda-recipe-cf/` | ||
| - **Benchmarks:** `benchmarks/` | ||
|
|
||
| ## Build/runtime basics | ||
| - Build system: `pyproject.toml` + `meson.build`, with options in `meson.options` | ||
| - Build deps: `mkl-devel`, `numpy`, `meson-python`, `cmake`, `ninja`, `cython`, | ||
| and a C compiler (CI uses `icx` and `clang`) | ||
| - Runtime deps: `numpy`; the conda recipes add the MKL and compiler runtimes | ||
| - Platforms: Linux and Windows; `.github/workflows/` is canonical for the matrix | ||
| - Setup, checks, and style: `CONTRIBUTING.md` | ||
| - Single test: `pytest mkl_umath/tests/<file>::<test>`, from a checkout with the | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It should be written as |
||
| editable install in `CONTRIBUTING.md` | ||
| - Single-file lint: `pre-commit run --files <path>` | ||
|
|
||
| ## Development guardrails | ||
| - Preserve NumPy ufunc behavior; patched loops stand in for NumPy's own. | ||
| - Edit the `*.src` templates and `generate_umath.py`, not generated C. | ||
| - Keep the floating-point precision flags in `meson.build` and, for Windows, in | ||
| both recipes' `conda_build_config.yaml`; Intel-only flags stay behind the | ||
| compiler checks in `meson.build`. | ||
| - `meson.build` changes must keep both the GCC-syntax (Linux) and MSVC-syntax | ||
| (Windows) branches working; CI covers both. | ||
| - Preserve the patching API (`patch_numpy_umath()`, `restore_numpy_umath()`, | ||
| `is_patched()`, and the `mkl_umath()` context manager) unless asked to change | ||
| it. The deprecated `use_in_numpy()` and `restore()` aliases stay until a | ||
| release removes them. | ||
| - Keep patching reversible, with `is_patched()` reporting the truth. | ||
| - Keep both extensions free-threading compatible. | ||
| - Pair behavior changes with tests and keep diffs minimal. | ||
| - Avoid hardcoding mutable versions/matrices/channels in docs. | ||
|
|
||
| ## Where truth lives | ||
| - Build/config: `pyproject.toml`, `meson.build`, `meson.options` | ||
| - Dependencies: `pyproject.toml`, `conda-recipe*/meta.yaml` | ||
| - CI/workflows: `.github/workflows/*.yml` | ||
| - Public API: `mkl_umath/__init__.py`, `mkl_umath/src/_patch_numpy.pyx` | ||
| - Tests: `mkl_umath/tests/` | ||
|
|
||
| For behavior policy, see `.github/copilot-instructions.md`. | ||
|
|
||
| ## Directory map | ||
| Below directories have local `AGENTS.md` for deeper context: | ||
| - `.github/AGENTS.md` — CI/CD workflows and automation | ||
| - `mkl_umath/AGENTS.md` — Python API and code generation | ||
| - `mkl_umath/src/AGENTS.md` — C/Cython implementation layer | ||
| - `mkl_umath/tests/AGENTS.md` — unit tests and validation | ||
| - `conda-recipe/AGENTS.md` — Intel channel conda packaging | ||
| - `conda-recipe-cf/AGENTS.md` — conda-forge compatible recipe | ||
| - `_vendored/AGENTS.md` — vendored NumPy utilities | ||
|
|
||
| --- | ||
|
|
||
| For broader IntelPython ecosystem context, see: | ||
| - `dpnp` (Data Parallel NumPy) | ||
| - `mkl_random` (MKL-based random number generation) | ||
| - `numba-dpex` (Numba + SYCL) | ||
| Use nearest local `AGENTS.md` when present: | ||
| - `.github/AGENTS.md` — CI workflows and automation policy | ||
| - `mkl_umath/AGENTS.md` — package modules, API, and code generation | ||
| - `mkl_umath/src/AGENTS.md` — loop templates and the two extensions | ||
| - `mkl_umath/tests/AGENTS.md` — test scope and conventions | ||
| - `conda-recipe/AGENTS.md` — Intel-channel conda packaging | ||
| - `conda-recipe-cf/AGENTS.md` — conda-forge recipe | ||
| - `_vendored/AGENTS.md` — vendored NumPy template tooling | ||
| - `benchmarks/AGENTS.md` — ASV performance suite | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.