Skip to content

mpi: Fix halo=0 SubDomain mask at rank edges - #3102

Open
EdCaunt wants to merge 3 commits into
mainfrom
fix-halo-zero-mpi-mask
Open

EdCaunt wants to merge 3 commits into
mainfrom
fix-halo-zero-mpi-mask

Conversation

@EdCaunt

@EdCaunt EdCaunt commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Makes the indicator for a subdomain use the global bounds rather than the local one

@EdCaunt
EdCaunt requested a review from mloubout November 3, 2026 16:30
@codecov

codecov Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 56.94444% with 31 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.45%. Comparing base (f7350b6) to head (063ab3d).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
tests/test_mpi.py 23.52% 26 Missing ⚠️
devito/types/dimension.py 82.75% 2 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3102      +/-   ##
==========================================
- Coverage   84.51%   84.45%   -0.07%     
==========================================
  Files         260      260              
  Lines       58493    58569      +76     
  Branches     4962     4968       +6     
==========================================
+ Hits        49438    49466      +28     
- Misses       8226     8270      +44     
- Partials      829      833       +4     
Flag Coverage Δ
pytest-gpu-aomp-amdgpuX 68.38% <25.80%> (-0.05%) ⬇️
pytest-gpu-gcc- 79.20% <56.94%> (-0.04%) ⬇️
pytest-gpu-icx- 79.12% <56.94%> (-0.05%) ⬇️
pytest-gpu-nvc-nvidiaX 69.09% <25.80%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread tests/test_mpi.py Outdated
subdomain = Region(grid=serial)

def functions(grid):
kwargs = {'grid': grid, 'space_order': 4, 'dtype': np.float64}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why fp64?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to have copied a pattern from test_dimension.py. Apparently you introduced that pattern 😂

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to do nothing, so removing it throughout the codebase

Comment thread devito/types/dimension.py Outdated
"""
kwargs = {'dtype': np.int32, 'is_const': True, 'nonnegative': True,
'root': self.root}
lofs = RankOffset(name=f"{self.root.name}_lofs", side=LEFT, **kwargs)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likely should be a (cached) property of self.root to avoid duplication, ...

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch

Comment thread devito/types/dimension.py Outdated
rank's local indices. Unlike `_interval`, these are not clipped to this
rank, so with domain decomposition they may lie in the halo or beyond.
"""
kwargs = {'dtype': np.int32, 'is_const': True, 'nonnegative': True,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just make an __init__ for RankOffset

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a pattern copied from where Thickness gets created, but turns out it's mostly unnecessary, so removing everywhere.

Comment thread devito/types/dimension.py
def _arg_values(self, grid=None, **kwargs):
root = self.root
if grid is not None:
root = next((d for d in grid.dimensions if d.name == root.name), root)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks weird

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lifted directly from Thickness as far as I can tell. Git blame says @FabioLuporini did it - any idea why?

Comment thread devito/types/dimension.py
if grid is not None:
root = next((d for d in grid.dimensions if d.name == root.name), root)

if grid is None or not grid.is_distributed(root):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is_distributed is always true for all grid dimensions so i think you're looking for is_decomposed

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is essentially just guarding the subsequent if block, ensuring that grid.distributor.decomposition[root] won't throw an error

@mloubout mloubout added MPI mpi-related compiler labels Nov 5, 2026
@EdCaunt
EdCaunt force-pushed the fix-halo-zero-mpi-mask branch from 8435a25 to 7249fa0 Compare November 6, 2026 13:11
@mloubout

mloubout commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Need rebase and gtg

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

compiler MPI mpi-related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants