Skip to content

mpi: Add basic2 mode - #2307

Closed
georgebisbas wants to merge 11 commits into
mainfrom
basic2mpi
Closed

mpi: Add basic2 mode#2307
georgebisbas wants to merge 11 commits into
mainfrom
basic2mpi

Conversation

@georgebisbas

Copy link
Copy Markdown
Contributor

Preallocated buffers using MPIMsg.
An MPIMsgEnriched version for send/recv, will follow

@georgebisbas georgebisbas added the MPI mpi-related label Feb 10, 2024
@georgebisbas georgebisbas self-assigned this Feb 10, 2024
@georgebisbas georgebisbas reopened this Feb 10, 2024
@georgebisbas
georgebisbas marked this pull request as draft February 10, 2024 18:14
@codecov

codecov Bot commented Feb 10, 2024

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 98.77301% with 2 lines in your changes missing coverage. Please review.

Project coverage is 87.30%. Comparing base (34dba05) to head (68631a5).

Files with missing lines Patch % Lines
devito/mpi/routines.py 98.69% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##           master    #2307    +/-   ##
========================================
  Coverage   87.29%   87.30%            
========================================
  Files         238      238            
  Lines       46063    46191   +128     
  Branches     4080     4098    +18     
========================================
+ Hits        40211    40325   +114     
- Misses       5162     5175    +13     
- Partials      690      691     +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Comment thread devito/mpi/routines.py
@georgebisbas
georgebisbas force-pushed the basic2mpi branch 4 times, most recently from ff919e2 to 895cef3 Compare February 16, 2024 18:40
@georgebisbas
georgebisbas marked this pull request as ready for review February 16, 2024 18:55
@georgebisbas
georgebisbas requested a review from EdCaunt February 16, 2024 18:55
Comment thread devito/mpi/routines.py Outdated
Comment thread devito/mpi/routines.py
Comment thread devito/mpi/routines.py Outdated
Comment thread devito/mpi/routines.py

return SendRecv('sendrecv%s' % key, iet, parameters, bufg, bufs)

def _call_sendrecv(self, name, *args, msg=None, haloid=None):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you need a haloid here ? never had to use it. What makes this mode require it?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly to the same method in OverlapHalo. Each call to sendrecv has its own message indexedpointer

Comment thread devito/mpi/routines.py
Comment thread devito/mpi/routines.py Outdated
Comment thread devito/mpi/routines.py
Comment thread devito/mpi/routines.py
mapper[(d0, side, region)] = (sizes)

i = 0
for d in f.dimensions:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not :

for i, halo in enumerate(self.halos):

like we have in the other message types?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I followed the basic style, which does not have yet a method to cleanup the redundant halos as in diag2 for example:
e.g.:

        # Only retain the halos required by the Diag scheme
        halos = sorted(i for i in hse.halos if isinstance(i.dim, tuple))

I tried this but did not manage to get it working nicely.

Comment thread tests/test_mpi.py Outdated
assert np.all(f.data_ro_domain[0, -1:, :-1] == side)

@pytest.mark.parallel(mode=[(8, 'basic'), (8, 'diag'), (8, 'overlap'),
@pytest.mark.parallel(mode=[(8, 'basic'), (8, 'basic2'), (8, 'diag'), (8, 'overlap'),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would drop basic here and below... or overlap perhaps. These tests can be quite expensive

@georgebisbas
georgebisbas force-pushed the basic2mpi branch 4 times, most recently from 5a5e826 to c266e14 Compare April 9, 2024 17:35
Comment thread devito/mpi/routines.py
assert args is not None
return int(subs_op_args(v, args))

def _allocate_buffers(self, f, shape, entry):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could use some whitespace to make it more readable

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree, added a docstring as well

Comment thread devito/mpi/routines.py Outdated

@EdCaunt EdCaunt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is documentation required somewhere to explain what the new Basic2 mode does?

Comment thread devito/mpi/routines.py Outdated
@georgebisbas
georgebisbas force-pushed the basic2mpi branch 2 times, most recently from 01ed5e2 to 28825e1 Compare December 10, 2024 12:54
@georgebisbas
georgebisbas force-pushed the basic2mpi branch 2 times, most recently from a54684a to 834a627 Compare December 19, 2024 10:29
@georgebisbas

Copy link
Copy Markdown
Contributor Author

Is documentation required somewhere to explain what the new Basic2 mode does?

you are right, will add!

@georgebisbas

Copy link
Copy Markdown
Contributor Author

@EdCaunt I just remember that I have some enhanced docs on another branch, so this can be fine for the moment!
THe branch is this:

https://github.com/devitocodes/devito/tree/bench-refresh_II

@georgebisbas

Copy link
Copy Markdown
Contributor Author

Superseded by #2995, which brings this branch up to date with main and adds a basic3 mode on top (a further-optimized basic2 that reads ranks/offsets off the message struct inside a single loop instead of unrolling one sendrecv() call per direction). Closing this one in favor of #2995.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

MPI mpi-related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants