* Boundary lazy comm v2

* fixes

* use pytato branch

* use staple_distributed_send

* add lazy option to wave-op-mpi

* ci fix

* initial actx support

* _RankBoundaryCommunicationLazy: do not convert to/from numpy, flatten

* Prototype MPIPytatoPyOpenCLArrayContext

* Drop evt.wait in distributed actx

* cross_rank_trace_pairs: set 'unique' tag per obj array component

* Drop commented-out graph vis from mpi+lazy actx

* add clone()

* only use Lazy boundary comm when running with lazy actx

* fix git branch

* remove duplicate requirement

* accept distributed+parallel actx

* wave-op-mpi: add simple timing

* fix MPISingleGridWorkBalancingPytatoArrayContex

* fix spelling

* initial lazy batched comm

* select comm class based on actx

* flake8

* use meshmode lazy actx

* test: use random tag

* undo random tag change

* use random tag again

* another tag fix

* debug tracepair

* add dbg

* more dbg

* Use structured tags for lazily-communicated array containers

* MPISingleGridWorkBalancingPytatoArrayContext: Convert symbolic to numeric comm tags

* more type annotation

* support running eager comm as well

* force SingleGridWorkBalancingPytatoArrayContext

* wave-op-mpi: Fix name of MPI+Pytato array context

* wave-op-mpi: Add missing mpi_communicator arg to DiscretizationCollection

* Distributed actx: deduplicate data wrappers before partitioning

* undo pytato branch

* add loopy req

* test with MPI

* use serial actx if parallel not available

* rework dependency chain

* fix lint

* make wave-op-mpi run without older lazy actx

* adapt to recent arraycontext changes

* use DistributedLazyArrayContext to simplify lazy detection

* document tag arg

* Simplify distributed-lazy array context usage (#218)

* add get_reasonable_array_context_class

* add get_reasonable_array_context_class

* document tag in interior_trace_pairs

* update MPI test infrastructure

* restructure tests

* Weak wave operator: add comm tag

* Generalize MPI test orchestration for lazy

* MPI test: use actx.compile

* test fixes

* fix sin()

* Drop leap from test_mpi_communication

* Add norm assert to test_mpi_communication

* restore final_t

* Introduce an eager-MPI array context with a sym->num tag map

Co-authored-by: Andreas Kloeckner <inform@tiker.net>

* Add Euler module

* Add simple vortex example

* Document Euler operator and add pulse experiment

* Test euler examples in lazy mode

* Make rk4_step importable

* Remove copypasta from example headers

* Make vortex initial condition importable

* Add Euler convergence test to test suite

* Rename: EulerContainer -> EulerField

* Clean up vortex initial condition

* Add type annotations and rename array container

* Clean up helper functions

* Add Euler module to the docs

* Force a single-value per element for elementwise reductions when applicable

* Fix RST formatting in docstring

Co-authored-by: Kaushik Kulkarni <15399010+kaushikcfd@users.noreply.github.com>

* Fix surface area computation in dt helper function

* Simplify surface area branches

* Update comments in dt utils

* Branch surface area computation based on actx broadcasting properties

* Test geometric factors using lazy actx

Co-authored-by: Kaushik Kulkarni <15399010+kaushikcfd@users.noreply.github.com>
Co-authored-by: Andreas Klöckner <inform@tiker.net>

* Add overintegration option for wave-op-mpi

* Add OI+lazy to examples CI testing

* Draft: Use P^0 discr on affine groups even if quad grid requested

* Refactor geoderiv connection/interpolation steps

* Fix quadrature interpolation

* Add pytato pytest array context factory

* Update metric unit tests

* Clarifying comment

Co-authored-by: Andreas Klöckner <inform@tiker.net>

* Add nonaffine option for wave op example

* Run lazy + nonaffine + overintegration in examples CI testing

* Add quadrature case for metric unit tests

* Add optional import for debugging outside of pytest

* Add docstrings for _use_geoderiv_connection

Co-authored-by: Andreas Kloeckner <inform@tiker.net>

Alexandru Fikl authored 3 years ago and Andreas Klöckner committed 3 years ago

gmsh generation was failing in v4.9 and couldn't find a setup
that worked in v4.8 and v4.9. Possible bug?

* wave-op-mpi: add simple timing

* fix

* Support array containers in core grudge DG ops

* Convert wave example to use array containers

* Update projection routine to support array containers

* Convert mpi communication

* Use DOFArray-based traversal routines

* Containerize distributed (eager) MPI communication

* Use array container in wave-op-mpi example

* Clean up wave-op-mpi demo

* Use non-recursive array traversal

* Use map_array_container in trace pair exchange

* Remove unused imports

* Remove sketchy assert; shape differs when evaluating fluxes vs storing state

* Refactor containerization routines

* Fix suppressed logging error

* Fix forward flattening map

* Fix type check in projection routine

* dofdesc-itize descriptors first in projection

* Make flattening methods private and provide short docs

* Clean up flatten/unflatten functions and document

* Carefully track ordering of data in flattened array

* Use container flatten/unflatten functions from arraycontext

* Containerize elementwise reductions and add unit tests

* Containerize nodal reductions and add unit tests

* Update documentation to reflect container support

* Catch scalars in op.norm

* Remove flat_norm and use op.norm instead

* Separate initialization and completion of send/recv

* Fix documentation in operator module

Co-authored-by: Andreas Klöckner <inform@tiker.net>

* Rename boundary communicator data/attributes

* Comment on nonblocking MPI communication process in communicator class

* Clean up comments related to rank-communication for trace pairs

* Clean up/clarify documentation related to array containers

* Update reduction unit tests

* Documentation updates

Co-authored-by: Andreas Klöckner <inform@tiker.net>

* Fix comment formatting

* Containerize grad/div functions

* Add checks for empty arrays in grad/div operators

* Fix gradient op for vector-valued components

* Correct div helper for handling empty and nested obj arrays

* Clear up documentation and add clarifying notes

* More shared code in array container support (#191)

* Refactor grad/weak grad to use the same helper, simplify div empty handling

* Factor out some redundant code in elementwise reductions

Co-authored-by: Andreas Klöckner <inform@tiker.net>

* reimplement norm() without recursion

* disallow norm of a scalar; fix tests

Return device scalars from `nodal_sum`/`nodal_min`/`nodal_max`

not needed anymore after pyopencl update