EUVIMEDEUVIMED entdecken
Crossref

mpi4py.futures: MPI-Based Asynchronous Task Execution for Python

Marcin Rogowski, Samar Aseeri, David Keyes, Lisandro Dalcin

IEEE Transactions on Parallel and Distributed Systems · 01.02.2023 · 34 (2) · 611-622

DOI: 10.1109/tpds.2022.3225481 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Parallel and Distributed Systems, erschienen am 01.02.2023. Verfasst von Marcin Rogowski und 3 weiteren Autor:innen. Die Arbeit wurde bisher 47-mal zitiert (Crossref) und nennt 36 Literaturangaben. Gefördert von: King Abdullah University of Science and Technology.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We present mpi4py.futures, a lightweight, asynchronous task execution framework targeting the Python programming language and using the Message Passing Interface (MPI) for interprocess communication. mpi4py.futures follows the interface of the concurrent.futures package from the Python standard library and can be used as its drop-in replacement, while allowing applications to scale over multiple compute nodes. We discuss the design, implementation, and feature set of mpi4py.futures and compare its performance to other solutions on both shared and distributed memory architectures. On a shared-memory system, we show mpi4py.futures to consistently outperform Python's concurrent.futures with speedup ratios between 1.4X and 3.7X in throughput (tasks per second) and between 1.9X and 2.9X in bandwidth. On a Cray XC40 system, we compare mpi4py.futures to Dask – a well-known Python parallel computing package. Although we note more varied results, we show mpi4py.futures to outperform Dask in most scenarios.

Förderung

  • King Abdullah University of Science and Technology

Zitieren

APA

Rogowski, M., Aseeri, S., Keyes, D., & Dalcin, L. (2023). mpi4py.futures: MPI-Based Asynchronous Task Execution for Python. IEEE Transactions on Parallel and Distributed Systems, 34(2), 611-622. https://doi.org/10.1109/tpds.2022.3225481

Vancouver

Rogowski M, Aseeri S, Keyes D, Dalcin L. mpi4py.futures: MPI-Based Asynchronous Task Execution for Python. IEEE Transactions on Parallel and Distributed Systems. 2023;34(2):611-622. doi:10.1109/tpds.2022.3225481

BibTeX
@article{Rogowski2023,
  author = {Rogowski, Marcin and Aseeri, Samar and Keyes, David and Dalcin, Lisandro},
  title = {mpi4py.futures: MPI-Based Asynchronous Task Execution for Python},
  journal = {IEEE Transactions on Parallel and Distributed Systems},
  year = {2023},
  volume = {34},
  number = {2},
  pages = {611--622},
  doi = {10.1109/tpds.2022.3225481}
}
Kostenlos starten →