ADIOS 2: The Adaptable Input Output (I/O) System version 2 is an open-source framework that addresses scientific data management challenges, e.g. scalable parallel I/O, as we approach the exascale era in high-performance computing (HPC). ADIOS 2 bindings are available in C++, C, Fortran, Python and can be used on supercomputers, personal computers, and cloud systems running on Linux, macOS and Windows. ADIOS 2 has out-of-the-box support for MPI and serial environments.
ADIOS 2 unified application programming interface (API) focuses on what scientific applications produce and consume in terms of n-dimensional Variables, Attributes, and Steps, while hiding the low-level details of how the data byte streams are transported as efficiently as possible from application memory to HPC networks, files, wide-area-networks, and direct memory access media.Typical use cases include file storage for checkpoint-restart and analysis, data streaming for code-coupling, and in situ analysis and visualization workflows. ADIOS 2 also provides high-level APIs that resemble native I/O libraries in Python (file) and C++ (fstream) for easy integration with their rich data analysis ecosystems. In addition, XML and YAML runtime configuration files are provided so users can fine tune available parameters to enable efficient data movements without recompiling their codes. ADIOS 2 also supports data compression via third party libraries for lossy: zfp, SZ, MGARD, and lossless: blosc, bzip2, png operations.
The ADIOS 2 development process adopts modern software engineering practices such as unit testing, continuous integration, and documentation to make the final product accessible to the scientific community. Our commitment is to release a new version every 6 months. Distributions are currently available via modern package management systems: conda, spack, homebrew (and more to come). Overall, applications using ADIOS 2 do not need to dramatically modify their source code to evaluate I/O performance trade-offs, thus reducing integration and maintenance costs in their development process. For those coming from ADIOS 1.x, ADIOS 2 maintains backwards compatibility with the binary pack (BP) file format from version 1.11 and onwards, .
ADIOS 2 is part of the United States Department of Energy (DOE) Exascale Computing Project (ECP) software technology stack for data and visualization.