Global Particle-in-Cell Simulation of Fusion Plasmas

GTC using ADIOS on Summit, can reach near-peak performance, 2.2 TB.
GTC using ADIOS on Summit, can reach near-peak performance, 2.2 TB.


  • Energetic particle (EP) confinement is a key physics issue for burning plasma experiment ITER, the crucial next step in the quest for clean and abundant energy, since ignition relies on self-heating by energetic fusion products (α-particles)
  • By enabling GTC with the ADIOS framework, we can finally write the majority of the physics data with minimal impact on the code performance on the Summit HPC resource at the OLCF

Significance and Impact

  • GTC can generate data, over 100 TB of physics data every hour
  • GTC has been equipped with ADIOS to allow all of the relevant physics information to be written to the Summit GPFS file system in less than 3% of the total runtime
  • New data analytics is being written for GTC to work in both post-processing and in situ workflows

Research Details

  • A new “engine” inside of ADIOS was developed to allow for extreme performance for Particle In Cell Codes  I/O
  • A new file format, BP4, was created to handle increasingly large metadata, enabling GTC to write all of it’s physics with minimal impact on the performance on Summit


The GTC CAAR code uses POSIX I/O natively that imposes significant challenges in obtaining highly performance I/O and overall management of data. As part of the RAPIDS SCIDAC project, ADIOS was adapted to manage data at extreme scales. Experiments were performed on Summit, and the effect of tuning I/O parameters in ADIOS was studied. Writing large-sized checkpointing data from GTC to the GPFS parallel file system shows peak I/O bandwidth of 2.2 TB/s and speeds of over 1 TB/s consistently. Using ADIOS also shows a 50x improvement in writing snapshot data. Furthermore, experiments were conducted to study the impact of BP4, the next generation metadata format in ADIOS. Results show that in comparison with the current BP3 format, BP4 shows a constant metadata overhead for increasing number of simulation timesteps. Our experiments also show the variability of I/O bandwidth on Summit, which leads to the necessity of developing I/O frameworks with dynamic tuning capabilities.