Lemon: An MPI parallel I/O library for data encapsulation using LIME
Abstract
We introduce Lemon, an MPI parallel I/O library that provides efficient parallel I/O of both binary and metadata on massively parallel architectures. Motivated by the demands of the Lattice Quantum Chromodynamics community, the data is stored in the SciDAC Lattice QCD Interchange Message Encapsulation format. This format allows for storing large blocks of binary data and corresponding metadata in the same file. Even if designed for LQCD needs, this format might be useful for any application with this type of data profile. The design, implementation and application of Lemon are described. We conclude with presenting the excellent scaling properties of Lemon on state-of-the-art high performance computers. Program summaryProgram title: Lemon Catalogue identifier: AELP_v1_0 Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AELP_v1_0.html Program obtainable from: CPC Program Library, Queen's University, Belfast, N. Ireland Licensing provisions: GNU General Public License version 3 No. of lines in distributed program, including test data, etc.: 32 860 No. of bytes in distributed program, including test data, etc.: 223 762 Distribution format: tar.gz Programming language: MPI and C Computer: Any which supports MPI I/O Operating system: Any Has the code been vectorised or parallelised?: Yes. Includes MPI directives. RAM: Depending on input used Classification: 11.5 External routines: MPI Nature of problem: Distributed file I/O with metadata Solution method: MPI parallel I/O based implementation of LIME format Running time: Varies depending on file and architecture size, in the order of seconds
- Publication:
-
Computer Physics Communications
- Pub Date:
- June 2012
- DOI:
- 10.1016/j.cpc.2012.01.016
- arXiv:
- arXiv:1106.4177
- Bibcode:
- 2012CoPhC.183.1321D
- Keywords:
-
- High Energy Physics - Lattice;
- Physics - Computational Physics
- E-Print:
- doi:10.1016/j.cpc.2012.01.016