1.3 Recent developments in MPI
The MPI user and developer community is currently going through dynamic changes. More and more MPI implementations are meanwhile available, each focusing on different aspects. The manufacturers of high-performance computing systems focus on improving the performance of MPI-1 functions as well as on implementing more and more of MPI-2. The two most widespread, freely available MPI libraries (MPICH, LAM/MPI) are both coming closer to support the full MPI-2 specification.
Many MPI implementations are dealing with aspects of distributed and Grid computing, focusing on the given hierarchies, security aspects as well as user-friendliness. Examples are PACX- MPI, MPICH-G2, Stampi, MetaMPICH or GridMPI.
Another set of projects is dealing with various aspects of fault-tolerance, e.g. MPI/FT, MPI- FT, MPICH-V or LA-MPI. Many of these projects are providing a checkpoint-restart interface, allowing an application to restart from the last consistent checkpoint in case an error occurs.
1.4 The need for extending the MPI specification
Summarizing the findings of the introduction so far, there is a mismatch between the capabilities of current high performance computing systems and the mainly used parallel programming paradigm. While the machines are getting more and more robust (hardware, network, operating systems, file systems) the MPI specification does not leave room for fully exploiting the capabilities of the current architectures. Checkpoint/restart, the only currently available option, does have its performance and conceptual limitations, if machines with tens of thousand of processors are considered. In fact, one of the main reasons for many research groups to stick still to the PVM communication library instead of switching to MPI is the capability of the first one to handle process failures.
If today’s and tomorrows high performance computing resources shall be used as a means to perform single, large scale simulations and not solemnly as a platform for high throughput computing, extending the main communication library of HPC systems, respectively the main programming paradigm, to deal with aspects of fault-tolerance is inevitable.
Therefore, we present in this document the results of work conducted during the last four years, which produced:
A specification called ‘Proposal for Extensions to the Message-Passing Interface for Process Fault-Tolerance’
An implementation of this specification
Numerous application scenarios showing the feasibility of the specification for scientific, high performance computing.
The rest of the document is organized as follows: Section 2 presents a summary of the specification for a Fault-Tolerant MPI (FT-MPI). The complete specification is attached as Appendix A. In section 3 we present some of the architectural decisions on implementing the FT- MPI specification. Section 4 presents a wide variety of application scenarios using the FT-MPI specification, ranging from dense linear algebra examples to parallel equation solvers and a master-slave code. Finally, section 5 summarizes the paper and presents the ongoing work.