Message passing in computer clusters
Recently, the use of computer clusters with more than one thousand nodes has been spreading. As the number of nodes in a cluster increases, the rapid growth in the complexity of the communication subsystem makes message passing delays over the
Specific tools may be used to simulate, visualize and understand the performance of message passing on computer clusters. Before a large computer cluster is assembled, a
Messages and computations
Approaches to message passing
Historically, the two typical approaches to communication between cluster nodes have been PVM, the Parallel Virtual Machine and MPI, the Message Passing Interface.[6] However, MPI has now emerged as the de facto standard for message passing on computer clusters.[7]
PVM predates MPI and was developed at the
Unlike PVM, which has a concrete implementation, MPI is a specification rather than a specific set of libraries. The specification emerged in the early 1990 out of discussions between 40 organizations, the initial effort having been supported by
Testing, evaluation and optimization
Computer clusters use a number of strategies for dealing with the distribution of processing over multiple nodes and the resulting communication overhead. Some computer clusters such as
One approach to reducing communication overhead is the use of local neighborhoods (also called locales) for specific tasks. Here computational tasks are assigned to specific "neighborhoods" in the cluster, to increase efficiency by using processors which are closer to each other.[3] However, given that in many cases the actual topology of the computer cluster nodes and their interconnections may not be known to application developers, attempting to fine tune performance at the application program level is quite difficult.[3]
Given that MPI has now emerged as the de facto standard on computer clusters, the increase in the number of cluster nodes has resulted in continued research to improve the efficiency and scalability of MPI libraries. These efforts have included research to reduce the memory footprint of MPI libraries.[7]
From the earliest days MPI provided facilities for performance profiling via the PMPI "profiling system".[12] The use of the PMIPI- prefix allows for the observation of the entry and exit points for messages. However, given the high level nature of this profile, this type of information only provides a glimpse at the real behavior of the communication system. The need for more information resulted in the development of the MPI-Peruse system. Peruse provides a more detailed profile by enabling applications to gain access to state-changes within the MPI-library. This is achieved by registering callbacks with Peruse, and then invoking them as triggers as message events take place.[13] Peruse can work with the PARAVER visualization system. PARAVER has two components, a trace component and a visual component for analyze the traces, the statistics related to specific events, etc. [14] PARAVER may use trace formats from other systems, or perform its own tracing. It operates at the task level, thread level, and in a hybrid format. Traces often include so much information that they are often overwhelming. Thus PARAVER summarizes them to allow users to visualize and analyze them.[13][14][15]
Performance analysis
When a large scale, often supercomputer level, parallel system is being developed, it is essential to be able to experiment with multiple configurations and simulate performance. There are a number of approaches to modeling message passing efficiency in this scenario, ranging from analytical models to trace-based simulation and some approaches rely on the use of test environments based on "artificial communications" to perform synthetic tests of message passing performance.[3] Systems such as BIGSIM provide these facilities by allowing the simulation of performance on various node topologies, message passing and scheduling strategies.[4]
Analytical approaches
At the analytical level, it is necessary to model the communication time T in term of a set of subcomponents such as the startup
Xu and Hwang generalized Hockney's model to include the number of processors, so that both the latency and the asymptotic bandwidth are functions of the number of processors.[16][17] Gunawan and Cai then generalized this further by introducing cache size, and separated the messages based on their sizes, obtaining two separate models, one for messages below cache size, and one for those above.[16]
Performance simulation
Specific tools may be used to simulate and understand the performance of message passing on computer clusters. For instance, CLUSTERSIM uses a Java-based visual environment for
Other simulation tools include MPI-sim and BIGSIM.[19] MPI-Sim is an execution-driven simulator that requires C or C++ programs to operate.[18][19] ClusterSim, on the other hand uses a hybrid higher-level modeling system independent of the programming language used for program execution.[18]
Unlike MPI-Sim, BIGSIM is a trace-driven system that simulates based on the logs of executions saved in files by a separate emulator program.[5][19] BIGSIM includes an emulator, and a simulator. The emulator executes applications on a small number of nodes and stores the results, so the simulator can use them and simulate activities on a much larger number of nodes.[5] The emulator stores information of sequential execution blocks (SEBs) for multiple processors in log files, with each SEB recording the messages sent, their sources and destinations, dependencies, timings, etc. The simulator reads the log files and simulates them, and may star additional messages which are then also stored as SEBs.[4][5] The simulator can thus provide a view of the performance of very large applications, based on the execution traces provided by the emulator on a much smaller number of nodes, before the entire machine is available, or configured.[5]
See also
- High-availability cluster
- Performance simulation
- Supercomputing
References
- ^ ISBN 0123747503 page 641 [1]
- ISBN 0262692759MIT Press pages 7–9
- ^ ISBN 3642244483pages 160–162
- ^ ISBN 1584889098pages 435–435
- ^ ISBN 3642195946pages 202–203
- ^ a b c Distributed services with OpenAFS: for enterprise and education by Franco Milicchio, Wolfgang Alexander Gehrke 2007, pp. 339-341
- ^ ISBN 3642037690page 231
- ^ ISBN 8120334280pages 109–112
- .
- ^ The TianHe-1A Supercomputer: Its Hardware and Software by Xue-Jun Yang, Xiang-Ke Liao, et al in the Journal of Computer Science and Technology, Volume 26, Number 3, May 2011, pages 344–351 "Archived copy". Archived from the original on 2011-06-21. Retrieved 2012-02-08.
{{cite web}}
: CS1 maint: archived copy as title (link) - ^ U.S. says China building 'entirely indigenous' supercomputer, by Patrick Thibodeau Computerworld, November 4, 2010 [2]
- ^ What is PMPI?
- ^ ISBN 354039110Xpage 347
- ^ a b PARAVER: A Tool to Visualize and Analyze Parallel Code by Vincent Pillet et al, Proceedings of the conference on Transputer and Occam Developments, 1995, pages 17–31
- ISBN 3540401970page 183
- ^ ISBN 3540338098pages 299–307
- ISBN 3540644431page 935
- ^ ISBN 0387240489pages 59–63
- ^ ISBN 3642233236page 16