- •Contents
- •List of Figures
- •List of Tables
- •Acknowledgments
- •Introduction to MPI
- •Overview and Goals
- •Background of MPI-1.0
- •Background of MPI-1.1, MPI-1.2, and MPI-2.0
- •Background of MPI-1.3 and MPI-2.1
- •Background of MPI-2.2
- •Who Should Use This Standard?
- •What Platforms Are Targets For Implementation?
- •What Is Included In The Standard?
- •What Is Not Included In The Standard?
- •Organization of this Document
- •MPI Terms and Conventions
- •Document Notation
- •Naming Conventions
- •Semantic Terms
- •Data Types
- •Opaque Objects
- •Array Arguments
- •State
- •Named Constants
- •Choice
- •Addresses
- •Language Binding
- •Deprecated Names and Functions
- •Fortran Binding Issues
- •C Binding Issues
- •C++ Binding Issues
- •Functions and Macros
- •Processes
- •Error Handling
- •Implementation Issues
- •Independence of Basic Runtime Routines
- •Interaction with Signals
- •Examples
- •Point-to-Point Communication
- •Introduction
- •Blocking Send and Receive Operations
- •Blocking Send
- •Message Data
- •Message Envelope
- •Blocking Receive
- •Return Status
- •Passing MPI_STATUS_IGNORE for Status
- •Data Type Matching and Data Conversion
- •Type Matching Rules
- •Type MPI_CHARACTER
- •Data Conversion
- •Communication Modes
- •Semantics of Point-to-Point Communication
- •Buffer Allocation and Usage
- •Nonblocking Communication
- •Communication Request Objects
- •Communication Initiation
- •Communication Completion
- •Semantics of Nonblocking Communications
- •Multiple Completions
- •Non-destructive Test of status
- •Probe and Cancel
- •Persistent Communication Requests
- •Send-Receive
- •Null Processes
- •Datatypes
- •Derived Datatypes
- •Type Constructors with Explicit Addresses
- •Datatype Constructors
- •Subarray Datatype Constructor
- •Distributed Array Datatype Constructor
- •Address and Size Functions
- •Lower-Bound and Upper-Bound Markers
- •Extent and Bounds of Datatypes
- •True Extent of Datatypes
- •Commit and Free
- •Duplicating a Datatype
- •Use of General Datatypes in Communication
- •Correct Use of Addresses
- •Decoding a Datatype
- •Examples
- •Pack and Unpack
- •Canonical MPI_PACK and MPI_UNPACK
- •Collective Communication
- •Introduction and Overview
- •Communicator Argument
- •Applying Collective Operations to Intercommunicators
- •Barrier Synchronization
- •Broadcast
- •Example using MPI_BCAST
- •Gather
- •Examples using MPI_GATHER, MPI_GATHERV
- •Scatter
- •Examples using MPI_SCATTER, MPI_SCATTERV
- •Example using MPI_ALLGATHER
- •All-to-All Scatter/Gather
- •Global Reduction Operations
- •Reduce
- •Signed Characters and Reductions
- •MINLOC and MAXLOC
- •All-Reduce
- •Process-local reduction
- •Reduce-Scatter
- •MPI_REDUCE_SCATTER_BLOCK
- •MPI_REDUCE_SCATTER
- •Scan
- •Inclusive Scan
- •Exclusive Scan
- •Example using MPI_SCAN
- •Correctness
- •Introduction
- •Features Needed to Support Libraries
- •MPI's Support for Libraries
- •Basic Concepts
- •Groups
- •Contexts
- •Intra-Communicators
- •Group Management
- •Group Accessors
- •Group Constructors
- •Group Destructors
- •Communicator Management
- •Communicator Accessors
- •Communicator Constructors
- •Communicator Destructors
- •Motivating Examples
- •Current Practice #1
- •Current Practice #2
- •(Approximate) Current Practice #3
- •Example #4
- •Library Example #1
- •Library Example #2
- •Inter-Communication
- •Inter-communicator Accessors
- •Inter-communicator Operations
- •Inter-Communication Examples
- •Caching
- •Functionality
- •Communicators
- •Windows
- •Datatypes
- •Error Class for Invalid Keyval
- •Attributes Example
- •Naming Objects
- •Formalizing the Loosely Synchronous Model
- •Basic Statements
- •Models of Execution
- •Static communicator allocation
- •Dynamic communicator allocation
- •The General case
- •Process Topologies
- •Introduction
- •Virtual Topologies
- •Embedding in MPI
- •Overview of the Functions
- •Topology Constructors
- •Cartesian Constructor
- •Cartesian Convenience Function: MPI_DIMS_CREATE
- •General (Graph) Constructor
- •Distributed (Graph) Constructor
- •Topology Inquiry Functions
- •Cartesian Shift Coordinates
- •Partitioning of Cartesian structures
- •Low-Level Topology Functions
- •An Application Example
- •MPI Environmental Management
- •Implementation Information
- •Version Inquiries
- •Environmental Inquiries
- •Tag Values
- •Host Rank
- •IO Rank
- •Clock Synchronization
- •Memory Allocation
- •Error Handling
- •Error Handlers for Communicators
- •Error Handlers for Windows
- •Error Handlers for Files
- •Freeing Errorhandlers and Retrieving Error Strings
- •Error Codes and Classes
- •Error Classes, Error Codes, and Error Handlers
- •Timers and Synchronization
- •Startup
- •Allowing User Functions at Process Termination
- •Determining Whether MPI Has Finished
- •Portable MPI Process Startup
- •The Info Object
- •Process Creation and Management
- •Introduction
- •The Dynamic Process Model
- •Starting Processes
- •The Runtime Environment
- •Process Manager Interface
- •Processes in MPI
- •Starting Processes and Establishing Communication
- •Reserved Keys
- •Spawn Example
- •Manager-worker Example, Using MPI_COMM_SPAWN.
- •Establishing Communication
- •Names, Addresses, Ports, and All That
- •Server Routines
- •Client Routines
- •Name Publishing
- •Reserved Key Values
- •Client/Server Examples
- •Ocean/Atmosphere - Relies on Name Publishing
- •Simple Client-Server Example.
- •Other Functionality
- •Universe Size
- •Singleton MPI_INIT
- •MPI_APPNUM
- •Releasing Connections
- •Another Way to Establish MPI Communication
- •One-Sided Communications
- •Introduction
- •Initialization
- •Window Creation
- •Window Attributes
- •Communication Calls
- •Examples
- •Accumulate Functions
- •Synchronization Calls
- •Fence
- •General Active Target Synchronization
- •Lock
- •Assertions
- •Examples
- •Error Handling
- •Error Handlers
- •Error Classes
- •Semantics and Correctness
- •Atomicity
- •Progress
- •Registers and Compiler Optimizations
- •External Interfaces
- •Introduction
- •Generalized Requests
- •Examples
- •Associating Information with Status
- •MPI and Threads
- •General
- •Initialization
- •Introduction
- •File Manipulation
- •Opening a File
- •Closing a File
- •Deleting a File
- •Resizing a File
- •Preallocating Space for a File
- •Querying the Size of a File
- •Querying File Parameters
- •File Info
- •Reserved File Hints
- •File Views
- •Data Access
- •Data Access Routines
- •Positioning
- •Synchronism
- •Coordination
- •Data Access Conventions
- •Data Access with Individual File Pointers
- •Data Access with Shared File Pointers
- •Noncollective Operations
- •Collective Operations
- •Seek
- •Split Collective Data Access Routines
- •File Interoperability
- •Datatypes for File Interoperability
- •Extent Callback
- •Datarep Conversion Functions
- •Matching Data Representations
- •Consistency and Semantics
- •File Consistency
- •Random Access vs. Sequential Files
- •Progress
- •Collective File Operations
- •Type Matching
- •Logical vs. Physical File Layout
- •File Size
- •Examples
- •Asynchronous I/O
- •I/O Error Handling
- •I/O Error Classes
- •Examples
- •Subarray Filetype Constructor
- •Requirements
- •Discussion
- •Logic of the Design
- •Examples
- •MPI Library Implementation
- •Systems with Weak Symbols
- •Systems Without Weak Symbols
- •Complications
- •Multiple Counting
- •Linker Oddities
- •Multiple Levels of Interception
- •Deprecated Functions
- •Deprecated since MPI-2.0
- •Deprecated since MPI-2.2
- •Language Bindings
- •Overview
- •Design
- •C++ Classes for MPI
- •Class Member Functions for MPI
- •Semantics
- •C++ Datatypes
- •Communicators
- •Exceptions
- •Mixed-Language Operability
- •Problems With Fortran Bindings for MPI
- •Problems Due to Strong Typing
- •Problems Due to Data Copying and Sequence Association
- •Special Constants
- •Fortran 90 Derived Types
- •A Problem with Register Optimization
- •Basic Fortran Support
- •Extended Fortran Support
- •The mpi Module
- •No Type Mismatch Problems for Subroutines with Choice Arguments
- •Additional Support for Fortran Numeric Intrinsic Types
- •Language Interoperability
- •Introduction
- •Assumptions
- •Initialization
- •Transfer of Handles
- •Status
- •MPI Opaque Objects
- •Datatypes
- •Callback Functions
- •Error Handlers
- •Reduce Operations
- •Addresses
- •Attributes
- •Extra State
- •Constants
- •Interlanguage Communication
- •Language Bindings Summary
- •Groups, Contexts, Communicators, and Caching Fortran Bindings
- •External Interfaces C++ Bindings
- •Change-Log
- •Bibliography
- •Examples Index
- •MPI Declarations Index
- •MPI Function Index
Bibliography
[1]V. Bala and S. Kipnis. Process groups: a mechanism for the coordination of and communication among processes in the Venus collective communication library. Technical report, IBM T. J. Watson Research Center, October 1992. Preprint. 1.2
[2]V. Bala, S. Kipnis, L. Rudolph, and Marc Snir. Designing e cient, scalable, and portable collective communication libraries. Technical report, IBM T. J. Watson Research Center, October 1992. Preprint. 1.2
[3]Purushotham V. Bangalore, Nathan E. Doss, and Anthony Skjellum. MPI++: Issues and Features. In OON-SKI '94, page in press, 1994. 6.1
[4]A. Beguelin, J. Dongarra, A. Geist, R. Manchek, and V. Sunderam. Visualization and debugging in a heterogeneous environment. IEEE Computer, 26(6):88{95, June 1993. 1.2
[5]Luc Bomans and Rolf Hempel. The Argonne/GMD macros in FORTRAN for portable parallel programming and their implementation on the Intel iPSC/2. Parallel Computing, 15:119{132, 1990. 1.2
[6]Rajesh Bordawekar, Juan Miguel del Rosario, and Alok Choudhary. Design and evaluation of primitives for parallel I/O. In Proceedings of Supercomputing '93, pages 452{461, 1993. 13.1
[7]R. Butler and E. Lusk. User's guide to the p4 programming system. Technical Report TM-ANL{92/17, Argonne National Laboratory, 1992. 1.2
[8]Ralph Butler and Ewing Lusk. Monitors, messages, and clusters: The p4 parallel programming system. Parallel Computing, 20(4):547{564, April 1994. Also Argonne National Laboratory Mathematics and Computer Science Division preprint P362-0493. 1.2
[9]Robin Calkin, Rolf Hempel, Hans-Christian Hoppe, and Peter Wypior. Portable programming with the PARMACS message-passing library. Parallel Computing, 20(4):615{632, April 1994. 1.2
[10]S. Chittor and R. J. Enbody. Performance evaluation of mesh-connected wormholerouted networks for interprocessor communication in multicomputers. In Proceedings of the 1990 Supercomputing Conference, pages 647{656, 1990. 7.1
[11]S. Chittor and R. J. Enbody. Predicting the e ect of mapping on the communication performance of large multicomputers. In Proceedings of the 1991 International Conference on Parallel Processing, vol. II (Software), pages II{1 { II{4, 1991. 7.1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
601
602 |
BIBLIOGRAPHY |
1[12] Parasoft Corporation. Express version 1.0: A communication environment for parallel
2computers, 1988. 1.2, 7.4
3
4[13] Juan Miguel del Rosario, Rajesh Bordawekar, and Alok Choudhary. Improved parallel
5I/O via a two-phase run-time access strategy. In IPPS '93 Workshop on Input/Output
6in Parallel Computer Systems, pages 56{70, 1993. Also published in Computer Archi-
7
8
tecture News 21(5), December 1993, pages 31{38. 13.1
[14] J. Dongarra, A. Geist, R. Manchek, and V. Sunderam. Integrated PVM framework
9
supports heterogeneous network computing. Computers in Physics, 7(2):166{75, April
10
1993. 1.2
11
12[15] J. J. Dongarra, R. Hempel, A. J. G. Hey, and D. W. Walker. A proposal for a user-
13level, message passing interface in a distributed memory environment. Technical Report
14TM-12231, Oak Ridge National Laboratory, February 1993. 1.2
15
16[16] Edinburgh Parallel Computing Centre, University of Edinburgh. CHIMP Concepts,
17June 1991. 1.2
18
19
20
[17]Edinburgh Parallel Computing Centre, University of Edinburgh. CHIMP Version 1.0 Interface, May 1992. 1.2
21[18] Margaret A. Ellis and Bjarne Stroustrup. The Annotated C++ Reference Manual.
22Addison Wesley, 1990.
23
24[19] D. Feitelson. Communicators: Object-based multiparty interactions for parallel pro-
25gramming. Technical Report 91-12, Dept. Computer Science, The Hebrew University
26of Jerusalem, November 1991. 6.1.2
27
[20] C++ Forum. Working paper for draft proposed international standard for informa-
28
tion systems | programming language C++. Technical report, American National
29
Standards Institute, 1995.
30
31[21] Message Passing Interface Forum. MPI: A Message-Passing Interface standard. The In-
32ternational Journal of Supercomputer Applications and High Performance Computing,
338, 1994. 1.3
34
35[22] Message Passing Interface Forum. MPI: A Message-Passing Interface standard (version
361.1). Technical report, 1995. http://www.mpi-forum.org. 1.3
37
[23] Al Geist, Adam Beguelin, Jack Dongarra, Weicheng Jiang, Bob Manchek, and Vaidy
38
Sunderam. PVM: Parallel Virtual Machine|A User's Guide and Tutorial for Network
39
Parallel Computing. MIT Press, 1994. 10.1
40
41[24] G. A. Geist, M. T. Heath, B. W. Peyton, and P. H. Worley. PICL: A portable in-
42strumented communications library, C reference manual. Technical Report TM-11130,
43Oak Ridge National Laboratory, Oak Ridge, TN, July 1990. 1.2
44
45[25] William D. Gropp and Barry Smith. Chameleon parallel programming tools users
46manual. Technical Report ANL-93/23, Argonne National Laboratory, March 1993. 1.2
47
48
BIBLIOGRAPHY |
|
|
603 |
|
[26] Michael |
Hennecke. |
A Fortran 90 interface to MPI version 1.1. |
Technical |
|
Report |
Internal |
Report 63/96, Rechenzentrum, Universit•at Karlsruhe, D- |
||
76128 Karlsruhe, |
Germany, June 1996. |
Available via world wide |
web from |
http://www.uni-karlsruhe.de/~Michael.Hennecke/Publications/#MPI_F90. 16.2.4
[27]Institute of Electrical and Electronics Engineers, New York. IEEE Standard for Binary Floating-Point Arithmetic, ANSI/IEEE Standard 754-1985, 1985. 13.5.2
[28]International Organization for Standardization, Geneva. Information processing | 8-bit single-byte coded graphic character sets | Part 1: Latin alphabet No. 1, 1987. 13.5.2
[29]International Organization for Standardization, Geneva. Information technology | Portable Operating System Interface (POSIX) | Part 1: System Application Program Interface (API) [C Language], December 1996. 12.4, 13.2.1
[30]Charles H. Koelbel, David B. Loveman, Robert S. Schreiber, Guy L. Steele Jr., and Mary E. Zosel. The High Performance Fortran Handbook. MIT Press, 1993. 4.1.4
[31]David Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the 1994 Symposium on Operating Systems Design and Implementation, pages 61{74, November 1994. Updated as Dartmouth TR PCS-TR94-226 on November 8, 1994. 13.1
[32]O. Kr•amer and H. M•uhlenbein. Mapping strategies in message-based multiprocessor systems. Parallel Computing, 9:213{225, 1989. 7.1
[33]S. J. Le et, R. S. Fabry, W. N. Joy, P. Lapsley, S. Miller, and C. Torek. An advanced 4.4BSD interprocess communication tutorial, Unix programmer's supplementary documents (PSD) 21. Technical report, Computer Systems Research Group, Depertment of Electrical Engineering and Computer Science, University of California, Berkeley, 1993. Also available at http://www.netbsd.org/Documentation/lite2/psd/. 10.5.5
[34]nCUBE Corporation. nCUBE 2 Programmers Guide, r2.0, December 1990. 1.2
[35]Bill Nitzberg. Performance of the iPSC/860 Concurrent File System. Technical Report RND-92-020, NAS Systems Division, NASA Ames, December 1992. 13.1
[36]William J. Nitzberg. Collective Parallel I/O. PhD thesis, Department of Computer and Information Science, University of Oregon, December 1995. 13.1
[37]4.4BSD Programmer's Supplementary Documents (PSD). O'Reilly and Associates, 1994. 10.5.5
[38]Paul Pierce. The NX/2 operating system. In Proceedings of the Third Conference on Hypercube Concurrent Computers and Applications, pages 384{390. ACM Press, 1988. 1.2
[39]K. E. Seamons, Y. Chen, P. Jones, J. Jozwiak, and M. Winslett. Server-directed collective I/O in Panda. In Proceedings of Supercomputing '95, December 1995. 13.1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
604 |
BIBLIOGRAPHY |
1[40] A. Skjellum and A. Leung. Zipcode: a portable multicomputer communication library
2atop the reactive kernel. In D. W. Walker and Q. F. Stout, editors, Proceedings of the
3Fifth Distributed Memory Concurrent Computing Conference, pages 767{776. IEEE
4Press, 1990. 1.2, 6.1.2
5
6[41] A. Skjellum, S. Smith, C. Still, A. Leung, and M. Morari. The Zipcode message passing
7system. Technical report, Lawrence Livermore National Laboratory, September 1992.
8
9
10
11
12
13
1.2
[42]Anthony Skjellum, Nathan E. Doss, and Purushotham V. Bangalore. Writing Libraries in MPI. In Anthony Skjellum and Donna S. Reese, editors, Proceedings of the Scalable Parallel Libraries Conference, pages 166{173. IEEE Computer Society Press, October 1993. 6.1
14[43] Anthony Skjellum, Nathan E. Doss, and Kishore Viswanathan. Inter-communicator
15extensions to MPI in the MPIX (MPI eXtension) Library. Technical Report MSU-
16
17
18
940722, Mississippi State University | Dept. of Computer Science, April 1994. http://www.erc.msstate.edu/mpi/mpix.html. 5.2.2
19[44] Anthony Skjellum, Ziyang Lu, Purushotham V. Bangalore, and Nathan E. Doss. Ex-
20plicit parallel programming in C++ based on the message-passing interface (MPI). In
21Gregory V. Wilson, editor, Parallel Programming Using C++, Engineering Computa-
22tion Series. MIT Press, July 1996. ISBN 0-262-73118-5.
23
[45] Anthony Skjellum, Steven G. Smith, Nathan E. Doss, Alvin P. Leung, and Manfred
24
Morari. The Design and Evolution of Zipcode. Parallel Computing, 20(4):565{596,
25
April 1994. 6.1.2, 6.5.6
26
27[46] Rajeev Thakur and Alok Choudhary. An Extended Two-Phase Method for Accessing
28Sections of Out-of-Core Arrays. Scienti c Programming, 5(4):301{317, Winter 1996.
2913.1
30 |
[47] The Unicode Standard, Version 2.0. Addison-Wesley, 1996. ISBN 0-201-48345-9. 13.5.2 |
31 |
32
[48] D. Walker. Standards for message passing in a distributed memory environment. Tech-
33
nical Report TM-12147, Oak Ridge National Laboratory, August 1992. 1.2
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48