- •Contents
- •List of Figures
- •List of Tables
- •Acknowledgments
- •Introduction to MPI
- •Overview and Goals
- •Background of MPI-1.0
- •Background of MPI-1.1, MPI-1.2, and MPI-2.0
- •Background of MPI-1.3 and MPI-2.1
- •Background of MPI-2.2
- •Who Should Use This Standard?
- •What Platforms Are Targets For Implementation?
- •What Is Included In The Standard?
- •What Is Not Included In The Standard?
- •Organization of this Document
- •MPI Terms and Conventions
- •Document Notation
- •Naming Conventions
- •Semantic Terms
- •Data Types
- •Opaque Objects
- •Array Arguments
- •State
- •Named Constants
- •Choice
- •Addresses
- •Language Binding
- •Deprecated Names and Functions
- •Fortran Binding Issues
- •C Binding Issues
- •C++ Binding Issues
- •Functions and Macros
- •Processes
- •Error Handling
- •Implementation Issues
- •Independence of Basic Runtime Routines
- •Interaction with Signals
- •Examples
- •Point-to-Point Communication
- •Introduction
- •Blocking Send and Receive Operations
- •Blocking Send
- •Message Data
- •Message Envelope
- •Blocking Receive
- •Return Status
- •Passing MPI_STATUS_IGNORE for Status
- •Data Type Matching and Data Conversion
- •Type Matching Rules
- •Type MPI_CHARACTER
- •Data Conversion
- •Communication Modes
- •Semantics of Point-to-Point Communication
- •Buffer Allocation and Usage
- •Nonblocking Communication
- •Communication Request Objects
- •Communication Initiation
- •Communication Completion
- •Semantics of Nonblocking Communications
- •Multiple Completions
- •Non-destructive Test of status
- •Probe and Cancel
- •Persistent Communication Requests
- •Send-Receive
- •Null Processes
- •Datatypes
- •Derived Datatypes
- •Type Constructors with Explicit Addresses
- •Datatype Constructors
- •Subarray Datatype Constructor
- •Distributed Array Datatype Constructor
- •Address and Size Functions
- •Lower-Bound and Upper-Bound Markers
- •Extent and Bounds of Datatypes
- •True Extent of Datatypes
- •Commit and Free
- •Duplicating a Datatype
- •Use of General Datatypes in Communication
- •Correct Use of Addresses
- •Decoding a Datatype
- •Examples
- •Pack and Unpack
- •Canonical MPI_PACK and MPI_UNPACK
- •Collective Communication
- •Introduction and Overview
- •Communicator Argument
- •Applying Collective Operations to Intercommunicators
- •Barrier Synchronization
- •Broadcast
- •Example using MPI_BCAST
- •Gather
- •Examples using MPI_GATHER, MPI_GATHERV
- •Scatter
- •Examples using MPI_SCATTER, MPI_SCATTERV
- •Example using MPI_ALLGATHER
- •All-to-All Scatter/Gather
- •Global Reduction Operations
- •Reduce
- •Signed Characters and Reductions
- •MINLOC and MAXLOC
- •All-Reduce
- •Process-local reduction
- •Reduce-Scatter
- •MPI_REDUCE_SCATTER_BLOCK
- •MPI_REDUCE_SCATTER
- •Scan
- •Inclusive Scan
- •Exclusive Scan
- •Example using MPI_SCAN
- •Correctness
- •Introduction
- •Features Needed to Support Libraries
- •MPI's Support for Libraries
- •Basic Concepts
- •Groups
- •Contexts
- •Intra-Communicators
- •Group Management
- •Group Accessors
- •Group Constructors
- •Group Destructors
- •Communicator Management
- •Communicator Accessors
- •Communicator Constructors
- •Communicator Destructors
- •Motivating Examples
- •Current Practice #1
- •Current Practice #2
- •(Approximate) Current Practice #3
- •Example #4
- •Library Example #1
- •Library Example #2
- •Inter-Communication
- •Inter-communicator Accessors
- •Inter-communicator Operations
- •Inter-Communication Examples
- •Caching
- •Functionality
- •Communicators
- •Windows
- •Datatypes
- •Error Class for Invalid Keyval
- •Attributes Example
- •Naming Objects
- •Formalizing the Loosely Synchronous Model
- •Basic Statements
- •Models of Execution
- •Static communicator allocation
- •Dynamic communicator allocation
- •The General case
- •Process Topologies
- •Introduction
- •Virtual Topologies
- •Embedding in MPI
- •Overview of the Functions
- •Topology Constructors
- •Cartesian Constructor
- •Cartesian Convenience Function: MPI_DIMS_CREATE
- •General (Graph) Constructor
- •Distributed (Graph) Constructor
- •Topology Inquiry Functions
- •Cartesian Shift Coordinates
- •Partitioning of Cartesian structures
- •Low-Level Topology Functions
- •An Application Example
- •MPI Environmental Management
- •Implementation Information
- •Version Inquiries
- •Environmental Inquiries
- •Tag Values
- •Host Rank
- •IO Rank
- •Clock Synchronization
- •Memory Allocation
- •Error Handling
- •Error Handlers for Communicators
- •Error Handlers for Windows
- •Error Handlers for Files
- •Freeing Errorhandlers and Retrieving Error Strings
- •Error Codes and Classes
- •Error Classes, Error Codes, and Error Handlers
- •Timers and Synchronization
- •Startup
- •Allowing User Functions at Process Termination
- •Determining Whether MPI Has Finished
- •Portable MPI Process Startup
- •The Info Object
- •Process Creation and Management
- •Introduction
- •The Dynamic Process Model
- •Starting Processes
- •The Runtime Environment
- •Process Manager Interface
- •Processes in MPI
- •Starting Processes and Establishing Communication
- •Reserved Keys
- •Spawn Example
- •Manager-worker Example, Using MPI_COMM_SPAWN.
- •Establishing Communication
- •Names, Addresses, Ports, and All That
- •Server Routines
- •Client Routines
- •Name Publishing
- •Reserved Key Values
- •Client/Server Examples
- •Ocean/Atmosphere - Relies on Name Publishing
- •Simple Client-Server Example.
- •Other Functionality
- •Universe Size
- •Singleton MPI_INIT
- •MPI_APPNUM
- •Releasing Connections
- •Another Way to Establish MPI Communication
- •One-Sided Communications
- •Introduction
- •Initialization
- •Window Creation
- •Window Attributes
- •Communication Calls
- •Examples
- •Accumulate Functions
- •Synchronization Calls
- •Fence
- •General Active Target Synchronization
- •Lock
- •Assertions
- •Examples
- •Error Handling
- •Error Handlers
- •Error Classes
- •Semantics and Correctness
- •Atomicity
- •Progress
- •Registers and Compiler Optimizations
- •External Interfaces
- •Introduction
- •Generalized Requests
- •Examples
- •Associating Information with Status
- •MPI and Threads
- •General
- •Initialization
- •Introduction
- •File Manipulation
- •Opening a File
- •Closing a File
- •Deleting a File
- •Resizing a File
- •Preallocating Space for a File
- •Querying the Size of a File
- •Querying File Parameters
- •File Info
- •Reserved File Hints
- •File Views
- •Data Access
- •Data Access Routines
- •Positioning
- •Synchronism
- •Coordination
- •Data Access Conventions
- •Data Access with Individual File Pointers
- •Data Access with Shared File Pointers
- •Noncollective Operations
- •Collective Operations
- •Seek
- •Split Collective Data Access Routines
- •File Interoperability
- •Datatypes for File Interoperability
- •Extent Callback
- •Datarep Conversion Functions
- •Matching Data Representations
- •Consistency and Semantics
- •File Consistency
- •Random Access vs. Sequential Files
- •Progress
- •Collective File Operations
- •Type Matching
- •Logical vs. Physical File Layout
- •File Size
- •Examples
- •Asynchronous I/O
- •I/O Error Handling
- •I/O Error Classes
- •Examples
- •Subarray Filetype Constructor
- •Requirements
- •Discussion
- •Logic of the Design
- •Examples
- •MPI Library Implementation
- •Systems with Weak Symbols
- •Systems Without Weak Symbols
- •Complications
- •Multiple Counting
- •Linker Oddities
- •Multiple Levels of Interception
- •Deprecated Functions
- •Deprecated since MPI-2.0
- •Deprecated since MPI-2.2
- •Language Bindings
- •Overview
- •Design
- •C++ Classes for MPI
- •Class Member Functions for MPI
- •Semantics
- •C++ Datatypes
- •Communicators
- •Exceptions
- •Mixed-Language Operability
- •Problems With Fortran Bindings for MPI
- •Problems Due to Strong Typing
- •Problems Due to Data Copying and Sequence Association
- •Special Constants
- •Fortran 90 Derived Types
- •A Problem with Register Optimization
- •Basic Fortran Support
- •Extended Fortran Support
- •The mpi Module
- •No Type Mismatch Problems for Subroutines with Choice Arguments
- •Additional Support for Fortran Numeric Intrinsic Types
- •Language Interoperability
- •Introduction
- •Assumptions
- •Initialization
- •Transfer of Handles
- •Status
- •MPI Opaque Objects
- •Datatypes
- •Callback Functions
- •Error Handlers
- •Reduce Operations
- •Addresses
- •Attributes
- •Extra State
- •Constants
- •Interlanguage Communication
- •Language Bindings Summary
- •Groups, Contexts, Communicators, and Caching Fortran Bindings
- •External Interfaces C++ Bindings
- •Change-Log
- •Bibliography
- •Examples Index
- •MPI Declarations Index
- •MPI Function Index
13.2. FILE MANIPULATION |
391 |
le size and end of le The size of an MPI le is measured in bytes from the beginning of the le. A newly created le has a size of zero bytes. Using the size as an absolute displacement gives the position of the byte immediately following the last byte in thele. For any given view, the end of le is the o set of the rst etype accessible in the current view starting after the last byte in the le.
le pointer A le pointer is an implicit o set maintained by MPI. \Individual le pointers" are le pointers that are local to each process that opened the le. A \shared le pointer" is a le pointer that is shared by the group of processes that opened the le.
le handle A le handle is an opaque object created by MPI_FILE_OPEN and freed by MPI_FILE_CLOSE. All operations on an open le reference the le through the le handle.
13.2 File Manipulation
13.2.1 Opening a File
MPI_FILE_OPEN(comm, lename, amode, info, fh)
IN |
comm |
communicator (handle) |
IN |
lename |
name of le to open (string) |
IN |
amode |
le access mode (integer) |
IN |
info |
info object (handle) |
OUT |
fh |
new le handle (handle) |
int MPI_File_open(MPI_Comm comm, char *filename, int amode, MPI_Info info, MPI_File *fh)
MPI_FILE_OPEN(COMM, FILENAME, AMODE, INFO, FH, IERROR)
CHARACTER*(*) FILENAME
INTEGER COMM, AMODE, INFO, FH, IERROR
fstatic MPI::File MPI::File::Open(const MPI::Intracomm& comm,
const char* filename, int amode, const MPI::Info& info)
(binding deprecated, see Section 15.2) g
MPI_FILE_OPEN opens the le identi ed by the le name lename on all processes in the comm communicator group. MPI_FILE_OPEN is a collective routine: all processes must provide the same value for amode, and all processes must provide lenames that reference the same le. (Values for info may vary.) comm must be an intracommunicator; it is erroneous to pass an intercommunicator to MPI_FILE_OPEN. Errors in MPI_FILE_OPEN are raised using the default le error handler (see Section 13.7, page 447). A process can open a le independently of other processes by using the MPI_COMM_SELF communicator. The le handle returned, fh, can be subsequently used to access the le until the le is closed using MPI_FILE_CLOSE. Before calling MPI_FINALIZE, the user is required to close (via MPI_FILE_CLOSE) all les that were opened with MPI_FILE_OPEN. Note that the
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
392 |
CHAPTER 13. I/O |
1communicator comm is una ected by MPI_FILE_OPEN and continues to be usable in all
2MPI routines (e.g., MPI_SEND). Furthermore, the use of comm will not interfere with I/O
3behavior.
4The format for specifying the le name in the lename argument is implementation
5
6
dependent and must be documented by the implementation.
7Advice to implementors. An implementation may require that lename include a
8
9
string or strings specifying additional information about the le. Examples include the type of lesystem (e.g., a pre x of ufs:), a remote hostname (e.g., a pre x of
10machine.univ.edu:), or a le password (e.g., a su x of /PASSWORD=SECRET).
11(End of advice to implementors.)
12
13
14
15
16
17
18
Advice to users. On some implementations of MPI, the le namespace may not be identical from all processes of all applications. For example, \/tmp/foo" may denote di erent les on di erent processes, or a single le may have many names, dependent on process location. The user is responsible for ensuring that a single le is referenced by the lename argument, as it may be impossible for an implementation to detect this type of namespace error. (End of advice to users.)
19Initially, all processes view the le as a linear byte stream, and each process views data
20in its own native representation (no data representation conversion is performed). (POSIX
21les are linear byte streams in the native representation.) The le view can be changed via
22the MPI_FILE_SET_VIEW routine.
23The following access modes are supported (speci ed in amode, a bit vector OR of the
24following integer constants):
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
MPI_MODE_RDONLY | read only,
MPI_MODE_RDWR | reading and writing,
MPI_MODE_WRONLY | write only,
MPI_MODE_CREATE | create the le if it does not exist,
MPI_MODE_EXCL | error if creating le that already exists,
MPI_MODE_DELETE_ON_CLOSE | delete le on close,
MPI_MODE_UNIQUE_OPEN | le will not be concurrently opened elsewhere,
MPI_MODE_SEQUENTIAL | le will only be accessed sequentially,
MPI_MODE_APPEND | set initial position of all le pointers to end of le.
Advice to users. C/C++ users can use bit vector OR (j) to combine these constants; Fortran 90 users can use the bit vector IOR intrinsic. Fortran 77 users can use (nonportably) bit vector IOR on systems that support it. Alternatively, Fortran users can portably use integer addition to OR the constants (each constant should appear at most once in the addition.). (End of advice to users.)
46Advice to implementors. The values of these constants must be de ned such that
47the bitwise OR and the sum of any distinct set of these constants is equivalent. (End
48of advice to implementors.)
13.2. FILE MANIPULATION |
393 |
The modes MPI_MODE_RDONLY, MPI_MODE_RDWR, MPI_MODE_WRONLY,
MPI_MODE_CREATE, and MPI_MODE_EXCL have identical semantics to their POSIX counterparts [29]. Exactly one of MPI_MODE_RDONLY, MPI_MODE_RDWR, or MPI_MODE_WRONLY, must be speci ed. It is erroneous to specify MPI_MODE_CREATE or MPI_MODE_EXCL in conjunction with MPI_MODE_RDONLY; it is erroneous to specify MPI_MODE_SEQUENTIAL together with MPI_MODE_RDWR.
The MPI_MODE_DELETE_ON_CLOSE mode causes the le to be deleted (equivalent to performing an MPI_FILE_DELETE) when the le is closed.
The MPI_MODE_UNIQUE_OPEN mode allows an implementation to optimize access by eliminating the overhead of le locking. It is erroneous to open a le in this mode unless the le will not be concurrently opened elsewhere.
Advice to users. For MPI_MODE_UNIQUE_OPEN, not opened elsewhere includes both inside and outside the MPI environment. In particular, one needs to be aware of potential external events which may open les (e.g., automated backup facilities). When MPI_MODE_UNIQUE_OPEN is speci ed, the user is responsible for ensuring that no such external events take place. (End of advice to users.)
The MPI_MODE_SEQUENTIAL mode allows an implementation to optimize access to some sequential devices (tapes and network streams). It is erroneous to attempt nonsequential access to a le that has been opened in this mode.
Specifying MPI_MODE_APPEND only guarantees that all shared and individual le pointers are positioned at the initial end of le when MPI_FILE_OPEN returns. Subsequent positioning of le pointers is application dependent. In particular, the implementation does not ensure that all writes are appended.
Errors related to the access mode are raised in the class MPI_ERR_AMODE.
The info argument is used to provide information regarding le access patterns and le system speci cs (see Section 13.2.8, page 398). The constant MPI_INFO_NULL can be used when no info needs to be speci ed.
Advice to users. Some le attributes are inherently implementation dependent (e.g.,le permissions). These attributes must be set using either the info argument or facilities outside the scope of MPI. (End of advice to users.)
Files are opened by default using nonatomic mode le consistency semantics (see Section 13.6.1, page 437). The more stringent atomic mode consistency semantics, required for atomicity of con icting accesses, can be set using MPI_FILE_SET_ATOMICITY.
13.2.2 Closing a File
MPI_FILE_CLOSE(fh)
INOUT fh le handle (handle)
int MPI_File_close(MPI_File *fh)
MPI_FILE_CLOSE(FH, IERROR)
INTEGER FH, IERROR
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
394 |
CHAPTER 13. I/O |
1fvoid MPI::File::Close() (binding deprecated, see Section 15.2) g
2
3MPI_FILE_CLOSE rst synchronizes le state (equivalent to performing an
4 |
MPI_FILE_SYNC), then closes the le associated with fh. The le is deleted if it was |
|
5opened with access mode MPI_MODE_DELETE_ON_CLOSE (equivalent to performing an
6 |
MPI_FILE_DELETE). MPI_FILE_CLOSE is a collective routine. |
|
|
||
7 |
Advice to users. If the le is deleted on close, and there are other processes currently |
|
8 |
||
accessing the le, the status of the le and the behavior of future accesses by these |
||
9 |
||
processes are implementation dependent. (End of advice to users.) |
||
10 |
||
|
||
11 |
The user is responsible for ensuring that all outstanding nonblocking requests and |
|
|
||
|
split collective operations associated with fh made by a process have completed before that |
|
|
process calls MPI_FILE_CLOSE. |
|
15 |
The MPI_FILE_CLOSE routine deallocates the le handle object and sets fh to |
|
MPI_FILE_NULL. |
||
12 |
|
|
13 |
|
|
14 |
|
|
16 |
|
|
17 |
13.2.3 Deleting a File |
|
18 |
||
|
||
19 |
|
|
20 |
|
21
22
23
24
25
MPI_FILE_DELETE( lename, info)
IN |
lename |
name of le to delete (string) |
IN |
info |
info object (handle) |
26int MPI_File_delete(char *filename, MPI_Info info)
27MPI_FILE_DELETE(FILENAME, INFO, IERROR)
28CHARACTER*(*) FILENAME
29INTEGER INFO, IERROR
30 |
fstatic void MPI::File::Delete(const char* filename, const MPI::Info& info) |
31 |
|
32 |
(binding deprecated, see Section 15.2) g |
33MPI_FILE_DELETE deletes the le identi ed by the le name lename. If the le does
34not exist, MPI_FILE_DELETE raises an error in the class MPI_ERR_NO_SUCH_FILE.
35The info argument can be used to provide information regarding le system speci cs
36(see Section 13.2.8, page 398). The constant MPI_INFO_NULL refers to the null info, and
37can be used when no info needs to be speci ed.
38If a process currently has the le open, the behavior of any access to the le (as well
39as the behavior of any outstanding accesses) is implementation dependent. In addition,
40whether an open le is deleted or not is also implementation dependent. If the le is not
41deleted, an error in the class MPI_ERR_FILE_IN_USE or MPI_ERR_ACCESS will be raised.
42Errors are raised using the default error handler (see Section 13.7, page 447).
43
44
45
46
47
48