Exploring Parallel Computing using MPI and C++: Part 3 - Sending and Receiving Messages using MPI

Introduction

Welcome back to our blog series on parallel computing using MPI and C++. In the previous posts, we covered the basics of parallel computing, introduced MPI programming concepts, and explored collective communication operations in MPI. In this third instalment, we will dive into the core functionality of MPI: sending and receiving messages between processes. We will explore the different communication modes, and point-to-point communication functions, and provide practical examples in C++ to demonstrate how to send and receive messages using MPI.

Understanding Point-to-Point Communication in MPI

Point-to-point communication is a fundamental aspect of parallel computing, where individual processes exchange messages with one another. MPI provides several functions for point-to-point communication, allowing processes to send and receive messages efficiently. The two primary modes of point-to-point communication in MPI are:

MPI provides the following functions for sending messages:

1. MPI_Send: This function sends a message from the sender process to a specific receiver process. It takes the data buffer, the number of elements to send, the data type, the rank of the receiver process, and a tag as parameters. The tag can be used to identify different types or categories of messages.

2. MPI_Isend: This non-blocking version of MPI_Send initiates the message send operation but allows the sender to continue its execution immediately without waiting for the completion of the send operation. It returns an MPI_Request object that can be used to check the completion status later using MPI_Test or MPI_Wait.

Receiving Messages using MPI: MPI provides the following functions for receiving messages:

1. MPI_Recv: This function receives a message into the receiver process from a specific sender process. It takes the data buffer, the number of elements to receive, the data type, the rank of the sender process, and a tag as parameters. The tag should match the tag used in the corresponding send operation to ensure proper message matching.

2. MPI_Irecv: This non-blocking version of MPI_Recv initiates the receive operation but allows the receiver to continue its execution immediately without waiting for the arrival of the message. It returns an MPI_Request object that can be used to check the completion status later using MPI_Test or MPI_Wait.

Example: Sending and Receiving Messages using MPI in C++

Let's illustrate the concepts of sending and receiving messages using MPI with a practical example. Consider a scenario where multiple processes want to exchange integer data with each other. The root process sends an integer value to all other processes, and each process multiplies the received value by its rank and sends it back to the root process. Here's the code:

#include <iostream>
#include <mpi.h>

int main(int argc, char** argv) {
    MPI_Init(&argc, &argv);

    int rank, size;
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &size);

    int value;
    if (rank == 0) {
        value = 10;
        std::cout << "Root process (Rank 0) sending value: " << value

Conclusion

In this blog post, we explored the essential concept of sending and receiving messages using MPI in parallel computing. Point-to-point communication is a fundamental aspect of parallel programming, allowing processes to exchange data and synchronize their execution.

We learned about the two modes of point-to-point communication in MPI: synchronous and asynchronous. Synchronous mode ensures coordination between sender and receiver, while asynchronous mode provides more flexibility but may require additional synchronization mechanisms.

We also discussed the MPI_Send and MPI_Recv functions, which are used for sending and receiving messages, respectively. These functions allow processes to exchange data, specifying the data buffer, the number of elements, data types, and the rank of the sender and receiver processes.

Additionally, we explored the non-blocking versions of the send and receive functions, MPI_Isend and MPI_Irecv, which allow processes to continue their execution immediately without waiting for the completion of the communication operation. These non-blocking functions provide more flexibility in program design.

To illustrate the concepts, we provided a practical example in C++, where multiple processes exchange integer data. The root process sends a value to all other processes, and each process multiplies the received value by its rank before sending it back to the root process.

Understanding the concepts of sending and receiving messages using MPI is crucial for developing efficient parallel programs. By leveraging point-to-point communication, processes can collaborate, exchange data, and synchronize their execution to solve complex problems more efficiently.

In the next part of our series, we will delve into MPI collective I/O, which provides parallel input/output operations for accessing and manipulating external files in parallel programs. We will explore how collective I/O can significantly improve I/O performance by allowing multiple processes to read from or write to a file simultaneously.

Stay tuned for Part 4, where we will discuss MPI collective I/O and further advance our understanding of MPI programming concepts. In the meantime, start practising sending and receiving messages in your MPI programs to enable efficient communication among processes and enhance the scalability and performance of your parallel applications. Keep Bussing!!!