7. Collective Communication

Collective operations involve communication among a group of processes rather than just between pairs of processes. These operations are essential for parallel algorithms and can significantly improve the efficiency of parallel programs by allowing multiple processes to work together.

7.1. Common Operations

  1. MPI_Bcast (Broadcast):

This operation sends a message from one process (the root) to all other processes in the communicator.

MPI_Bcast(void* buffer, int count, MPI_Datatype datatype, int root, MPI_Comm comm);

  1. MPI_Scatter (Scatter):

This operation divides data on the root process and distributes the obtained chunks to the other processes in the communicator.

MPI_Scatter(void* sendbuf, int sendcount, MPI_Datatype sendtype,
            void* recvbuf, int recvcount, MPI_Datatype recvtype,
            int root, MPI_Comm comm);

  1. MPI_Gather (Gather):

The inverse of MPI_Scatter, this operation collects data chunks from all processes and sends it to the root process.

MPI_Gather(void* sendbuf, int sendcount, MPI_Datatype sendtype,
           void* recvbuf, int recvcount, MPI_Datatype recvtype,
           int root, MPI_Comm comm);

  1. MPI_Reduce (Reduce):

This operation performs a reduction operation (e.g., sum, product, maximum) on data ditributed across all processes. The result stored on a specified root process.

MPI_Reduce(void* sendbuf, void* recvbuf, int count, MPI_Datatype datatype,
           MPI_Op op, int root, MPI_Comm comm);

  1. MPI_Allreduce (Allreduce):

Similar to MPI_Reduce, but the result is broadcasted to all processes.

MPI_Allreduce(void* sendbuf, void* recvbuf, int count, MPI_Datatype datatype,
              MPI_Op op, MPI_Comm comm);

  1. MPI_Barrier (Barrier):

This operation synchronizes all processes in the communicator.

MPI_Barrier(MPI_Comm comm);
mpi_tree_broadcast

Fig 7.1.1 Broadcast with a tree approach.

Task

  1. Initialize an array of double values with a user-defined size, N, specified through the Terminal.

  2. Write a C/C++ function named “tree_broadcast” that uses a tree approach (like Fig. 7.1.1) for broadcasting, employing blocking communication.

  3. Compare the runtime of your “tree_broadcast” function with the standard “MPI_Bcast” function.

  4. Submit your work using a batch script. Include the script, the job output, and the implementation itself in the submission.