.. _ch:mpi_collective:


Collective Communication
===========================

`Collective operations <https://www.mpi-forum.org/docs/mpi-1.1/mpi-11-html/node64.html>`_ involve communication among a group of processes rather than just between pairs of processes.
These operations are essential for parallel algorithms and can significantly improve the efficiency of parallel programs by allowing multiple processes to work together.

Common Operations
-------------------

1. **MPI_Bcast (Broadcast):** 

  This operation sends a message from one process (the root) to all other processes in the communicator.

  .. code-block:: cpp

    MPI_Bcast(void* buffer, int count, MPI_Datatype datatype, int root, MPI_Comm comm);

2. **MPI_Scatter (Scatter):**

  This operation divides data on the root process and distributes the obtained chunks to the other processes in the communicator.

  .. code-block:: cpp

      MPI_Scatter(void* sendbuf, int sendcount, MPI_Datatype sendtype,
                  void* recvbuf, int recvcount, MPI_Datatype recvtype,
                  int root, MPI_Comm comm);

3. **MPI_Gather (Gather):**

  The inverse of MPI_Scatter, this operation collects data chunks from all processes and sends it to the root process.

  .. code-block:: cpp

    MPI_Gather(void* sendbuf, int sendcount, MPI_Datatype sendtype,
               void* recvbuf, int recvcount, MPI_Datatype recvtype,
               int root, MPI_Comm comm);

4. **MPI_Reduce (Reduce):**

  This operation performs a reduction operation (e.g., sum, product, maximum) on data ditributed across all processes.
  The result stored on a specified root process.

  .. code-block:: cpp

    MPI_Reduce(void* sendbuf, void* recvbuf, int count, MPI_Datatype datatype,
               MPI_Op op, int root, MPI_Comm comm);

5. **MPI_Allreduce (Allreduce):**

  Similar to MPI_Reduce, but the result is broadcasted to all processes.

  .. code-block:: cpp

    MPI_Allreduce(void* sendbuf, void* recvbuf, int count, MPI_Datatype datatype,
                  MPI_Op op, MPI_Comm comm);

6. **MPI_Barrier (Barrier):**

  This operation synchronizes all processes in the communicator.

  .. code-block:: cpp

    MPI_Barrier(MPI_Comm comm);

.. image:: /images/mpi_tree_broadcast.gif
  :alt: mpi_tree_broadcast
  :width: 200
  :align: center

.. centered:: Fig 7.1.1 Broadcast with a tree approach. 

.. admonition:: Task

  #. Write a C/C++ function named "my_broadcast" in which the root process broadcasts the values of an array of doubles to all other processes.
  #. Initialize the array of double values with a user-defined size, N, specified through the Terminal.
  #. Use only Point-to-Point communication. Your approach could look like a tree (Fig. 7.1.1).
  #. Benchmark your "my_broadcast" function for N=100000 and 10 processes on the ``short`` or ``standard`` partition of the draco cluster. Compare the runtime with the standard "MPI_Bcast" function. Use the "MPI_Wtime()" function described in Section 6.3 for your measurements.
  #. Include the slurm script, the job output, the implementation itself and all necessary documentation in your submission.

  Bonus:
    - We compare the runtime of your implementation with the implementations of the other teams
    - The winner receives 2 extra points
    - submit an anonymous name for your team

Communicators
-------------------
In all previous exercises, we have used the communicator ``MPI_COMM_WORLD``. Often this is sufficient because we only communicate with one or all processes at a time. For more complex applications, we may need to communicate with only a few processes at a time. In this case we can create a new communicator.

**MPI_Comm_split:**
Creates new communicators by “splitting” a communicator into a group of sub-communicators based on the input values color and key.

.. code-block:: cpp

  int MPI_Comm_split( MPI_Comm comm, 
  		      int color, 
  		      int key, 
  		      MPI_Comm *newcomm);

- ``comm``: The old communicator.
- ``color``: Determines to which new communicator each process will belong. All processes that pass the same value for ``color`` are assigned to the same communicator.
- ``key``: Determines the rank within each new communicator. The process that passes the smallest value for ``key`` will be rank 0.
- ``newcomm``: The new communicator.

.. admonition:: Task 

  **Custom communicators**

  #. Create two communicators by splitting ``MPI_COMM_WORLD``  with ``MPI_Comm_Split`` based on odd and even ranks.
  #. Use the ranks of ``MPI_COMM_WORLD`` as ``key`` for your new communicators.
  #. Broadcast a random integer from rank 0 to all other ranks in the respective communicators.
  #. Print the received data on each rank after the broadcast
  #. Free your communicators with ``MPI_Comm_free``.