Collective communication involves all the processes in a communicator. (Communicators were introduced in
Basics of MPI Programming.) The purpose of collective communication is to manipulate a shared
piece or set of information. The collective communication routines have been built upon point-to-point
communication routines. You could build your own collective communication routines in this way, but it might
involve a lot of tedious work and might not be as efficient.
Although other message-passing libraries provide some collective communication calls, none of
them provides a set that is as complete and robust as the one provided by MPI.
In this module, we introduce these routines in three categories: synchronization, data movement,
and global computation.