ChainerMN

chainerMN ChainerMN depends on MPI for inter-node communication, such that it can benefit from faster network infrastructure such as InfiniBand. NVIDIA’s NCCL is then used for intra-node communication between GPUs.

The following shows the benefits of leveraging InfiniBand for performance and allows ChainerMN to achieve ~72% accuracy.

chainerMN
For more information on ChainerMN, read more here.