Based on this insight, we propose DirectReduce, a fully offloading ring all-reduce architecture that is comprised of three components: (i) the GateKeeper module, responsible for evaluating outgoing ...

DirectReduce: A Scalable Ring AllReduce Offloading Architecture for Torus Topologies Abstract: The all-reduce operation is critically important for communication-intensive workloads emerging at the ...