-
Profiled DDL RAMP
Collection of profiled models used to estimate the disrtibuted training time for different Transformer Encoder models partiotioned using Megatron partitioning strategy, for... -
Data for figures and additional analysis for RAMP
Figures in pdf, with the relevant data in csv format for the resul figures in the paper "RAMP: A Flat Nanosecond Optical Network and MPI Operations for Distributed Computing and...