Home

Welcome to the dist-gnn wiki!

Preliminary Experiments

gloo Reddit 200 full barrier

33.75s 19200 barrier 165.67s 6400 broadcast 15.91s 6400 spmm 218.40s 200 train

gloo Reddit 100 full barrier

18.67s 9600 barrier 83.10s 3200 broadcast 8.03s 3200 spmm 111.82s 100 train

gloo Reddit 100 no barrier with sync

97.54s 3200 broadcast 6.07s 3200 spmm 0.62s 600 mm 0.80s 200 all reduce 105.73s 100 train

gloo Reddit 100 full barrier with sync

19.32s 9600 barrier 85.14s 3200 broadcast 6.12s 3200 spmm 0.84s 600 mm 0.66s 200 all reduce 112.86s 100 train

nccl Reddit 100 no barrier with sync

12.89s 3200 broadcast 4.85s 3200 spmm 0.69s 600 mm 0.13s 200 all reduce 19.08s 100 train

nccl Reddit 100 full barrier with sync

3.69s 9600 barrier 12.28s 3200 broadcast 4.72s 3200 spmm 0.45s 600 mm 0.05s 200 all reduce 21.50s 100 train

nccl Reddit 100 no barrier with sync naive P2P

38.55s 3200 broadcast 4.87s 3200 spmm 0.48s 600 mm 0.52s 200 all reduce 44.95s 100 train