inter_pipeline_hetero_decentralize.txt

Yuan, Binhang, et al. "Decentralized training of foundation models in heterogeneous environments." Advances in Neural Information Processing Systems 35 (2022): 25464-25477.;https://proceedings.neurips.cc/paper_files/paper/2022/hash/a37d615b61f999a5fa276adb14643476-Abstract-Conference.html;55;
Park, Jay H., et al. "{HetPipe}: Enabling large {DNN} training on (whimpy) heterogeneous {GPU} clusters through integration of pipelined model parallelism and data parallelism." 2020 USENIX Annual Technical Conference (USENIX ATC 20). 2020.;https://www.usenix.org/conference/atc20/presentation/park;122;