Data Parallelism vs. Model Parallelism (IMAGE)

University of Texas at Austin, Texas Advanced Computing Center

Data Parallelism vs. Model Parallelism

Caption

This is a schematic showing data parallelism vs. model parallelism, as they relate to neural network training.

Credit

Yang You, Zhao Zhang, Cho-Jui Hsieh, James Demmel, Kurt Keutzer

Usage Restrictions

None

License

Licensed content

Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.