Abstract
Generative Adversarial Networks (GANs) are gaining popularity with applications in unsupervised, supervised as well as reinforcement learning for generating images, videos, and other artefacts. However, the inherently sequential nature of GANs is an Achilles heel to widespread adoption. In this paper, we propose and experimentally evaluate a novel sophisticated pipelined parallel version of GANs by dividing the training process into different balanced pipeline stages. Our experimental evaluation of the proposed technique shows significant performance gain up to 30% and 23% with an average speed-up close to 23% and 15% as compared to the serial implementation in the context of NumPy and Pytorch respectively when used to accurately classify real and fake images from standard MNIST and Fashion MNIST datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27, 2672–2680 (2014)
Narayanan, D., Harlap, A., Phanishayee, A., Seshadri, V., Devanur, N.R., Ganger, G.R., Gibbons, P.B., Zaharia, M.: PipeDream: generalized pipeline parallelism for DNN training. In: Proceedings of the 27th ACM Symposium on Operating Systems Principles (SOSP ’19). Association for Computing Machinery, New York, NY, USA, pp. 1–15 (2019)
Huang, Y., Cheng, Y., Chen, D., Lee, H., Ngiam, J., Le, Q.V., Chen, Z.: Efficient Training of Giant Neural Networks using Pipeline Parallelism, GPipe (2018)
Lee, S., Jha, D., Agrawal, A., Choudhary, A., Liao, W.: Parallel deep convolutional neural network training by exploiting the overlapping of computation and communication. In: 2017 IEEE 24th International Conference on High Performance Computing (HiPC), Jaipur, pp. 183–192 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Chandan, R., Pentapati, N., Koushik, R.M., Nagpal, R. (2021). A High Performance Pipelined Parallel Generative Adversarial Network (PipeGAN). In: Saini, H.S., Sayal, R., Govardhan, A., Buyya, R. (eds) Innovations in Computer Science and Engineering. Lecture Notes in Networks and Systems, vol 171. Springer, Singapore. https://doi.org/10.1007/978-981-33-4543-0_81
Download citation
DOI: https://doi.org/10.1007/978-981-33-4543-0_81
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-33-4542-3
Online ISBN: 978-981-33-4543-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)