Abstract
In order to offer guidelines for physics-informed neural network (PINN) implementation, this study presents a comprehensive review of PINN, an emerging field at the intersection of deep learning and computational physics. PINN offers a novel approach to solve physics problems by leveraging the flexibility and scalability of neural networks, even with small or no data. First, a general description of different physics problem types and target tasks addressable with PINN was provided. A generic PINN architecture was described in detail using a component-wise approach, with components ranging from collocation points to optimization methods. Then, we surveyed studies that sought to improve upon each of these components. To offer practical insights, we highlighted studies that focused on key issues of PINN implementation and showcased three practical applications. Lastly, a summary and potential research directions were provided to offer guidelines for reliable and customized PINN implementations.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
S. W. Kim, I. Kim and J. Lee, Seungchul knowledge integration into deep learning in dynamical systems: an overview and taxonomy, Journal of Mechanical Science and Technology, 35 (2021) 1331–1342.
M. P. Raissi and G. E. P. Karniadakis, Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational Physics, 378 (2019) 686–707.
S. Cai et al., Physics-informed neural networks (PINNs) for fluid mechanics: a review, Acta Mechanica Sinica, 37 (12) (2021) 1727–1738.
S. Cai et al., Physics-informed neural networks for heat transfer problems, J. Heat Transfer, 143 (6) (2021) 060801.
G. E. Karniadakis et al., Physics-informed machine learning, Nature Reviews Physics, 3 (6) (2021) 422–440.
S. Cuomo et al., Scientific Machine learning through physics-informed neural networks: where we are and what’s next, Journal of Scientific Computing, 92 (3) (2022) 88.
S. Das and S. Tesfamariam, State-of-the-art review of design of experiments for physics-informed deep learning, arXiv.2202.06416 (2022) DOI: https://doi.org/10.48550/arXiv.2202.06416.
X. Mou, Q. Fang and S. Li, A hybrid neural network and data sampling solver for forward and backward modiied diffusion equations, Research Square (2022) https://www.research-square.com/article/rs-2059725/v1 (Preprint).
L. Lu et al., DeepXDE: a deep learning library for solving differential equations, SIAM Review, 63 (1) (2021) 208–228.
J. M. Hanna et al., Residual-based adaptivity for two-phase flow simulation in porous media using physics-informed neural networks, Computer Methods in Applied Mechanics and Engineering, 396 (2022) 115100.
C. Wu et al., A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks, Computer Methods in Applied Mechanics and Engineering, 403 (2023) 115671.
W. Peng et al., RANG: a residual-based adaptive node generation method for physics-informed neural networks, arXiv:2205.01051 (2022) DOI: https://doi.org/10.48550/arXiv.2205.01051.
S. Subramanian et al., Adaptive self-supervision algorithms for physics-informed neural networks, arXiv:2207.04084 (2022) DOI: https://doi.org/10.48550/arXiv.2207.04084.
A. D. Jagtap, K. Kawaguchi and G. E. Karniadakis, Adaptive activation functions accelerate convergence in deep and physics-informed neural networks, Journal of Computational Physics, 404 (2020) 109136.
A. D. Jagtap, K. Kawaguchi and G. Em Karniadakis, Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 476 (2239) (2020) 20200334.
R. Gnanasambandam et al., Self-scalable tanh (stan): faster convergence and better generalization in physics-informed neural networks, arXiv:2204.12589 (2022) DOI: https://doi.org/10.48550/arXiv.2204.12589.
J. Abbasi and P. Ø. Andersen, Physical activation functions (pafs): an approach for more efficient induction of physics into physics-informed neural networks (PINNs), arXiv:2205.14630 (2022) DOI: https://doi.org/10.48550/arXiv.2205.14630.
W. Peng et al., Accelerating physics-informed neural network training with prior dictionaries, arXiv:2004.08151 (2020) DOI: https://doi.org/10.48550/arXiv.2004.08151.
K. He et al., Deep residual learning for image recognition, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA (2016).
C. Cheng and G.-T. Zhang, Deep learning method based on physics informed neural network with resnet block for solving fluid flow problems, Water, 13 (2021) 423, DOI: https://doi.org/10.3390/w13040423.
C. Moya and G. Lin, DAE-PINN: a physics-informed neural network model for simulating differential algebraic equations with application to power networks, Neural Computing and Applications, 35 (5) (2023) 3789–3804.
V. Dwivedi and B. Srinivasan, Physics informed extreme learning machine (PIELM)–a rapid method for the numerical solution of partial differential equations, Neurocomputing, 391 (2020) 96–118.
E. Schiassi et al., Extreme theory of functional connections: A fast physics-informed neural network method for solving ordinary and partial differential equations, Neurocomputing, 457 (2021) 334–356.
A. A. Ramabathiran and P. Ramachandran, SPINN: sparse, physics-based, and partially interpretable neural networks for PDEs, Journal of Computational Physics, 445 (2021) 110600.
H. Gao, L. Sun and J.-X. Wang, PhyGeoNet: physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain, Journal of Computational Physics, 428 (2021) 110079.
L. Sun et al., Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data, Computer Methods in Applied Mechanics and Engineering, 361 (2020) 112732.
L. Lu et al., Physics-informed neural networks with hard constraints for inverse design, SIAM Journal on Scientific Computing, 43 (6) (2021) B1105–B1132.
R. Gong and Z. Tang, Further investigation of convolutional neural networks applied in computational electromagnetism under physics-informed consideration, IET Electr. Power Appl., 16 (6) (2022) 653–674.
X. Zhao et al., Physics-informed convolutional neural networks for temperature field prediction of heat source layout without labeled data, Engineering Applications of Artificial Intelligence, 117 (2023) 105516.
Y. Yang and P. Perdikaris, Adversarial uncertainty quantification in physics-informed neural networks, Journal of Computational Physics, 394 (2019) 136–152.
L. Yang et al., Highly-scalable, physics-informed GANs for learning solutions of stochastic PDEs, 2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS), Penver, CO, USA (2019).
A. Daw, M. Maruf and A. Karpatne, PID-GAN: a GAN framework based on a physics-informed discriminator for uncertainty quantification with physics, Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, Association for Computing Machinery, Virtual Event, Singapore (2021) 237–247.
W. Zhong and H. Meidani, PI-VAE: physics-informed variational auto-encoder for stochastic differential equations, Computer Methods in Applied Mechanics and Engineering, 403 (2023) 115664.
Y. A. Yucesan and F. A. C. Viana, A hybrid physics-informed neural network for main bearing fatigue prognosis under grease quality variation, Mechanical Systems and Signal Processingy, 171 (2022) 108875.
R. Zhang, Y. Liu and H. Sun, Physics-informed multi-LSTM networks for metamodeling of nonlinear structures, Computer Methods in Applied Mechanics and Engineering, 369 (2020) 113226.
P. Ren et al., PhyCRNet: physics-informed convolutional-recurrent network for solving spatiotemporal PDEs, Computer Methods in Applied Mechanics and Engineering, 389 (2022) 114399.
L. Lu et al., Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators, Nature Machine Intelligence, 3 (3) (2021) 218–229.
K. Hornik and M. W. Stinchcombe, Halbert, multilayer feedforward networks are universal approximators, Neural Networks, 2 (1989) 359–366.
S. Wang, H. Wang and P. Perdikaris, Learning the solution operator of parametric partial differential equations with physics-informed DeepONets, Science Advances, 7 (40) (2021) eabi8605.
S. Cai et al., DeepM&Mnet: inferring the electroconvection multiphysics fields based on operator approximation by neural networks, Journal of Computational Physics, 436 (2021) 110296.
L. Yang, X. Meng and G. E. Karniadakis, B-PINNs: bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data, Journal of Computational Physics, 425 (2021) 109913.
J. Li, J. Chen and B. Li, Gradient-optimized physics-informed neural networks (GOPINNs): a deep learning method for solving the complex modified KdV equation, Nonlinear Dynamics, 107 (1) (2022) 781–792.
H. Gao, M. J. Zahr and J.-X. Wang, Physics-informed graph neural Galerkin networks: a unified framework for solving PDE-governed forward and inverse problems, Computer Methods in Applied Mechanics and Engineering, 390 (2022) 114502.
W. Liu and M. J. Pyrcz, Physics-informed graph neural network for spatial-temporal production forecasting, Geoenergy Science and Engineering, 223 (2023) 211486.
M. Yang and J. T. Foster, Multi-output physics-informed neural networks for forward and inverse PDE problems with uncertainties, Computer Methods in Applied Mechanics and Engineering, 402 (2022) 115041.
P.-H. Chiu et al., CAN-PINN: a fast physics-informed neural network based on coupled-automatic-numerical differentiation method, Computer Methods in Applied Mechanics and Engineering, 395 (2022) 114909.
C. L. Wight and J. Zhao, Solving allen-cahn and cahn-hilliard equations using the adaptive physics informed neural networks, arXiv:2007.04542 (2020) DOI: https://doi.org/10.48550/arXiv.2007.05452.
S. Wang, Y. Teng and P. Perdikaris, Understanding and mitigating gradient flow pathologies in physics-informed neural networks, SIAM Journal on Scientific Computing, 43 (5) (2021) 3055–3081.
J. Yu et al., Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems, Computer Methods in Applied Mechanics and Engineering, 393 (2022) 114823.
F. Xiong et al., Gradient-weighted physics-informed neural networks for one- dimensional euler equation dimensional euler equation, TechRxiv. (2022) DOI: https://doi.org/10.36227/techrxiv.20099957.v1(Preprint).
Y. Liu et al., Physics-informed neural networks based on adaptive weighted loss functions for Hamilton-Jacobi equations, Mathematical Biosciences and Engineering, 19 (12) (2022) 12866–12896.
S. Wang, X. Yu and P. Perdikaris, When and why PINNs fail to train: a neural tangent kernel perspective, Journal of Computational Physics, 449 (2022) 110768.
D. Liu and Y. Wang, A dual-dimer method for training physics-constrained neural networks with minimax architecture, Neural Networks, 136 (2021) 112–125.
L. D. McClenny and U. M. Braga-Neto, Self-adaptive physics-informed neural networks, Journal of Computational Physics, 474 (2023) 111722.
P. Nasiri and R. Dargazany, Reduced-PINN: an integration-based physics-informed neural networks for stiff ODEs, arXiv:2208.12045v1 (2022) DOI: https://doi.org/10.48550/arXiv.2208.12045.
S. Maddu et al., Inverse dirichlet weighting enables reliable training of physics informed neural networks, Machine Learning: Science and Technology, 3 (1) (2022) 015026.
E. Kharazmi, Z. Zhang and G. E. Karniadakis, Variational physics-informed neural networks for solving partial differential equations, arXiv:1912.00873 (2019) DOI: https://doi.org/10.48550/arXiv.1912.00873.
R. Khodayi-Mehr and M. Zavlanos, VarNet: variational neural networks for the solution of partial differential equations, Proceedings of the 2nd Conference on Learning for Dynamics and Control (2020) 298–307.
E. Kharazmi, Z. Zhang and G. E. M. Karniadakis, hp-VPINNs: variational physics-informed neural networks with domain decomposition, Computer Methods in Applied Mechanics and Engineering, 374 (2021) 113547.
W. E. and B. Yu, The deep ritz method: a deep learning-based numerical algorithm for solving variational problems, Communications in Mathematics and Statistics, 6 (1) (2018) 1–12.
J. Bai et al., A physics-informed neural network technique based on a modified loss function for computational 2D and 3D solid mechanics, Computational Mechanics, 71 (2022) 1–20.
C. Wang et al., Is L2 physics-informed loss always suitable for training physics-informed neural network? Advances in Neural Information Proceeding Systems 35 (NeurIPS 2022), New Orleans, USA (2022).
C. Davi and U. Braga-Neto, PSO-PINN: physics-informed neural networks trained with particle swarm optimization, arXiv:2202.01943 (2022) DOI: https://doi.org/10.48550/arXiv.2202.01943.
B. Lu, C. Moya and G. Lin, NSGA-PINN: a multi-objective optimization method for physics-informed neural network training, Algorithms, 16 (4) (2023) 194.
M. A. Nabian, R. J. Gladstone and H. Meidani, Efficient training of physics-informed neural networks via importance sampling, Computer-Aided Civil and Infrastructure Engineering, 36 (8) (2021) 962–977.
Z. Yang, Z. Qiu and D. Fu, DMIS: dynamic mesh-based importance sampling for training physics-informed neural networks, arXiv:2211.13944 (2022) DOI: https://doi.org/10.48550/arXiv.2211.13944.
D. He et al., Learning physics-informed neural networks without stacked back-propagation, arXiv:2202.09340 (2022) DOI: https://doi.org/10.48550/arXiv.2202.09340.
S. Markidis, The old and the new: can physics-informed deep-learning replace traditional linear solvers?, Frontiers in Big Data, 4 (2021).
A. D. Jagtap, E. Kharazmi and G. E. Karniadakis, Conservative physics-informed neural networks on discrete domains for conservation laws: applications to forward and inverse problems, Computer Methods in Applied Mechanics and Engineering, 365 (2020) 113028.
X. Meng et al., PPINN: parareal physics-informed neural network for time-dependent PDEs, Computer Methods in Applied Mechanics and Engineering, 370 (2020) 113250.
A. D. Jagtap and G. E. Karniadakis, Extended physics-informed neural networks (XPINNs): a generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations, Communications in Computational Physics, 28 (5) (2020) https://doi.org/10.4208/cicp.oa-2020-0164.
P. Stiller et al., Large-scale neural solvers for partial differential equations, Driving Scientific and Engineering Discoveries Through the Convergence of HPC, Big Data and AI., Cham: Springer International Publishing (2020).
Z. Hu et al., Augmented Physics-Informed Neural Networks (APINNs): a gating network-based soft domain decomposition methodology, arXiv:2211.08939 (2022) DOI: https://doi.org/10.48550/arXiv.2211.08939.
S. Wang, S. Sankaran and P. Perdilcaris, Respecting causality is all you need for training physics-informed neural networks, arXiv:2203.07404 (2022) DOI: https://doi.org/10.48550/arXiv.2203.07404.
A. Daw et al., Mitigating propagation failures in PINNs using evolutionary sampling, arXiv:2207.02338 (2022) DOI: https://doi.org/10.48550/arXiv.2207.02338.
A. E. Eiben and J. E. Smith, Introduction to Evolutionary Computing, Springer-Verlag Berlin Heidelberg, Germany (2015).
J. Guo, H. Wang and C. Hou, A novel adaptive causal sampling method for physics-informed neural networks, arXiv:2210.12914 (2022) DOI: https://doi.org/10.48550/arXiv.2210.12914.
R. Mattey and S. Ghosh, A novel sequential method to train physics informed neural networks for allen cahn and cahn hilliard equations, Computer Methods in Applied Mechanics and Engineering, 390 (2022) 114474.
M. Penwarden et al., A unified scalable framework for causal sweeping strategies for physics-informed neural networks (PINNs) and their temporal decompositions, arXiv:2302.14227 (2023) DOI: https://doi.org/10.48550/arXiv.2302.14227.
A. F. Psaros, K. Kawaguchi and G. E. Karniadakis, Meta-learning PINN loss functions, Journal of Computational Physics, 458 (2022) 111121.
S. Goswami et al., Transfer learning enhanced physics informed neural network for phase-field modeling of fracture, Theoretical and Applied Fracture Mechanics, 106 (2020) 102447.
B. Bahmani and W. Sun, Training multi-objective/multi-task collocation physics-informed neural network with student/teachers transfer learnings, arXiv:2107.11496 (2021) DOI: https://doi.org/10.48550/arXiv.2107.11496.
S. Desai et al., One-shot transfer learning of physics-informed neural networks, arXiv:2110.11286 (2021) DOI: https://doi.org/10.48550/arXiv.2110.11286.
C. Xu et al., Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios, Computer Methods in Applied Mechanics and Engineering, 405 (2023) 115852.
M. Penwarden et al., A metalearning approach for physics-informed neural networks (PINNs): application to parameterized PDEs, Journal of Computational Physics, 477 (2023) 111912.
W. Chen et al., Physics-informed machine learning for reduced-order modeling of nonlinear problems, Journal of Computational Physics, 446 (2021) 110666.
K. Haitsiukevich and A. Ilin, Improved training of physics-informed neural networks with model ensembles, arXiv:2204.05108 (2022) DOI: https://doi.org/10.48550/arXiv.2204.05108.
J. Kim et al., DPM: a novel training method for physics-informed neural networks in extrapolation, Proceedings of the AAAI Conference on Artificial Intelligence, 35 (9) (2021) 8146–8154.
K. Linka et al., Bayesian physics informed neural networks for real-world nonlinear dynamical systems, Computer Methods in Applied Mechanics and Engineering, 402 (2022) 115346.
J. Sirignano and K. Spiliopoulos, DGM: a deep learning algorithm for solving partial differential equations, Journal of Computational Physics, 375 (2018) 1339–1364.
B. Chudomelka et al., Deep neural network for solving differential equations motivated by legendre-galerkin approximation, arXiv:2010.12975 (2020) DOI: https://doi.org/10.48550/arXiv.2010.12975.
J. Choi, N. Kim and Y. Hong, Unsupervised legendre-galerkin neural network for solving partial differential equations, IEEE Access, 11 (2023) 23433–23446.
S. Amini Niaki et al., Physics-informed neural network for modelling the thermochemical curing process of composite-tool systems during manufacture, Computer Methods in Applied Mechanics and Engineering, 384 (2021) 113959.
H. K. Lee and S. In, Neural algorithm for solving differential equations, Journal of Computational Physics, 91 (1990) 110–131.
D. C. U. Psichogios and H. Lyle, A hybrid neural network-first principles approach to process modeling, AIChE J., 38 (1992) 1499–1511.
M. W. M. G. Dissanayake and N. Phan-Thien, Neural-network-based approximations for solving partial differential equations, Communications in Numerical Methods in Engineering, 10 (1994) 195–201.
I. E. Lagaris, A. Likas and D. I. Fotiadis, Artificial neural networks for solving ordinary and partial differential equations, IEEE Transactions on Neural Networks, 9 (1998) 987–1000.
P. Ramuhalli, L. Udpa and S. S. Udpa, Finite-element neural networks for solving differential equations, IEEE Transactions on Neural Networks, 16 (6) (2005) 1381–1392.
A. S. B. Malek, Numerical solution for high order differential equations using a hybrid neural network—optimization method, Applied Mathematics and Computation, 183 (2006) 260–271.
R. M. Shekari Beidokhti, Solving initial-boundary value problems for systems of partial differential equations using neural networks and optimization techniques, Journal of the Franklin Institute, 346 (2009) 898–913.
M. Kumar and N. Yadav, Multilayer perceptrons and radial basis function neural network methods for the solution of differential equations: a survey, Computers & Mathematics with Applications, 62 (2011) 3796–3811.
A. Griewank, On Automatic differentiation and algorithmic linearization, Pesquisa Operacional, 34 (3) (2014) 621–645.
M. B. Abadi et al., TensorFlow: a system for large-scale machine learning, Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), Savannah, GA, USA (2016).
M. P. Raissi, Paris karniadakis, george em, physics informed deep learning (part I): data-driven solutions of nonlinear partial differential equations, arXiv:1711.10561 (2017) DOI: https://doi.org/10.48550/arXiv.1711.10561.
M. Raissi, P. Perdikaris and G. E. Karniadakis, Physics informed deep learning (part II): data-driven discovery of nonlinear partial differential equations, arXiv:1711.10566 (2017) DOI: 10.48550.arXiv.1711.10566.
Acknowledgments
This research was supported by Korea Institute of Marine Science & Technology Promotion (KIMST) funded by the Ministry of Oceans and Fisheries (20220210).
Author information
Authors and Affiliations
Corresponding author
Additional information
Ikhyun Ryu received a B.S. degree in Mechanical Engineering from Karlsruhe Institute of Technology, Karlsruhe, Germany, in 2019. He then received a M.S. degree in Mechanical Engineering from Hanyang University, Seoul, South Korea, in 2022. He is now a Senior Research Engineer in PIDOTECH Inc. since 2022. His research interests include heat transfer, multidisciplinary design optimization, and Physics-Informed Neural Networks.
Gyu-Byung Park received a B.S. degree in Mechanical Engineering from Hanyang University, South Korea, in 2006. He then received a Ph.D. degree in Mechanical Engineering from Hanyang University, South Korea, in 2016. He is now a Senior Research Engineer in PIDOTECH Inc. since 2016. His research interests include Artificial Intelligence, Deep Learning, Design Optimization and Physics-Informed Neural Networks.
Yongbin Lee received a B.S. degree in Mechanical Engineering from Hanyang University, Seoul, South Korea, in 2002. He then received a M.S. degree in Mechanical Engineering from Hanyang University, Seoul, South Korea, in 2004. He then received a Ph.D. degree in Mechanical Engineering from Hanyang University, Seoul, South Korea, in 2009. He is now a Senior Research Engineer in PIDOTECH since 2012. His research interests include Machine Learning, Metamodeling, Design of Experiments, and Design Optimization.
Dong-Hoon Choi graduated from Seoul National University with a B.S. in mechanical engineering in 1975. He then graduated from KAIST in 1977 with an M.S. in Mechanical Engineering. He earned his Ph.D. in mechanical engineering from the University of Wisconsin-Madison. From 1986 to 2018, he served as a Professor of Mechanical Engineering at Hanyang University. He has been the CEO of PIDOTECH Inc. since 2003. His research interest includes AI-aided design optimization, multidisciplinary design optimization, surrogate-based design optimization, and AI applications for simulation and engineering design.
Rights and permissions
About this article
Cite this article
Ryu, I., Park, GB., Lee, Y. et al. Physics-informed neural network for engineers: a review from an implementation aspect. J Mech Sci Technol 38, 3499–3519 (2024). https://doi.org/10.1007/s12206-024-0624-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12206-024-0624-9