Abstract
The advent of technology including big data has allowed machine learning technology to strengthen its place in solving different science and engineering complex problems. Conventional deep machine learning algorithms work as a black box while dealing with various complex physics-driven problems. This problem can be reduced by integrating the physical laws with machine learning algorithms to ensure the developed models are complied with the physics and are potentially more explainable. This physics-informed machine learning (PIML) approach allows the integration of physical laws in the form of PDEs into the loss function of the neural network, hence, constraining the training of the complex problems based on both the physical, experimental, and mathematical boundaries. This, hence, allows the development of a more general predictive model for different science, engineering, and optimization tasks. Considering such advancements in the machine learning domain, this review presents the systematic progress in the development of integrating physics into the neural networks and recent applications in solving various forward and inverse problems in science and engineering. This paper can serve as a reference for the researchers, developers, and users to get all information they need before developing, implementing, and deploying AI models and smart systems that are equipped with the PIML methodology. It highlights the benefits and points out its limitations and recommendations for further development. The review also compares the traditional data-driven machine learning and PIML approach in dealing with the physics of complex problems. In general, the PIML has been found to provide consistent results with the exact solutions and physical nature of the system. However, similar to other AI system development, a more robust and complex AI algorithm requires more computational power which is also the case in PIML development and implementation. It should be noted that different terminologies such as physics-informed neural networks (PINN), science-informed neural networks, physics-inspired neural networks, and physics-constrained neural networks have been used in the literature that describes the very similar concept of integrating physical laws with machine intelligence. For consistency, we use the PIML term throughout this paper which covers all listed terminologies in this regard.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Adalsteinsson D, Sethian JA (1995) A fast level set method for propagating interfaces. J Comput Phys 118(2):269–277
Adeli H (2001) Neural networks in civil engineering: 1989–2000. Comput Aided Civ Infrastruct Eng 16(2):126–142
Adie J, Yang J, Zhang M, See S (2018) Deep learning for computational science and engineering. In: GPU technology conference No. S8242
Ahmad H, Seadawy AR, Khan TA (2020) Numerical solution of Korteweg–de Vries–Burgers equation by the modified variational iteration algorithm-II arising in shallow water waves. Phys Scr 95(4):045210
Almajid MM, Abu-Alsaud MO (2020) Prediction of fluid flow in porous media using physics informed neural networks. In: Abu Dhabi international petroleum exhibition & conference. OnePetro, November 2020
Alvino C, Unal G, Slabaugh G, Peny B, Fang T (2007) Efficient segmentation based on Eikonal and diffusion equations. Int J Comput Math 84(9):1309–1324
Amezquita-Sanchez JP, Valtierra-Rodriguez M, Aldwaik M, Adeli H (2016) Neurocomputing in civil infrastructure. Sci Iran 23(6):2417–2428
Araz JY, Criado JC, Spannowsky M (2021) Elvet—a neural network-based differential equation and variational problem solver. arXiv preprint. arXiv:2103.14575
Arzani A, Wang JX, D’Souza RM (2021) Uncovering near-wall blood flow from sparse data with physics-informed neural networks. Phys Fluids 33(7):071905
Ayati AH, Haghighi A, Ghafouri HR (2022) Machine learning-assisted model for leak detection in water distribution networks using hydraulic transient flows. J Water Resour Plan Manag 148(2):04021104
Aydin H, Akin S, Senturk E (2020) A proxy model for determining reservoir pressure and temperature for geothermal wells. Geothermics 88:101916
Ball JE, Anderson DT, Chan CS Sr (2017) Comprehensive survey of deep learning in remote sensing: theories, tools, and challenges for the community. J Appl Remote Sens 11(4):042609
Baydin AG, Pearlmutter BA, Radul AA, Siskind JM (2018) Automatic differentiation in machine learning: a survey. J Mach Learn Res 18:1–43
Behler J, Reuter K, Scheffler M (2008) Nonadiabatic effects in the dissociation of oxygen molecules at the Al (111) surface. Phys Rev B 77(11):115421
Bengio Y (2000) Gradient-based optimization of hyperparameters. Neural Comput 12(8):1889–1900
Bergen KJ, Johnson PA, Maarten V, Beroza GC (2019) Machine learning for data-driven discovery in solid Earth geoscience. Science 363(6433):eaau0323
Bhaskaran PK, Rajesh Kumar R, Barman R, Muthalagu R (2010) A new approach for deriving temperature and salinity fields in the Indian Ocean using artificial neural networks. J Mar Sci Technol 15(2):160–175
Biot MA (1941) General theory of three-dimensional consolidation. J Appl Phys 12(2):155–164
Biot MA, Willis DG (1957) The elastic coefficients of the theory of consolidation. J Appl Mech 24(4):594–601
Bisdom K, Bertotti G, Nick HM (2016) The impact of in-situ stress and outcrop-based fracture geometry on hydraulic aperture and upscaled permeability in fractured reservoirs. Tectonophysics 690:63–75
Blank TB, Brown SD, Calhoun AW, Doren DJ (1995) Neural network models of potential energy surfaces. J Chem Phys 103(10):4129–4137
Brown DF, Gibbs MN, Clary DC (1996) Combining ab initio computations, neural networks, and diffusion Monte Carlo: an efficient method to treat weakly bound molecules. J Chem Phys 105(17):7597–7604
Browne M, Castelle B, Strauss D, Tomlinson R, Blumenstein M, Lane C (2007) Near-shore swell estimation from a global wind-wave model: spectral process, linear, and artificial neural network models. Coast Eng 54(5):445–460
Buckley SE, Leverett M (1942) Mechanism of fluid displacement in sands. Trans AIME 146(01):107–116
Byrd RH, Lu P, Nocedal J, Zhu C (1995) A limited memory algorithm for bound constrained optimization. SIAM J Sci Comput 16(5):1190–1208
Cai S, Li H, Zheng F, Kong F, Dao M, Karniadakis GE, Suresh S (2021a) Artificial intelligence velocimetry and microaneurysm-on-a-chip for three-dimensional analysis of blood flow in physiology and disease. Proc Natl Acad Sci U S A 118(13):e2100697118
Cai S, Wang Z, Wang S, Perdikaris P, Karniadakis GE (2021b) Physics-informed neural networks for heat transfer problems. J Heat Transf 143(6):060801
Cai S, Mao Z, Wang Z, Yin M, Karniadakis GE (2022) Physics-informed neural networks (PINNs) for fluid mechanics: a review. Acta Mech Sin 37:1727–1738
Carbogno C, Behler J, Groß A, Reuter K (2008) Fingerprints for spin-selection rules in the interaction dynamics of O2 at Al (111). Phys Rev Lett 101(9):096104
Carleo G, Troyer M (2017) Solving the quantum many-body problem with artificial neural networks. Science 355(6325):602–606
Chan S, Elsheikh AH (2017) Parametrization and generation of geological models with generative adversarial networks. arXiv preprint. arXiv:1708.01810
Chaurasia V, Pal S (2014) Data mining approach to detect heart diseases. Int J Adv Comput Sci Inf Technol (IJACSIT) 2:56–66
Chen M, Hao Y, Hwang K, Wang L, Wang L (2017) Disease prediction by machine learning over big data from healthcare communities. IEEE Access 5:8869–8879
Chen RT, Rubanova Y, Bettencourt J, Duvenaud DK (2018) Neural ordinary differential equations. In: Advances in neural information processing systems 31 (NeurIPS 2018)
Chen F, Sondak D, Protopapas P, Mattheakis M, Liu S, Agarwal D, Di Giovanni M (2020) NeuroDiffEq: a Python package for solving differential equations with neural networks. J Open Source Softw 5(46):1931
Cho C, Kim K, Park J, Cho YK (2018) Data-driven monitoring system for preventing the collapse of scaffolding structures. J Constr Eng Manag 144(8):04018077
Choo J, Lee S (2018) Enriched Galerkin finite elements for coupled poromechanics with local mass conservation. Comput Methods Appl Mech Eng 341:311–332
Crnkovic-Friis L, Erlandson M (2015) Geology driven EUR prediction using deep learning. In: SPE annual technical conference and exhibition. OnePetro, September 2015
Cuomo S, Di Cola VS, Giampaolo F, Rozza G, Raissi M, Piccialli F (2022) Scientific machine learning through physics-informed neural networks: where we are and what's next. arXiv preprint. arXiv:2201.05624
Dallora AL, Eivazzadeh S, Mendes E, Berglund J, Anderberg P (2017) Machine learning and microsimulation techniques on the prognosis of dementia: A systematic literature review. PLoS ONE 12(6):e0179804
D’Cruz J, Jadhav A, Dighe A, Chavan V, Chaudhari J (2016) Detection of lung cancer using backpropagation neural networks and genetic algorithm. Comput Technol Appl 6:823–827
Davi C, Braga-Neto U (2022) PSO-PINN: physics-informed neural networks trained with particle swarm optimization. arXiv preprint. arXiv:2202.01943
De Ryck T, Mishra S (2021) Error analysis for physics informed neural networks (PINNs) approximating Kolmogorov PDEs. arXiv preprint. arXiv:2106.14473
Dissanayake MWMG, Phan-Thien N (1994) Neural-network-based approximations for solving partial differential equations. Commun Numer Methods Eng 10(3):195–201
Dwivedi V, Srinivasan B (2020) Physics informed extreme learning machine (PIELM)—a rapid method for the numerical solution of partial differential equations. Neurocomputing 391:96–118
Eivazi H, Tahani M, Schlatter P, Vinuesa R (2021) Physics-informed neural networks for solving Reynolds-averaged Navier–Stokes equations. arXiv preprint. arXiv:2107.10711
Esfe MH, Saedodin S, Sina N, Afrand M, Rostami S (2015a) Designing an artificial neural network to predict thermal conductivity and dynamic viscosity of ferromagnetic nanofluid. Int Commun Heat Mass Transf 68:50–57
Esfe MH, Rostamian H, Afrand M, Karimipour A, Hassani M (2015b) Modeling and estimation of thermal conductivity of MgO–water/EG (60:40) by artificial neural network and correlation. Int Commun Heat Mass Transf 68:98–103
Fahle S, Prinz C, Kuhlenkötter B (2020) Systematic review on machine learning (ML) methods for manufacturing processes–Identifying artificial intelligence (AI) methods for field application. Procedia CIRP 93:413–418
Fletcher R (2013) Practical methods of optimization. Wiley, Hoboken
Fraces CG, Papaioannou A, Tchelepi H (2020) Physics informed deep learning for transport in porous media. Buckley Leverett Problem. arXiv preprint. arXiv:2001.05172
Frank M, Drikakis D, Charissis V (2020) Machine-learning methods for computational science and engineering. Computation 8(1):15
Fuks O, Tchelepi HA (2020) Limitations of physics informed machine learning for nonlinear two-phase transport in porous media. J Mach Learn Model Comput 1(1):10
Gao H, Sun L, Wang JX (2021) PhyGeoNet: physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain. J Comput Phys 428:110079
Gardner JR, Pleiss G, Bindel D, Weinberger KQ, Wilson AG (2018) Gpytorch: blackbox matrix–matrix Gaussian process inference with GPU acceleration. arXiv preprint. arXiv:1809.11165
Garg A, Mago V (2021) Role of machine learning in medical research: a survey. Comput Sci Rev 40:100370
Geneva N, Zabaras N (2020) Modeling the dynamics of PDE systems with physics-constrained deep auto-regressive networks. J Comput Phys 403:109056
Ghaboussi J, Garrett JH, Wu X (1990) Material modeling with neural networks. In: Proceedings of the international conference on numerical methods in engineering: theory and applications, Swansea, UK, January 1990, pp 701–717
Ghaboussi J, Garrett JH Jr, Wu X (1991) Knowledge-based modeling of material behavior with neural networks. J Eng Mech 117(1):132–153
Ghahramani Z (2015) Probabilistic machine learning and artificial intelligence. Nature 521(7553):452–459
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014). Generative adversarial nets. In: Advances in Neural Information Processing Systems 27 (NIPS 2014)
Grechka V, De La Pena A, Schisselé-Rebel E, Auger E, Roux PF (2015) Relative location of microseismicity. Geophysics 80(6):WC1–WC9
Grimm R, Behrens T, Märker M, Elsenbeer H (2008) Soil organic carbon concentrations and stocks on Barro Colorado Island—Digital soil mapping using Random Forests analysis. Geoderma 146(1–2):102–113
Gu J, Zhang Y, Dong H (2018) Dynamic behaviors of interaction solutions of (3 + 1)-dimensional shallow water wave equation. Comput Math Appl 76(6):1408–1419
Guo R, Li M, Yang F, Xu S, Abubakar A (2019) First arrival traveltime tomography using supervised descent learning technique. Inverse Prob 35(10):105008
Guo Y, Cao X, Liu B, Gao M (2020) Solving partial differential equations using deep learning and physical constraints. Appl Sci 10(17):5917
Gupta S, Li L (2022) The potential of machine learning for enhancing CO2 sequestration, storage, transportation, and utilization-based processes: a brief perspective. JOM 74:414–428
Haga JB, Osnes H, Langtangen HP (2012) On the causes of pressure oscillations in low-permeable and low-compressible porous media. Int J Numer Anal Meth Geomech 36(12):1507–1522
Haghighat E, Juanes R (2021) Sciann: a keras/tensorflow wrapper for scientific computations and physics-informed deep learning using artificial neural networks. Comput Methods Appl Mech Eng 373:113552
Hansen K, Montavon G, Biegler F, Fazli S, Rupp M, Scheffler M, Von Lilienfeld OA, Tkatchenko A, Muller KR (2013) Assessment and validation of machine learning methods for predicting molecular atomization energies. J Chem Theory Comput 9(8):3404–3419
He Q, Tartakovsky AM (2021) Physics-Informed neural network method for forward and backward advection-dispersion equations. Water Resour Res 57(7):e2020WR029479
He Q, Barajas-Solano D, Tartakovsky G, Tartakovsky AM (2020) Physics-informed neural networks for multiphysics data assimilation with application to subsurface transport. Adv Water Resour 141:103610
Hengl T, Mendes de Jesus J, Heuvelink GB, Ruiperez Gonzalez M, Kilibarda M, Blagotić A, Shangguan W, Wright MN, Geng X, Bauer-Marschallinger B, Guevara MA (2017) SoilGrids250m: global gridded soil information based on machine learning. PLoS ONE 12(2):e0169748
Hennigh O, Narasimhan S, Nabian MA, Subramaniam A, Tangsali K, Fang Z, Rietmann M, Byeon W, Choudhry S (2021) NVIDIA SimNet™: an AI-accelerated multi-physics simulation framework. In: International conference on computational science, June 2021. Springer, Cham, pp 447–461
Hsieh WW (2009) Machine learning methods in the environmental sciences: neural networks and kernels. Cambridge University Press, Cambridge
Hsieh YA, Tsai YJ (2020) Machine learning for crack detection: review and model performance comparison. J Comput Civ Eng 34(5):04020038
Iakovlev V, Heinonen M, Lähdesmäki H (2020) Learning continuous-time PDEs from sparse data with graph neural networks. arXiv preprint. arXiv:2006.08956
Ibarra-Berastegi G, Saénz J, Esnaola G, Ezcurra A, Ulazia A (2015) Short-term forecasting of the wave energy flux: analogues, random forests, and physics-based models. Ocean Eng 104:530–539
Ippolito M, Ferguson J, Jenson F (2021) Improving facies prediction by combining supervised and unsupervised learning methods. J Petrol Sci Eng 200:108300
Islam M, Thakur MSH, Mojumder S, Hasan MN (2021) Extraction of material properties through multi-fidelity deep learning from molecular dynamics simulation. Comput Mater Sci 188:110187
Jagtap AD, Karniadakis GE (2020) Extended physics-informed neural networks (XPINNs): a generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. Commun Comput Phys 28(5):2002–2041
Jagtap AD, Kawaguchi K, Karniadakis GE (2020a) Adaptive activation functions accelerate convergence in deep and physics-informed neural networks. J Comput Phys 404:109136
Jagtap AD, Kawaguchi K, Em Karniadakis G (2020b) Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks. Proc R Soc A 476(2239):20200334
Jagtap AD, Kharazmi E, Karniadakis GE (2020c) Conservative physics-informed neural networks on discrete domains for conservation laws: applications to forward and inverse problems. Comput Methods Appl Mech Eng 365:113028
James SC, Zhang Y, O’Donncha F (2018) A machine learning framework to forecast wave conditions. Coast Eng 137:1–10
Jiang B, Guo H (2013) Permutation invariant polynomial neural network approach to fitting potential energy surfaces. J Chem Phys 139(5):054112
Juanes R, Jha B, Hager BH, Shaw JH, Plesch A, Astiz L, Dieterich JH, Frohlich C (2016) Were the May 2012 Emilia-Romagna earthquakes induced? A coupled flow-geomechanics modeling assessment. Geophys Res Lett 43(13):6891–6897
Kadeethum T, Jørgensen TM, Nick HM (2020) Physics-informed neural networks for solving nonlinear diffusivity and Biot’s equations. PLoS ONE 15(5):e0232683
Karniadakis GE, Kevrekidis IG, Lu L, Perdikaris P, Wang S, Yang L (2021) Physics-informed machine learning. Nat Rev Phys 3(6):422–440
Karpatne A, Atluri G, Faghmous JH, Steinbach M, Banerjee A, Ganguly A, Shekhar S, Samatova N, Kumar V (2017) Theory-guided data science: a new paradigm for scientific discovery from data. IEEE Trans Knowl Data Eng 29(10):2318–2331
Karpatne A, Ebert-Uphoff I, Ravela S, Babaie HA, Kumar V (2018) Machine learning for the geosciences: challenges and opportunities. IEEE Trans Knowl Data Eng 31(8):1544–1554
Kauwe SK, Graser J, Vazquez A, Sparks TD (2018) Machine learning prediction of heat capacity for solid inorganics. Integr Mater Manuf Innov 7(2):43–51
Kennedy J, Eberhart R (1995) Particle swarm optimization. In Proceedings of ICNN'95-international conference on neural networks, vol 4, November 1995. IEEE, Piscataway, pp 1942–1948
Kharazmi E, Zhang Z, Karniadakis GE (2019) Variational physics-informed neural networks for solving partial differential equations. arXiv preprint. arXiv:1912.00873
Kharazmi E, Zhang Z, Karniadakis GE (2021) hp-VPINNs: Variational physics-informed neural networks with domain decomposition. Comput Methods Appl Mech Eng 374:113547
Kim D (2019) A modified PML acoustic wave equation. Symmetry 11(2):177
Kim SK, Ames S, Lee J, Zhang C, Wilson AC, Williams D (2017a) Massive scale deep learning for detecting extreme climate events. In: 7th International workshop on clmate informatics
Kim S, Hong S, Joh M, Song SK (2017b) Deeprain: ConvLSTM network for precipitation prediction using multichannel radar data. arXiv preprint. arXiv:1711.02316
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint. arXiv:1412.6980
Kissas G, Yang Y, Hwuang E, Witschey WR, Detre JA, Perdikaris P (2020) Machine learning in cardiovascular flows modeling: predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks. Comput Methods Appl Mech Eng 358:112623
Kollmannsberger S, Dangella D, Jokeit M, Herrmann L (2021) Deep learning in computational mechanics. Springer, Cham
Kong Q, Allen RM, Schreier L, Kwon YW (2016) MyShake: a smartphone seismic network for earthquake early warning and beyond. Sci Adv 2(2):e1501055
Koryagin A, Khudorozkov R, Tsimfer S (2019) PyDEns: a Python framework for solving differential equations with neural networks. arXiv preprint. arXiv:1909.11544
Krishnaiah V, Narsimha G, Chandra NS (2013) Diagnosis of lung cancer prediction system using data mining classification techniques. Int J Comput Sci Inf Technol 4(1):39–45
Krishnapriyan A, Gholami A, Zhe S, Kirby R, Mahoney MW (2021) Characterizing possible failure modes in physics-informed neural networks. Adv Neural Inf Process Syst 34:26548–26560
Kumar Y, Chakraborty S (2021) GrADE: a graph based data-driven solver for time-dependent nonlinear partial differential equations. arXiv preprint. arXiv:2108.10639.
Kumar M, Yadav N (2011) Multilayer perceptrons and radial basis function neural network methods for the solution of differential equations: a survey. Comput Math Appl 62(10):3796–3811
Kumar A, Murali A, Priyadarshan A (2020) Subsurface velocity profiling by application of physics informed neural networks. In: Abu Dhabi international petroleum exhibition & conference. OnePetro, November 2020
Kuruvilla J, Gunavathi K (2014) Lung cancer classification using neural networks for CT images. Comput Methods Programs Biomed 113(1):202–209
Kutz JN (2017) Deep learning in fluid dynamics. J Fluid Mech 814:1–4
Kylasa S, Roosta F, Mahoney MW, Grama A (2019) GPU accelerated sub-sampled Newton’s method for convex classification problems. In: Proceedings of the 2019 SIAM international conference on data mining, May 2019. Society for Industrial and Applied Mathematics, Philadelphia, pp 702–710
Lagaris IE, Likas A, Fotiadis DI (1998) Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans Neural Netw 9(5):987–1000
Lagaris IE, Likas AC, Papageorgiou DG (2000) Neural-network methods for boundary value problems with irregular boundaries. IEEE Trans Neural Netw 11(5):1041–1049
Le HM, Raff LM (2010) Molecular dynamics investigation of the bimolecular reaction BeH+ H2→ BeH2+ H on an ab initio potential-energy surface obtained using neural network methods with both potential and gradient accuracy determination. J Phys Chem A 114(1):45–53
Lee H, Kang IS (1990) Neural algorithm for solving differential equations. J Comput Phys 91(1):110–131
Lee S, Wheeler MF, Wick T (2016) Pressure and fluid-driven fracture propagation in porous media using an adaptive finite element phase field model. Comput Methods Appl Mech Eng 305:111–132
Li J, Jiang B, Guo H (2013) Permutation invariant polynomial neural network approach to fitting potential energy surfaces. II. Four-atom systems. J Chem Phys 139(20):204103
Li J, Feng Z, Schuster G (2017) Wave-equation dispersion inversion. Geophys J Int 208(3):1567–1578
Li Z, Zheng H, Kovachki N, Jin D, Chen H, Liu B, Azizzadenesheli K, Anandkumar A (2021a) Physics-informed neural operator for learning partial differential equations. arXiv preprint. arXiv:2111.03794
Li X, Zhang L, Khan F, Han Z (2021b) A data-driven corrosion prediction model to support digitization of subsea operations. Process Saf Environ Prot 153:413–421
Liang Y, Liao L, Guo Y (2019) A big data study: correlations between EUR and petrophysics/engineering/production parameters in shale formations by data regression and interpolation analysis. In: SPE hydraulic fracturing technology conference and exhibition. OnePetro, January 2019
Liao Y, Ming P (2019) Deep Nitsche method: deep Ritz method with essential boundary conditions. arXiv preprint. arXiv:1912.01309
Liao L, Zeng Y, Liang Y, Zhang H (2020) Data mining: a novel strategy for production forecast in tight hydrocarbon resource in Canada by random forest analysis. In: International petroleum technology conference. OnePetro, January 2020
Liu J, Tavener S, Wang Z (2018) Lowest-order weak Galerkin finite element method for Darcy flow on convex polygonal meshes. SIAM J Sci Comput 40(5):B1229–B1252
Liu M, Liang L, Sun W (2020) A generic physics-informed neural network-based constitutive model for soft biological tissues. Comput Methods Appl Mech Eng 372:113402
Liu HH, Zhang J, Liang F, Temizel C, Basri MA, Mesdour R (2021) Incorporation of physics into machine learning for production prediction from unconventional reservoirs: a brief review of the gray-box approach. SPE Reservoir Evaluation & Engineering, The Hague, pp 1–12
Lorenz S, Groß A, Scheffler M (2004) Representing high-dimensional potential-energy surfaces for reactions at surfaces by neural networks. Chem Phys Lett 395(4–6):210–215
Lorenz S, Scheffler M, Gross A (2006) Descriptions of surface chemical reactions using a neural network representation of the potential-energy surface. Phys Rev B 73(11):115431
Lou Q, Meng X, Karniadakis GE (2020) Physics-informed neural networks for solving forward and inverse flow problems via the Boltzmann–BGK formulation. arXiv preprint. arXiv:2010.09147
Lu L, Meng X, Mao Z, Karniadakis GE (2021a) DeepXDE: a deep learning library for solving differential equations. SIAM Rev 63(1):208–228
Lu L, Jin P, Pang G, Zhang Z, Karniadakis GE (2021b) Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nat Mach Intell 3(3):218–229
Ludwig J, Vlachos DG (2007) Ab initio molecular dynamics of hydrogen dissociation on metal surfaces using neural networks and novelty sampling. J Chem Phys 127(15):154716
Luo G, Tian Y, Bychina M, Ehlig-Economides C (2018, September) Production optimization using machine learning in Bakken shale. In: Unconventional resources technology conference, Houston, TX, 23–25 July 2018. Society of Exploration Geophysicists, American Association of Petroleum Geologists, Society of Petroleum Engineers, pp 2174–2197
Makarynskyy O (2004) Improving wave predictions with artificial neural networks. Ocean Eng 31(5–6):709–724
Mall S, Chakraverty S (2016) Application of Legendre neural network for solving ordinary differential equations. Appl Soft Comput 43:347–356
Malladi R, Sethian JA (1996) A unified approach to noise removal, image enhancement, and shape recovery. IEEE Trans Image Process 5(11):1554–1568
Manzhos S, Wang X, Dawes R, Carrington T (2006) A nested molecule-independent neural network approach for high-quality potential fits. J Phys Chem A 110(16):5295–5304
Mao Z, Jagtap AD, Karniadakis GE (2020) Physics-informed neural networks for high-speed flows. Comput Methods Appl Mech Eng 360:112789
Markidis S (2021) The old and the new: can physics-informed deep-learning replace traditional linear solvers? Front Big Data. https://doi.org/10.3389/fdata.2021.669097
Masson YJ, Pride SR (2010) Finite-difference modeling of Biot’s poroelastic equations across all frequencies. Geophysics 75(2):N33–N41
Matsuoka D, Nakano M, Sugiyama D, Uchida S (2017) Detecting precursors of tropical cyclone using deep neural networks. In: 7th International workshop on clmate informatics
Mattey R, Ghosh S (2022) A novel sequential method to train physics informed neural networks for Allen Cahn and Cahn Hilliard equations. Comput Methods Appl Mech Eng 390:114474
Mazumder RK, Salman AM, Li Y (2021) Failure risk analysis of pipelines using data-driven machine learning algorithms. Struct Saf 89:102047
McClenny LD, Haile MA, Braga-Neto UM (2021) TensorDiffEq: scalable multi-GPU forward and inverse solvers for physics informed neural networks. arXiv preprint. arXiv:2103.16034
McFall KS, Mahan JR (2009) Artificial neural network method for solution of boundary value problems with exact satisfaction of arbitrary boundary conditions. IEEE Trans Neural Networks 20(8):1221–1233
Meade AJ Jr, Fernandez AA (1994a) The numerical solution of linear ordinary differential equations by feedforward neural networks. Math Comput Model 19(12):1–25
Meade AJ Jr, Fernandez AA (1994b) Solution of nonlinear ordinary differential equations by feedforward neural networks. Math Comput Model 20(9):19–44
Meng X, Karniadakis GE (2020) A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems. J Comput Phys 401:109020
Meng X, Li Z, Zhang D, Karniadakis GE (2020) PPINN: parareal physics-informed neural network for time-dependent PDEs. Comput Methods Appl Mech Eng 370:113250
Mishra S, Molinaro R (2021) Physics informed neural networks for simulating radiative transfer. J Quant Spectrosc Radiat Transfer 270:107705
Montavon G, Samek W, Müller KR (2018) Methods for interpreting and understanding deep neural networks. Digit Signal Process 73:1–15
Moseley B, Markham A, Nissen-Meyer T (2018) Fast approximate simulation of seismic waves with deep learning. arXiv preprint. arXiv:1807.06873
Mouatadid S, Easterbrook S, Erler A (2017) Non-uniform spatial downscaling of climate variables. In: 7th International workshop on clmate informatics
Muther T, Syed FI, Dahaghi AK, Negahban S (2022a) Socio-inspired multi-cohort intelligence and teaching-learning-based optimization for hydraulic fracturing parameters design in tight formations. J Energy Resour Technol 144(7):073201
Muther T, Syed FI, Lancaster AT, Salsabila FD, Dahaghi AK, Negahban S (2022b) Geothermal 4.0: AI-enabled geothermal reservoir development-current status, potentials, limitations, and ways forward. Geothermics 100:102348
Nick HM, Raoof A, Centler F, Thullner M, Regnier P (2013) Reactive dispersive contaminant transport in coastal aquifers: numerical simulation of a reactive Henry problem. J Contam Hydrol 145:90–104
Nordbotten JM (2014) Cell-centered finite volume discretizations for deformable porous media. Int J Numer Meth Eng 100(6):399–418
Ochoa LH, Niño LF, Vargas CA (2018) Fast magnitude determination using a single seismological station record implementing machine learning techniques. Geodesy Geodyn 9(1):34–41
Oldenburg J, Borowski F, Öner A, Schmitz KP, Stiehm M (2022) Geometry aware physics informed neural network surrogate for solving Navier–Stokes equation (GAPINN). Adv Model Simul Eng Sci 9:8
Paitz P, Gokhberg A, Fichtner A (2018) A neural network for noise correlation classification. Geophys J Int 212(2):1468–1474
Pang G, Lu L, Karniadakis GE (2019) fPINNs: fractional physics-informed neural networks. SIAM J Sci Comput 41(4):A2603–A2626
Papale D, Valentini R (2003) A new assessment of European forests carbon exchanges by eddy fluxes and artificial neural network spatialization. Glob Change Biol 9(4):525–535
Peng W, Zhang J, Zhou W, Zhao X, Yao W, Chen X (2021) IDRLnet: a physics-informed neural network library. arXiv preprint. arXiv:2107.04320
Pérez-Zárate D, Santoyo E, Acevedo-Anicasio A, Díaz-González L, García-López C (2019) Evaluation of artificial neural networks for the prediction of deep reservoir temperatures using the gas-phase composition of geothermal fluids. Comput Geosci 129:49–68
Perol T, Gharbi M, Denolle M (2018) Convolutional neural network for earthquake detection and location. Sci Adv 4(2):e1700578
Prudente FV, Neto JS (1998) The fitting of potential energy surfaces using neural networks. Application to the study of the photodissociation processes. Chem Phys Lett 287(5–6):585–589
Rachman A, Zhang T, Ratnayake RC (2021) Applications of machine learning in pipeline integrity management: a state-of-the-art review. Int J Press Vessels Pip 193:104471
Raff LM, Malshe M, Hagan M, Doughan DI, Rockley MG, Komanduri R (2005) Ab initio potential-energy surfaces for complex, multichannel systems using modified novelty sampling and feedforward neural networks. J Chem Phys 122(8):084104
Raissi M, Karniadakis GE (2018) Hidden physics models: machine learning of nonlinear partial differential equations. J Comput Phys 357:125–141
Raissi M, Perdikaris P, Karniadakis GE (2017) Physics informed deep learning (Part I): data-driven solutions of nonlinear partial differential equations. arXiv preprint arXiv:1711.10561
Raissi M, Perdikaris P, Karniadakis GE (2018) Numerical Gaussian processes for time-dependent and nonlinear partial differential equations. SIAM J Sci Comput 40(1):A172–A198
Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686–707
Raissi M, Yazdani A, Karniadakis GE (2020) Hidden fluid mechanics: learning velocity and pressure fields from flow visualizations. Science 367(6481):1026–1030
Razakh TM, Wang B, Jackson S, Kalia RK, Nakano A, Nomura KI, Vashishta P (2021) PND: physics-informed neural-network software for molecular dynamics applications. SoftwareX 15:100789
Reddy R, Nair RR (2013) The efficacy of support vector machines (SVM) in robust determination of earthquake early warning magnitudes in central Japan. J Earth Syst Sci 122(5):1423–1434
Reich Y (1997) Machine learning techniques for civil engineering problems. Comput Aided Civ Infrastruct Eng 12(4):295–310
Reichstein M, Camps-Valls G, Stevens B, Jung M, Denzler J, Carvalhais N (2019) Deep learning and process understanding for data-driven Earth system science. Nature 566(7743):195–204
Ren P, Rao C, Liu Y, Wang J, Sun H (2021) PhyCRNet: physics-informed convolutional-recurrent network for solving spatiotemporal PDEs. arXiv preprint. arXiv:2106.14103
Richardson, A., 2018. Seismic full-waveform inversion using deep learning tools and techniques. arXiv preprint arXiv:1801.07232.
Rocha Filho TMD, Oliveira ZT Jr, Malbouisson LAC, Gargano R, Soares Neto JJ (2003) The use of neural networks for fitting potential energy surfaces: a comparative case study for the H molecule. Int J Quantum Chem 95(3):281–288
Rodriguez-Torrado R, Ruiz P, Cueto-Felgueroso L, Green MC, Friesen T, Matringe S, Togelius J (2021) Physics-informed attention-based neural network for solving non-linear partial differential equations. arXiv preprint. arXiv:2105.07898
Rouy E, Tourin A (1992) A viscosity solutions approach to shape-from-shading. SIAM J Numer Anal 29(3):867–884
Runge J, Petoukhov V, Donges JF, Hlinka J, Jajcay N, Vejmelka M, Hartman D, Marwan N, Paluš M, Kurths J (2015) Identifying causal gateways and mediators in complex spatio-temporal systems. Nat Commun 6(1):1–10
Rupp M (2015) Machine learning for quantum mechanics in a nutshell. Int J Quantum Chem 115(16):1058–1073
Rupp M, Tkatchenko A, Müller KR, Von Lilienfeld OA (2012) Fast and accurate modeling of molecular atomization energies with machine learning. Phys Rev Lett 108(5):058301
Samokhin A (2017) On nonlinear superposition of the KdV–Burgers shock waves and the behavior of solitons in a layered medium. Differ Geom Appl 54:91–99
Sapitang M, Ridwan W, Faizal Kushiar K, Najah Ahmed A, El-Shafie A (2020) Machine learning application in reservoir water level forecasting for sustainable hydropower generation strategy. Sustainability 12(15):6121
Sattarin S, Muther T, Dahaghi AK, Negahban S (2021a) MicroPoreNet: complex and multilevels microporosity characterization of carbonate rocks through semisupervised CNN. In: 2021 IEEE International Conference on Imaging Systems and Techniques (IST), August 2021. IEEE, Piscataway, pp 1–5
Sattarin S, Muther T, Dahaghi AK, Negahban S, Bell B (2021b) GeoPixAI: from pixels to intelligent, unbiased and automated fast track subsurface characterization. In: 2021 IEEE International Conference on Imaging Systems and Techniques (IST), August 2021. IEEE, Piscataway, pp 1–5
Seghier MEAB, Keshtegar B, Tee KF, Zayed T, Abbassi R, Trung NT (2020) Prediction of maximum pitting corrosion depth in oil and gas pipelines. Eng Fail Anal 112:104505
Seo S, Mohegh A, Ban-Weiss G, Liu Y (2017) Graph convolutional autoencoder with recurrent neural networks for spatiotemporal forecasting. In: 7th International workshop on clmate informatics
Shahin MA (2015) A review of artificial intelligence applications in shallow foundations. Int J Geotech Eng 9(1):49–60
Shahin MA (2016) State-of-the-art review of some artificial intelligence applications in pile foundations. Geosci Front 7(1):33–44
Shahnas MH, Yuen DA, Pysklywec RN (2018) Inverse problems in geodynamics using machine learning algorithms. J Geophys Res Solid Earth 123(1):296–310
Shang Y, Wang F, Sun J (2022) Deep Petrov–Galerkin method for solving partial differential equations. arXiv preprint. arXiv:2201.12995
Sherman CS, Mellors RJ, Morris JP (2019) Subsurface monitoring via physics-informed deep neural network analysis of DAS. In: 53rd US rock mechanics/geomechanics symposium. OnePetro, June 2019
Shirvany Y, Hayati M, Moradian R (2009) Multilayer perceptron neural networks with novel unsupervised training method for numerical solution of the partial differential equations. Appl Soft Comput 9(1):20–29
Shoji D, Noguchi R, Otsuki S, Hino H (2018) Classification of volcanic ash particles using a convolutional neural network and probability. Sci Rep 8(1):1–12
Shukla K, Jagtap AD, Blackshire JL, Sparkman D, Karniadakis GE (2021a) A physics-informed neural network for quantifying the microstructure properties of polycrystalline nickel using ultrasound data. arXiv preprint. arXiv:2103.14104
Shukla K, Jagtap AD, Karniadakis GE (2021b) Parallel physics-informed neural networks via domain decomposition. J Comput Phys 447:110683
Sirignano J, Spiliopoulos K (2018) DGM: A deep learning algorithm for solving partial differential equations. J Comput Phys 375:1339–1364
Sokolova I, Bastisya MG, Hajibeygi H (2019) Multiscale finite volume method for finite-volume-based simulation of poroelasticity. J Comput Phys 379:309–324
Song Y, Sung W, Jang Y, Jung W (2020) Application of an artificial neural network in predicting the effectiveness of trapping mechanisms on CO2 sequestration in saline aquifers. Int J Greenhouse Gas Control 98:103042
Soomro AA, Mokhtar AA, Kurnia JC, Lashari N, Lu H, Sambo C (2022) Integrity assessment of corroded oil and gas pipelines using machine learning: a systematic review. Eng Fail Anal 131:105810
Sprunger C, Muther T, Syed FI, Dahaghi AK, Neghabhan S (2021) State of the art progress in hydraulic fracture modeling using AI/ML techniques. Model Earth Syst Environ 8:1–13
Sumpter BG, Noid DW (1992) Potential energy surfaces for macromolecules. A neural network technique. Chem Phys Lett 192(5–6):455–462
Sun L, Gao H, Pan S, Wang JX (2020) Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data. Comput Methods Appl Mech Eng 361:112732
Syed FI, Alshamsi M, Dahaghi AK, Neghabhan S (2020a) Artificial lift system optimization using machine learning applications. Petroleum 8(2):219–226
Syed FI, AlShamsi A, Dahaghi AK, Neghabhan S (2020b) Application of ML & AI to model petrophysical and geo-mechanical properties of shale reservoirs—a systematic literature review. Petroleum 8(2):158–166
Syed FI, Muther T, Dahaghi AK, Negahban S (2021a) AI/ML assisted shale gas production performance evaluation. J Petrol Explor Prod Technol 11(9):3509–3519
Syed FI, Alnaqbi S, Muther T, Dahaghi AK, Negahban S (2021b) Smart shale gas production performance analysis using machine learning applications. Petrol Res 7(1):21–31
Syed FI, Muther T, Dahaghi AK, Neghabhan S (2022a) CO2 EOR performance evaluation in an unconventional reservoir through mechanistic constrained proxy modeling. Fuel 310:122390
Syed FI, Muther T, Dahaghi AK, Negahban S (2022b) Low-rank tensors applications for dimensionality reduction of complex hydrocarbon reservoirs. Energy 244:122680
Syed FI, Dahaghi AK, Muther T (2022c) Laboratory to field scale assessment for EOR applicability in tight oil reservoirs. Petrol Sci 19(5):2131–2149
Tartakovsky AM, Marrero CO, Perdikaris P, Tartakovsky GD, Barajas-Solano D (2020) Physics-informed deep neural networks for learning parameters and constitutive relationships in subsurface flow problems. Water Resour Res 56(5):e2019WR026731
Taylor RA, Pare JR, Venkatesh AK, Mowafi H, Melnick ER, Fleischman W, Hall MK (2016) Prediction of in-hospital mortality in emergency department patients with sepsis: a local big data-driven, machine learning approach. Acad Emerg Med 23(3):269–278
Thanh HV, Sugai Y, Sasaki K (2020) Application of artificial neural network for predicting the performance of CO 2 enhanced oil recovery and storage in residual oil zones. Sci Rep 10(1):1–16
Tipireddy, R., Perdikaris, P., Stinis, P. and Tartakovsky, A., 2019. A comparative study of physics-informed neural network models for learning unknown dynamics and constitutive relations. arXiv preprint arXiv:1904.04058.
van Milligen BP, Tribaldos V, Jiménez JA (1995) Neural network differential equation and plasma equilibrium solver. Phys Rev Lett 75(20):3594
Voytan D, Sen MK (2020) Wave propagation with physics informed neural networks. In: SEG international exposition and annual meeting. OnePetro, October 2020
Waheed UB, Haghighat E, Alkhalifah T, Song C, Hao Q (2021) PINNeik: Eikonal solution using physics-informed neural networks. Comput Geosci 155:104833
Wang HF (2017) Theory of linear poroelasticity with applications to geomechanics and hydrogeology. Princeton University Press, Princeton
Wang Z, Di H, Shafiq MA, Alaudah Y, AlRegib G (2018) Successful leveraging of image processing and machine learning in seismic structural interpretation: a review. Lead Edge 37(6):451–461
Wang Z, Poon J, Sun S, Poon S (2019) Attention-based multi-instance neural network for medical diagnosis from incomplete and low quality data. In: 2019 International joint conference on neural networks (IJCNN), July 2019. IEEE, Piscataway, pp 1–8
Wang B, Cai J, Liu C, Yang J, Ding X (2020) Harnessing a novel machine-learning-assisted evolutionary algorithm to co-optimize three characteristics of an electrospun oil sorbent. ACS Appl Mater Interfaces 12(38):42842–42849
Wang S, Sankaran S, Perdikaris P (2022) Respecting causality is all you need for training physics-informed neural networks. arXiv preprint. arXiv:2203.07404
Wei H, Zhao S, Rong Q, Bao H (2018) Predicting the effective thermal conductivities of composite materials and porous media by machine learning methods. Int J Heat Mass Transf 127:908–916
Wenzlau F, Müller TM (2009) Finite-difference modeling of wave propagation and diffusion in poroelastic media. Geophysics 74(4):T55–T66
Willard J, Jia X, Xu S, Steinbach M, Kumar V (2020) Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems. ACM Comput Surv 10(1145/1122445):1122456
Wiszniowski J, Plesiewicz BM, Trojanowski J (2014) Application of real time recurrent neural network for detection of small natural earthquakes in Poland. Acta Geophys 62(3):469–485
Wright LG, Onodera T, Stein MM, Wang T, Schachter DT, Hu Z, McMahon PL (2021) Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems. arXiv preprint arXiv:2104.13386
Xie Y, Ebad Sichani M, Padgett JE, DesRoches R (2020) The promise of implementing machine learning in earthquake engineering: a state-of-the-art review. Earthq Spectra 36(4):1769–1801
Xu K, Darve E (2020) ADCME: learning spatially-varying physical fields using deep neural networks. arXiv preprint. arXiv:2011.11955
Yang Y, Perdikaris P (2019) Adversarial uncertainty quantification in physics-informed neural networks. J Comput Phys 394:136–152
Yang L, Zhang D, Karniadakis GE (2020) Physics-informed generative adversarial networks for stochastic differential equations. SIAM J Sci Comput 42(1):A292–A317
Yang L, Meng X, Karniadakis GE (2021) B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data. J Comput Phys 425:109913
Yeh IC (1998) Modeling of strength of high-performance concrete using artificial neural networks. Cem Concr Res 28(12):1797–1808
Yu B (2018) The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems. Commun Math Stat 6(1):1–12
Zhang Z, Gu GX (2021) Physics-informed deep learning for digital materials. Theor Appl Mech Lett 11(1):100220
Zhang L, Zhang L, Du B (2016) Deep learning for remote sensing data: a technical tutorial on the state of the art. IEEE Geosci Remote Sens Mag 4(2):22–40
Zhang D, Lu L, Guo L, Karniadakis GE (2019) Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems. J Comput Phys 397:108850
Zhang D, Guo L, Karniadakis GE (2020) Learning in modal space: solving time-dependent stochastic PDEs using physics-informed neural networks. SIAM J Sci Comput 42(2):A639–A665
Zhou L, Zhang Y, Hu Z, Yu Z, Luo Y, Lei Y, Lei H, Lei Z, Ma Y (2019) Analysis of influencing factors of the production performance of an enhanced geothermal system (EGS) with numerical simulation and artificial neural network (ANN). Energy Build 200:31–46
Zhu Y, Zabaras N, Koutsourelakis PS, Perdikaris P (2019) Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data. J Comput Phys 394:56–81
Zimmerman DC, Hasselman T, Anderson M (2005) Approximation and calibration of nonlinear structural dynamics. Nonlinear Dyn 39(1):113–128
Zobeiry N, Humfeld KD (2021) A physics-informed machine learning approach for solving heat transfer equation in advanced manufacturing and engineering applications. Eng Appl Artif Intell 101:104232
Zubov K, McCarthy Z, Ma Y, Calisto F, Pagliarino V, Azeglio S, Bottero L, Luján E, Sulzer V, Bharambe A, Vinchhi N (2021) NeuralPDE: automating physics-informed neural networks (PINNs) with error approximations. arXiv preprint. arXiv:2107.09443
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Muther, T., Dahaghi, A.K., Syed, F.I. et al. Physical laws meet machine intelligence: current developments and future directions. Artif Intell Rev 56, 6947–7013 (2023). https://doi.org/10.1007/s10462-022-10329-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-022-10329-8