Skip to main content
SpringerLink
Log in

Deep Neural Networks for QSAR

  • Protocol
  • First Online:
Artificial Intelligence in Drug Design

Part of the book series: Methods in Molecular Biology ((MIMB,volume 2390))

Abstract

Quantitative structure–activity relationship (QSAR) models are routinely applied computational tools in the drug discovery process. QSAR models are regression or classification models that predict the biological activities of molecules based on the features derived from their molecular structures. These models are usually used to prioritize a list of candidate molecules for future laboratory experiments and to help chemists gain better insights into how structural changes affect a molecule’s biological activities. Developing accurate and interpretable QSAR models is therefore of the utmost importance in the drug discovery process. Deep neural networks, which are powerful supervised learning algorithms, have shown great promise for addressing regression and classification problems in various research fields, including the pharmaceutical industry. In this chapter, we briefly review the applications of deep neural networks in QSAR modeling and describe commonly used techniques to improve model performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Protocol
USD 49.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Muratov EN, Bajorath J, Sheridan RP, Tetko IV, Filimonov D, Poroikov V, Oprea TI, Baskin II, Varnek A, Roitberg A et al (2020) QSAR without borders. Chem Soc Rev

    Google Scholar 

  2. Ma J, Sheridan RP, Liaw A, Dahl GE, Svetnik V (2015) Deep neural nets as a method for quantitative structure–activity relationships. J Chem Inf Model 55(2):263–274

    Article  CAS  PubMed  Google Scholar 

  3. Svetnik V, Liaw A, Tong C, Culberson JC, Sheridan RP, Feuston BP (2003) Random forest: a classification and regression tool for compound classification and QSAR modeling. J Chem Inf Comput Sci 43(6):1947–1958

    Article  CAS  PubMed  Google Scholar 

  4. Gawehn E, Hiss JA, Schneider G (2016) Deep learning in drug discovery. Mol Inform 35(1):3–14

    Article  CAS  PubMed  Google Scholar 

  5. Cherkasov A, Muratov EN, Fourches D, Varnek A, Baskin II, Cronin M, Dearden J, Gramatica P, Martin YC, Todeschini R et al (2014) QSAR modeling: where have you been? Where are you going to? J Med Chem 57(12):4977–5010

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  CAS  PubMed  Google Scholar 

  7. Dahl GE, Jaitly N, Salakhutdinov R (2014) Multi-task neural networks for QSAR predictions. arXiv preprint arXiv:1406.1231

    Google Scholar 

  8. Goh GB, Siegel C, Vishnu A, Hodas NO, Baker N (2017) Chemception: a deep neural network with minimal chemistry knowledge matches the performance of expert-developed QSAR/qspr models. arXiv preprint arXiv:1706.06689

    Google Scholar 

  9. Duvenaud DK, Maclaurin D, Iparraguirre J, Bombarell R, Hirzel T, Aspuru-Guzik A, Adams RP (2015) Convolutional networks on graphs for learning molecular fingerprints. Adv Neural Inf Proces Syst:2224–2232

    Google Scholar 

  10. Yang K, Swanson K, Jin W, Coley C, Eiden P, Gao H, Guzman-Perez A, Hopper T, Kelley B, Mathea M et al (2019) Analyzing learned molecular representations for property prediction. J Chem Inf Model 59(8):3370–3388

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Chakravarti SK, Alla SRM (2019) Descriptor free QSAR modeling using deep learning with long short-term memory neural networks. Front Artif Intell 2:17

    Article  PubMed  PubMed Central  Google Scholar 

  12. Liaw A, Svetnik V (2014) QSAR modeling: prediction of biological activity from chemical structure. In: Statistical methods for evaluating safety in medical product development, pp 66–83

    Google Scholar 

  13. Carhart RE, Smith DH, Venkataraghavan R (1985) Atom pairs as molecular features in structure-activity studies: definition and applications. J Chem Inf Comput Sci 25(2):64–73

    Article  CAS  Google Scholar 

  14. Rogers D, Hahn M (2010) Extended-connectivity fingerprints. J Chem Inf Model 50(5):742–754

    Article  CAS  PubMed  Google Scholar 

  15. Cai C, Wang S, Xu Y, Zhang W, Tang K, Ouyang Q, Lai L, Pei J (2020) Transfer learning for drug discovery. J Med Chem 63(16):8683–8694

    Article  CAS  PubMed  Google Scholar 

  16. Sheridan RP, Karnachi P, Tudor M, Xu Y, Liaw A, Shah F, Cheng AC, Joshi E, Glick M, Alvarez J (2020) Experimental error, kurtosis, activity cliffs, and methodology: what limits the predictivity of quantitative structure– activity relationship models? J Chem Inf Model 60(4):1969–1982

    Article  CAS  PubMed  Google Scholar 

  17. Jim’enez-Luna J’e, Grisoni F, Schneider G (2020) Drug discovery with explainable artificial intelligence. Nat Mach Intell 2(10):573–584

    Article  Google Scholar 

  18. Fan F, Xiong J, Wang G (2020) On interpretability of artificial neural networks. arXiv preprint arXiv:2001.02522

    Google Scholar 

  19. Dudek AZ, Arodz T, G’alvez J (2006) Computational methods in developing quantitative structure-activity relationships (QSAR): a review. Comb Chem High Throughput Screen 9(3):213–228

    Article  CAS  PubMed  Google Scholar 

  20. Wiener H (1947) Structural determination of paraffin boiling points. J Am Chem Soc 69(1):17–20

    Article  CAS  PubMed  Google Scholar 

  21. Varnek A, Tropsha A (2008) Chemoinformatics approaches to virtual screening. Royal Society of Chemistry, London

    Book  Google Scholar 

  22. Todeschini R, Consonni V (2009) Molecular descriptors for chemoinformatics: volume I: alphabetical listing/volume II: appendices, references, vol 41. John Wiley & Sons, Hoboken, NJ

    Google Scholar 

  23. Burden FR (2001) Quantitative structure- activity relationship studies using gaussian processes. J Chem Inf Comput Sci 41(3):830–835

    Article  CAS  PubMed  Google Scholar 

  24. Obrezanova O, Cs’anyi G’a, Gola JMR, Segall MD (2007) Gaussian processes: a method for automatic QSAR modeling of adme properties. J Chem Inf Model 47(5):1847–1857

    Article  CAS  PubMed  Google Scholar 

  25. DiFranzo A, Sheridan RP, Liaw A, Tudor M (2020) Nearest neighbor gaussian process for quantitative structure–activity relationships. J Chem Inf Model 60(10):4653–4663

    Article  CAS  PubMed  Google Scholar 

  26. Svetnik V, Wang T, Tong C, Liaw A, Sheridan RP, Song Q (2005) Boosting: an ensemble learning tool for compound classification and QSAR modeling. J Chem Inf Model 45(3):786–799

    Article  CAS  PubMed  Google Scholar 

  27. Sheridan RP, Wang WM, Liaw A, Ma J, Gifford EM (2016) Extreme gradient boosting as a method for quantitative structure–activity relationships. J Chem Inf Model 56(12):2353–2360

    Article  CAS  PubMed  Google Scholar 

  28. Landrum G (2013) Rdkit documentation. Release 1:1–79

    Google Scholar 

  29. Meyer JG, Liu S, Miller IJ, Coon JJ, Gitter A (2019) Learning drug functions from chemical structures with convolutional neural networks and random forests. J Chem Inf Model 59(10):4438–4449

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Shi T, Yang Y, Huang S, Chen L, Kuang Z, Yu H, Hu M (2019) Molecular image-based convolutional neural network for the prediction of admet properties. Chemom Intell Lab Syst 194:103853

    Article  CAS  Google Scholar 

  31. Kearnes S, Goldman B, Pande V (2016) Modeling industrial admet data with multitask networks. arXiv preprint arXiv:1606.08793

    Google Scholar 

  32. Wu Z, Ramsundar B, Feinberg EN, Gomes J, Geniesse C, Pappu AS, Leswing K, Pande V (2018) Moleculenet: a benchmark for molecular machine learning. Chem Sci 9(2):513–530

    Article  CAS  PubMed  Google Scholar 

  33. Landrum G et al (2006) Rdkit: open-source cheminformatics. N/A

    Google Scholar 

  34. Weininger D (1988) Smiles, a chemical language and information system. 1. Introduction to methodology and encoding rules. J Chem Inf Comput Sci 28(1):31–36

    Article  CAS  Google Scholar 

  35. Xinhao Li and Denis Fourches. Inductive transfer learning for molecular activity prediction: Next-gen QSAR models with molpmofit. J Cheminformatics, 12:1–15, 2020

    Google Scholar 

  36. Jaeger S, Fulle S, Turk S (2018) Mol2vec: unsupervised machine learning approach with chemical intuition. J Chem Inf Model 58(1):27–35

    Article  CAS  PubMed  Google Scholar 

  37. Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781

    Google Scholar 

  38. LeCun Y, Bottou L’e, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  39. Fernandez M, Ban F, Woo G, Hsing M, Yamazaki T, LeBlanc E, Rennie PS, Welch WJ, Cherkasov A (2018) Toxic colors: the use of deep learning for predicting toxicity of compounds merely from their graphic images. J Chem Inf Model 58(8):1533–1543

    Article  CAS  PubMed  Google Scholar 

  40. Tran N, Kepple D, Shuvaev S, Koulakov A (2019) Deepnose: using artificial neural networks to represent the space of odorants. In: International conference on machine learning. PMLR, pp 6305–6314

    Google Scholar 

  41. Karpov P, Godin G, Tetko IV (2020) Transformer-cnn: Swiss knife for QSAR modeling and interpretation. J Cheminformatics 12(1):1–12

    Article  Google Scholar 

  42. Szegedy C, Ioffe S, Vanhoucke V, Alemi A (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the AAAI conference on artificial intelligence, vol 31

    Google Scholar 

  43. Zhang Z, Cui P, Zhu W (2020) Deep learning on graphs: a survey. IEEE Trans Knowl Data Eng

    Google Scholar 

  44. Kearnes S, McCloskey K, Berndl M, Pande V, Riley P (2016) Molecular graph convolutions: moving beyond fingerprints. J Comput Aided Mol Des 30(8):595–608

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE (2017) Neural message passing for quantum chemistry. In: Proceedings of the 34th international conference on machine learning, vol 70, pp 1263–1272

    Google Scholar 

  46. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323(6088):533–536

    Article  Google Scholar 

  47. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  CAS  PubMed  Google Scholar 

  48. Yu Y, Si X, Hu C, Zhang J (2019) A review of recurrent neural networks: Lstm cells and network architectures. Neural Comput 31(7):1235–1270

    Article  PubMed  Google Scholar 

  49. Goh GB, Hodas NO, Siegel C, Vishnu S (2017) Smiles2vec: An interpretable general-purpose deep neural network for predicting chemical properties. arXiv preprint arXiv:1712.02034

    Google Scholar 

  50. Altae-Tran H, Ramsundar B, Pappu AS, Pande V (2017) Low data drug discovery with one-shot learning. ACS Cent Sci 3(4):283–293

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13(1):281–305

    Google Scholar 

  52. Montavon G’e, Orr G’e, Müller K-R (2012) Neural networks: tricks of the trade, vol 7700. Springer, New York

    Book  Google Scholar 

  53. Henaff M, Bruna J, LeCun Y (2015) Deep convolutional networks on graphstructured data. arXiv preprint arXiv:1506.05163

    Google Scholar 

  54. Ramsundar B, Liu B, Wu Z, Verras A, Tudor M, Sheridan RP, Pande V (2017) Is multitask deep learning practical for pharma? J Chem Inf Model 57(8):2068–2076

    Article  CAS  PubMed  Google Scholar 

  55. Xu Y, Dai Z, Chen F, Gao S, Pei J, Lai L (2015) Deep learning for drug-induced liver injury. J Chem Inf Model 55(10):2085–2093

    Article  CAS  PubMed  Google Scholar 

  56. Koutsoukas A, Monaghan KJ, Li X, Huan J (2017) Deep-learning: investigating deep neural networks hyper-parameters and comparison of performance to shallow methods for modeling bioactivity data. J Cheminformatics 9(1):42

    Article  Google Scholar 

  57. Brochu E, Cora VM, De Freitas N (2010) A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599

    Google Scholar 

  58. Bergstra J, Bardenet R’e, Bengio Y, K’egl B’a (2011) Algorithms for hyperparameter optimization. Adv Neural Inf Proces Syst 24:2546–2554

    Google Scholar 

  59. Hutter F, Hoos HH, Leyton-Brown K (2011) Sequential model-based optimization for general algorithm configuration. In: International conference on learning and intelligent optimization. Springer, New York, pp 507–523

    Chapter  Google Scholar 

  60. Snoek J, Larochelle H, Adams RP (2012) Practical bayesian optimization of machine learning algorithms. Adv Neural Inf Process Syst 25:2951–2959

    Google Scholar 

  61. Shahriari B, Swersky K, Wang Z, Adams RP, De Freitas N (2015) Taking the human out of the loop: a review of bayesian optimization. Proc IEEE 104(1):148–175

    Article  Google Scholar 

  62. Jamieson K, Talwalkar A (2016) Non-stochastic best arm identification and hyperparameter optimization. In: Artificial intelligence and statistics, pp 240–248

    Google Scholar 

  63. Li L, Jamieson K, DeSalvo G, Rostamizadeh A, Talwalkar A (2017) Hyperband: a novel bandit-based approach to hyperparameter optimization. J Mach Learn Res 18(1):6765–6816

    Google Scholar 

  64. Pelikan M, Goldberg DE, Cantú-Paz E et al (1999) Boa: the bayesian optimization algorithm. In: Proceedings of the genetic and evolutionary computation conference GECCO99, vol 1. Citeseer, Princeton, NJ, pp 525–532

    Google Scholar 

  65. Wistuba M, Schilling N, Schmidt-Thieme L (2015) Learning hyperparameter optimization initializations. In: 2015 IEEE international conference on data science and advanced analytics (DSAA). IEEE, Washington, DC, pp 1–10

    Google Scholar 

  66. Perrone V, Shen H, Seeger MW, Archambeau C, Jenatton R (2019) Learning search spaces for bayesian optimization: another view of hyperparameter transfer learning. Adv Neural Inf Proces Syst:12771–12781

    Google Scholar 

  67. Caruana R (1997) Multitask learning. Mach Learn 28(1):41–75

    Article  Google Scholar 

  68. Sosnin S, Vashurina M, Withnall M, Karpov P, Fedorov M, Tetko IV (2019) A survey of multi-task learning methods in chemoinformatics. Mol Inform 38(4):1800108

    Article  CAS  Google Scholar 

  69. Sosnin S, Karlov D, Tetko IV, Fedorov MV (2018) Comparative study of multitask toxicity modeling on a broad chemical space. J Chem Inf Model 59(3):1062–1072

    Article  CAS  Google Scholar 

  70. Zakharov AV, Zhao T, Nguyen D-T, Peryea T, Sheils T, Yasgar A, Huang R, Southall N, Simeonov A (2019) Novel consensus architecture to improve performance of large-scale multitask deep learning QSAR models. J Chem Inf Model 59(11):4613–4624

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  71. Wenzel J, Matter H, Schmidt F (2019) Predictive multitask deep neural network models for ADME-Tox properties: learning from large data sets. J Chem Inf Model 59(3):1253–1268

    Article  CAS  PubMed  Google Scholar 

  72. Montanari F, Kuhnke L, Ter Laak A, Clevert D-A’e (2020) Modeling physico-chemical admet endpoints with multitask graph convolutional networks. Molecules 25(1):44

    Article  CAS  Google Scholar 

  73. Feinberg EN, Joshi E, Pande VS, Cheng AC (2020) Improvement in admet prediction with multitask deep featurization. J Med Chem 63(16):8835–8848

    Article  CAS  PubMed  Google Scholar 

  74. Xu Y, Ma J, Liaw A, Sheridan RP, Svetnik V (2017) Demystifying multitask deep neural networks for quantitative structure–activity relationships. J Chem Inf Model 57(10):2490–2504

    Article  CAS  PubMed  Google Scholar 

  75. Ruder S (2017) An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098

    Google Scholar 

  76. Zhang Y, Yang, Q (2017) A survey on multi-task learning. arXiv preprint arXiv:1707.08114

    Google Scholar 

  77. Lee J-H, Chan Y-M, Chen T-Y, Chen C-S Joint estimation of age and gender from unconstrained face images using lightweight multi-task cnn for mobile applications. In: 2018 IEEE conference on multimedia information processing and retrieval (MIPR), vol 2018. IEEE, Washington, DC, pp 162–165

    Google Scholar 

  78. Li S, Liu Z-Q, Chan AB (2014) Heterogeneous multi-task learning for human pose estimation with deep convolutional neural network. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 482–489

    Google Scholar 

  79. Duong L, Cohn T, Bird S, Cook P (2015) Low resource dependency parsing: cross-lingual parameter sharing in a neural network parser. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (vol 2: Short papers), pp 845–850

    Google Scholar 

  80. Yang Y, Hospedales TM (2016) Trace norm regularised deep multi-task learning. arXiv preprint arXiv:1606.04038

    Google Scholar 

  81. Teh Y, Bapst V, Czarnecki WM, Quan J, Kirkpatrick J, Hadsell R, Heess N, Pascanu R (2017) Distral: robust multitask reinforcement learning. Adv Neural Inf Proces Syst:4496–4506

    Google Scholar 

  82. Vandenhende S, Georgoulis S, De Brabandere B, Van Gool L (2019) Branched multi-task networks: deciding what layers to share. arXiv preprint arXiv:1904.02920

    Google Scholar 

  83. Misra I, Shrivastava A, Gupta A, Hebert M (2016) Cross-stitch networks for multi-task learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3994–4003

    Google Scholar 

  84. Hashimoto K, Xiong C, Tsuruoka Y, Socher R (2016) A joint many-task model: Growing a neural network for multiple nlp tasks. arXiv preprint arXiv:1611.01587

    Google Scholar 

  85. Kendall A, Gal Y, Cipolla R (2018) Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7482–7491

    Google Scholar 

  86. Chen Z, Badrinarayanan V, Lee C-Y, Rabinovich A (2018) Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In: International conference on machine learning. PMLR, pp 794–803

    Google Scholar 

  87. Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  88. Simoes RS, Maltarollo VG, Oliveira PR, Honorio KM (2018) Transfer and multi-task learning in QSAR modeling: advances and challenges. Front Pharmacol 9:74

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  89. Quionero-Candela J, Sugiyama M, Schwaighofer A, Lawrence ND (2009) Dataset shift in machine learning. The MIT Press, Cambridge, MA

    Google Scholar 

  90. Kouw WM, Loog M (2018) An introduction to domain adaptation and transfer learning. arXiv preprint arXiv:1812.11806

    Google Scholar 

  91. Howard J, Ruder S (2018) Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146

    Google Scholar 

  92. Gaulton A, Bellis LJ, Bento AP, Chambers J, Davies M, Hersey A, Light Y, McGlinchey S, Michalovich D, Al-Lazikani B et al (2012) Chembl: a large-scale bioactivity database for drug discovery. Nucleic Acids Res 40(D1):D1100–D1107

    Article  CAS  PubMed  Google Scholar 

  93. Goh GB, Siegel C, Vishnu A, Hodas N (2018) Using rule-based labels for weak supervised learning: a chemnet for transferable chemical property prediction. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 302–310

    Google Scholar 

  94. Iovanac NC, Savoie BM (2019) Improved chemical prediction from scarce data sets via latent space enrichment. J Phys Chem A 123(19):4295–4302

    Article  CAS  PubMed  Google Scholar 

  95. Rosenstein MT, Marx Z, Kaelbling LP, Dietterich TG (2005) To transfer or not to transfer. In: NIPS 2005 workshop on transfer learning, vol 898, pp 1–4

    Google Scholar 

  96. Torrey L, Shavlik J (2010) Transfer learning. In: Handbook of research on machine learning applications and trends: algorithms, methods, and techniques. IGI Global, Hershey, PA, pp 242–264

    Chapter  Google Scholar 

  97. Liang G, Gao J, Ngo H, Li K, Zhang A (2014) On handling negative transfer and imbalanced distributions in multiple source transfer learning. Stat Anal Data Min 7(4):254–271

    Article  Google Scholar 

  98. Paul A, Vogt K, Rottensteiner F, Ostermann J, Heipke C (2018) A comparison of two strategies for avoiding negative transfer in domain adaptation based on logistic regression. ISPRS 42(2):845–852

    Google Scholar 

  99. Qu C, Ji F, Qiu M, Yang L, Min Z, Chen H, Huang J, Croft WB (2019) Learning to selectively transfer: Reinforced transfer learning for deep text matching. In: Proceedings of the twelfth ACM international conference on web search and data mining, pp 699–707

    Google Scholar 

  100. Hu W, Liu B, Gomes J, Zitnik M, Liang P, Pande V, Leskovec F (2019) Strategies for pre-training graph neural networks. arXiv preprint arXiv:1905.12265

    Google Scholar 

  101. Cao Z, Long M, Wang J, Jordan MI (2018) Partial transfer learning with selective adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2724–2732

    Google Scholar 

  102. Wang Z, Dai Z, P’oczos B, Carbonell J (2019) Characterizing and avoiding negative transfer. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 11293–11302

    Google Scholar 

  103. Zhang J, Ding Z, Li W, Ogunbona P (2018) Importance weighted adversarial nets for partial domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8156–8164

    Google Scholar 

  104. National Research Council et al (2012) Assessing the reliability of complex models: mathematical and statistical foundations of verification, validation, and uncertainty quantification. National Academies Press, Washington, DC

    Google Scholar 

  105. Shafer G, Vovk V (2008) A tutorial on conformal prediction. J Mach Learn Res 9:371–421

    Google Scholar 

  106. Papadopoulos H (2008) Inductive conformal prediction: theory and application to neural networks. In: Tools in artificial intelligence. Citeseer, Princeton, NJ

    Google Scholar 

  107. Vovk V, Gammerman A, Shafer G (2005) Algorithmic learning in a random world. Springer Science & Business Media, Berlin

    Google Scholar 

  108. Lei J, G’Sell M, Rinaldo A, Tibshirani RJ, Wasserman L (2018) Distribution-free predictive inference for regression. J Am Stat Assoc 113(523):1094–1111

    Article  CAS  Google Scholar 

  109. Eklund M, Norinder U, Boyer S, Carlsson L (2012) Application of conformal prediction in QSAR. In: IFIP international conference on artificial intelligence applications and innovations. Springer, New York, pp 166–175

    Chapter  Google Scholar 

  110. Norinder U, Carlsson L, Boyer S, Eklund M (2014) Introducing conformal prediction in predictive modeling. a transparent and flexible alternative to applicability domain determination. J Chem Inf Model 54(6):1596–1603

    Article  CAS  PubMed  Google Scholar 

  111. Eklund M, Norinder U, Boyer S, Carlsson L (2015) The application of conformal prediction to the drug discovery process. Ann Math Artif Intell 74(1–2):117–132

    Article  Google Scholar 

  112. Norinder U, Boyer S (2016) Conformal prediction classification of a large data set of environmental chemicals from toxcast and tox21 estrogen receptor assays. Chem Res Toxicol 29(6):1003–1010

    Article  CAS  PubMed  Google Scholar 

  113. Sun J, Carlsson L, Ahlberg E, Norinder U, Engkvist O, Chen H (2017) Applying mondrian cross-conformal prediction to estimate prediction confidence on large imbalanced bioactivity data sets. J Chem Inf Model 57(7):1591–1598

    Article  CAS  PubMed  Google Scholar 

  114. Svensson F, Aniceto N, Norinder U, Cortes-Ciriano I, Spjuth O, Carlsson L, Bender A (2018) Conformal regression for quantitative structure–activity relationship modeling—quantifying prediction uncertainty. J Chem Inf Model 58(5):1132–1140

    Article  CAS  PubMed  Google Scholar 

  115. Bosc N, Atkinson F, Felix E, Gaulton A, Hersey A, Leach AR (2019) Large scale comparison of QSAR and conformal prediction methods and their applications in drug discovery. J Cheminformatics 11(1):4

    Article  Google Scholar 

  116. Cort’es-Ciriano I, Bender A (2019) Concepts and applications of conformal prediction in computational drug discovery. arXiv preprint arXiv:1908.03569

    Google Scholar 

  117. Cort’es-Ciriano I, Bender A (2018) Deep confidence: a computationally efficient framework for calculating reliable prediction errors for deep neural networks. J Chem Inf Model 59(3):1269–1281

    Article  CAS  Google Scholar 

  118. Huang G, Li Y, Pleiss G, Liu Z, Hopcroft JE, Weinberger KQ (2017) Snapshot ensembles: Train 1, get m for free. arXiv preprint arXiv:1704.00109

    Google Scholar 

  119. Cortes-Ciriano I, Bender A (2019) Reliable prediction errors for deep neural networks using test-time dropout. J Chem Inf Model 59(7):3330–3339

    Article  CAS  PubMed  Google Scholar 

  120. Gal Y, Ghahramani Z (2016) Dropout as a bayesian approximation: representing model uncertainty in deep learning. In: International conference on machine learning, pp 1050–1059

    Google Scholar 

  121. Nix DA, Weigend AS (1994) Estimating the mean and variance of the target probability distribution. In: Proceedings of 1994 IEEE international conference on neural networks (ICNN’94), vol 1. IEEE, Washington, DC, pp 55–60

    Chapter  Google Scholar 

  122. Khosravi A, Nahavandi S, Creighton D, Atiya AF (2011) Comprehensive review of neural network-based prediction intervals and new advances. IEEE Trans Neural Netw 22(9):1341–1356

    Article  PubMed  Google Scholar 

  123. Hirschfeld L, Swanson K, Yang K, Barzilay R, Coley CW (2020) Uncertainty quantification using neural networks for molecular property prediction. arXiv preprint arXiv:2005.10036

    Google Scholar 

  124. Scalia G, Grambow CA, Pernici B, Li Y-P, Green WH (2020) Evaluating scalable uncertainty estimation methods for deep learning-based molecular property prediction. J Chem Inf Model

    Google Scholar 

  125. Ovadia Y, Fertig E, Ren J, Nado Z, Sculley D, Nowozin S, Dillon J, Lakshminarayanan B, Snoek J (2019) Can you trust your model’s uncertainty? Evaluating predictive uncertainty under dataset shift. Adv Neural Inf Proces Syst:13991–14002

    Google Scholar 

  126. Sheridan RP (2019) Interpretation of QSAR models by coloring atoms according to changes in predicted activity: how robust is it? J Chem Inf Model 59(4):1324–1337

    Article  CAS  PubMed  Google Scholar 

  127. Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  Google Scholar 

  128. Guha R, Jurs PC (2005) Interpreting computational neural network QSAR models: a measure of descriptor importance. J Chem Inf Model 45(3):800–806

    Article  CAS  PubMed  Google Scholar 

  129. Fisher A, Rudin C, Dominici F (2019) All models are wrong, but many are useful: learning a variable’s importance by studying an entire class of prediction models simultaneously. J Mach Learn Res 20(177):1–81

    CAS  Google Scholar 

  130. Petsiuk V, Das A, Saenko K (2018) Rise: randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.07421

    Google Scholar 

  131. Goodfellow I, Bengio Y, Courville A, Bengio Y (2016) Deep learning, vol 1. The MIT Press, Cambridge, MA

    Google Scholar 

  132. Simonyan K, Vedaldi A, Zisserman A (2013) Deep inside convolutional networks: visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034

    Google Scholar 

  133. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: European conference on computer vision. Springer, New York, pp 818–833

    Google Scholar 

  134. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016

    Google Scholar 

  135. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision, pp 618–626

    Google Scholar 

  136. Shrikumar A, Greenside P, Kundaje A (2017) Learning important features through propagating activation differences. arXiv preprint arXiv:1704.02685

    Google Scholar 

  137. Bach S, Binder A, Montavon G’e, Klauschen F, Müller K-R, Samek W (2015) On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS One 10(7):e0130140

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  138. Smilkov D, Thorat N, Kim B, Vi’egas F, Wattenberg M (2017) Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825

    Google Scholar 

  139. Ancona M, Ceolini E, Oztireli C, Gross M (2017) Towards better understanding of gradient-based attribution methods for deep neural networks. arXiv preprint arXiv:1711.06104

    Google Scholar 

  140. Sundararajan M, Taly A, Yan Q (2017) Axiomatic attribution for deep networks. arXiv preprint arXiv:1703.01365

    Google Scholar 

  141. Sanchez-Lengeling B, Wei J, Lee B, Reif E, Wang P, Qian WW, McCloskey K, Colwell L, Wiltschko A (2020) Evaluating attribution for graph neural networks. Adv Neural Inf Process Syst 33

    Google Scholar 

  142. McCloskey K, Taly A, Monti F, Brenner MP, Colwell LJ (2019) Using attribution to decode binding mechanism in neural network models for chemistry. Proc Natl Acad Sci U S A 116(24):11624–11629

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  143. Ghorbani A, Abid A, Zou J (2019) Interpretation of neural networks is fragile. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 3681–3688

    Google Scholar 

  144. Kindermans P-J, Hooker S, Adebayo J, Alber M, Schütt KT, Dähne S, Erhan D, Kim B (2017) The (un) reliability of saliency methods. arXiv preprint arXiv:1711.00867

    Google Scholar 

  145. Ribeiro MT, Singh S, Guestrin C (2016) “Why should i trust you?” explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1135–1144

    Google Scholar 

  146. Polishchuk P (2017) Interpretation of quantitative structure–activity relationship models: past, present, and future. J Chem Inf Model 57(11):2618–2639

    Article  CAS  PubMed  Google Scholar 

  147. Eriksson L, Jaworska J, Worth AP, Cronin MTD, McDowell RM, Gramatica P (2003) Methods for reliability and uncertainty assessment and for applicability evaluations of classification-and regression-based QSARs. Environ Health Persp 111(10):1361–1375

    Article  CAS  Google Scholar 

  148. Weaver S, Gleeson MP (2008) The importance of the domain of applicability in QSAR modeling. J Mol Graph Modell 26(8):1315–1326

    Article  CAS  Google Scholar 

  149. Sheridan RP (2015) The relative importance of domain applicability metrics for estimating prediction errors in QSAR varies with training set diversity. J Chem Inf Model 55(6):1098–1107

    Article  CAS  PubMed  Google Scholar 

  150. Candela JQ, Sugiyama M, Schwaighofer A, Lawrence ND (2009) Dataset shift in machine learning. The MIT Press 1:5

    Google Scholar 

  151. Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7167–7176

    Google Scholar 

Download references

Acknowledgments

The author thanks Robert P. Sheridan and Andy Liaw for fruitful discussions and comments.

Author information

Authors and Affiliations

  1. Biometrics Research, Merck & Co., Inc., Rahway, NJ, USA

    Yuting Xu

Authors
  1. Yuting Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuting Xu .

Editor information

Editors and Affiliations

  1. Computational Drug Discovery, Evotec (UK) Ltd., Abingdon, Oxfordshire, UK

    Alexander Heifetz

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Science+Business Media, LLC, part of Springer Nature

About this protocol

Check for updates. Verify currency and authenticity via CrossMark

Cite this protocol

Xu, Y. (2022). Deep Neural Networks for QSAR . In: Heifetz, A. (eds) Artificial Intelligence in Drug Design. Methods in Molecular Biology, vol 2390. Humana, New York, NY. https://doi.org/10.1007/978-1-0716-1787-8_10

Download citation

  • DOI: https://doi.org/10.1007/978-1-0716-1787-8_10

  • Published:

  • Publisher Name: Humana, New York, NY

  • Print ISBN: 978-1-0716-1786-1

  • Online ISBN: 978-1-0716-1787-8

  • eBook Packages: Springer Protocols

Publish with us

Policies and ethics

Search

Navigation