Abstract
Datasets, especially those related to medicine, commonly suffer from missing data. The missing data originates from various sources. Examples include ever-changing medical diagnosis and treatment techniques, the absence of lab results, or even data collection errors. Most machine learning methods are trained on dense datasets. The sparse samples are either discarded or filled in with imputation. Imputation methods generate missing data by examining the variables in the relevant samples. Therefore, the performance of subsequence prediction models might be impacted by these methods. In this study, we explore neural network-based imputation methods to generate the missing data in medical datasets. The experimental results show that compared with traditional imputation methods, neural network imputation can be more effective in the classification and prediction tasks. We discuss some of the method’s differences and assess their suitability for the dataset’s specific characteristics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Azur, M.J., Stuart, E.A., Frangakis, C., Leaf, P.J.: Multiple imputation by chained equations: what is it and how does it work? Int. J. Methods Psychiatr. Res. 20(1), 40–49 (2011)
Bland, M.: An Introduction to Medical Statistics. Oxford University Press, Oxford (2015)
Buck, S.F.: A method of estimation of missing values in multivariate data suitable for use with an electronic computer. J. Roy. Statist. Soc.: Ser. B (Methodol.) 22(2), 302–306 (1960)
Candès, E.J., Tao, T.: The power of convex relaxation: near-optimal matrix completion. IEEE Trans. Inf. Theory 56(5), 2053–2080 (2010)
Cao, W., Wang, D., Li, J., Zhou, H., Li, L., Li, Y.: Brits: bidirectional recurrent imputation for time series. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
cBioPortal: Prostate Adenocarcinoma (TCGA Firehose Legacy). www.cbioportal.org/study/summary?id=prad_tcga
Du, W., Côté, D., Liu, Y.: Saits: Self-attention-based imputation for time series. arXiv preprint arXiv:2202.08516 (2022)
Hastie, T., Mazumder, R., Lee, J.D., Zadeh, R.: Matrix completion and low-rank SVD via fast alternating least squares. J. Mach. Learn. Res. 16(1), 3367–3402 (2015)
Hieronymus, H., et al.: Copy number alteration burden predicts prostate cancer relapse. Proc. Nat. Acad. Sci. 111(30), 11139–11144 (2014)
Hoadley, K.A., et al.: Cell-of-origin patterns dominate the molecular classification of 10,000 tumors from 33 types of cancer. Cell 173(2), 291–304 (2018)
Jamshidian, M., Bentler, P.M.: Ml estimation of mean and covariance structures with missing data using complete data routines. J. Educ. Behav. Stat. 24(1), 21–24 (1999)
Jarrett, D., Cebere, B.C., Liu, T., Curth, A., van der Schaar, M.: Hyperimpute: generalized iterative imputation with automatic model selection. In: International Conference on Machine Learning, pp. 9916–9937. PMLR (2022)
Jiang, C., Yang, Z.: CKNNI: an improved KNN-based missing value handling technique. In: Huang, D.-S., Han, K. (eds.) ICIC 2015. LNCS (LNAI), vol. 9227, pp. 441–452. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-22053-6_47
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114 (2013)
Kyono, T., Zhang, Y., Bellot, A., van der Schaar, M.: Miracle: causally-aware imputation via learning missing data mechanisms. Adv. Neural Inf. Proces. Syst. 34, 23806–23817 (2021)
Lall, R., Robinson, T.: The MIDAS touch: accurate and scalable missing-data imputation with deep learning. Polit. Anal. 30(2), 179–196 (2022)
Lee, J.Y., Styczynski, M.P.: NS-kNN: a modified k-nearest neighbors approach for imputing metabolomics data. Metabolomics 14(12), 1–12 (2018)
Liang, C., Zhang, L., Wan, Z., Li, D., Li, D., Li, W.: An improved kNN method based on spearman’s rank correlation for handling medical missing values. In: 2022 International Conference on Machine Learning and Knowledge Engineering (MLKE), pp. 139–142. IEEE (2022)
Little, R.J., Rubin, D.B.: Statistical Analysis with Missing Data, vol. 793. John Wiley, Hoboken (2019)
Luo, Y., Zhang, Y., Cai, X., Yuan, X.: E2gan: end-to-end generative adversarial network for multivariate time series imputation. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 3094–3100. AAAI Press (2019)
Mattei, P.A., Frellsen, J.: Miwae: deep generative modelling and imputation of incomplete data sets. In: International Conference on Machine Learning, pp. 4413–4423. PMLR (2019)
Muzellec, B., Josse, J., Boyer, C., Cuturi, M.: Missing data imputation using optimal transport. In: International Conference on Machine Learning, pp. 7130–7140. PMLR (2020)
Rubin, D.B.: Inference and missing data. Biometrika 63(3), 581–592 (1976)
Srebro, N., Rennie, J., Jaakkola, T.: Maximum-margin matrix factorization. In: Advances in Neural Information Processing Systems, vol. 17 (2004)
Stekhoven, D.J., Bühlmann, P.: Missforest-non-parametric missing value imputation for mixed-type data. Bioinformatics 28(1), 112–118 (2012)
Stopsack, K.H., et al.: Oncogenic genomic alterations, clinical phenotypes, and outcomes in metastatic castration-sensitive prostate cancer. Clin. Cancer Res. 26(13), 3230–3238 (2020)
Yoon, J., Jordon, J., Schaar, M.: Gain: Missing data imputation using generative adversarial nets. In: International Conference on Machine Learning, pp. 5689–5698. PMLR (2018)
Acknowledgments.
This research was partially supported by Chiang Mai University.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kaveeta, V., Sugunnasil, P., Natwichai, J. (2023). Exploration of Neural Network Imputation Methods for Medical Datasets. In: Barolli, L. (eds) Advances in Internet, Data & Web Technologies. EIDWT 2023. Lecture Notes on Data Engineering and Communications Technologies, vol 161. Springer, Cham. https://doi.org/10.1007/978-3-031-26281-4_46
Download citation
DOI: https://doi.org/10.1007/978-3-031-26281-4_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-26280-7
Online ISBN: 978-3-031-26281-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)