The uncertainty associated with parameter estimations is essential for population model building, evaluation, and simulation. Summarized by the standard error (SE), its estimation is sometimes questionable. Herein, we evaluate SEs provided by different non linear mixed-effect estimation methods associated with their estimation performances. Methods based on maximum likelihood (FO and FOCE in NONMEMTM, nlme in SplusTM, and SAEM in MONOLIX) and Bayesian theory (WinBUGS) were evaluated on datasets obtained by simulations of a one-compartment PK model using 9 different designs. Bootstrap techniques were applied to FO, FOCE, and nlme. We compared SE estimations, parameter estimations, convergence, and computation time. Regarding SE estimations, methods provided concordant results for fixed effects. On random effects, SAEM and WinBUGS, tended respectively to under or over-estimate them. With sparse data, FO provided biased estimations of SE and discordant results between bootstrapped and original datasets. Regarding parameter estimations, FO showed a systematic bias on fixed and random effects. WinBUGS provided biased estimations, but only with sparse data. SAEM and WinBUGS converged systematically while FOCE failed in half of the cases. Applying bootstrap with FOCE yielded CPU times too large for routine application and bootstrap with nlme resulted in frequent crashes. In conclusion, FO provided bias on parameter estimations and on SE estimations of random effects. Methods like FOCE provided unbiased results but convergence was the biggest issue. Bootstrap did not improve SEs for FOCE methods, except when confidence interval of random effects is needed. WinBUGS gave consistent results but required long computation times. SAEM was in-between, showing few under-estimated SE but unbiased parameter estimations.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Meibohm B. and Derendorf H. (2002) Pharmacokinetic/pharmacodynamic studies in drug product development. J. Pharm. Sci. 91:18–31
Sheiner L.B. and Steimer J.L. (2000) Pharmacokinetic/pharmacodynamic modeling in drug development. Annu. Rev. Pharmacol. Toxicol. 40:67–95
Evans W.E., Relling M.V., Rodman J.H., Crom W.R., Boyett J.M., Pui C.H. (1998) Conventional compared with individualized chemotherapy for childhood acute lymphoblastic leukemia. N. Engl. J. Med. 338:499–505
Jelliffe R.W. (1987) Clinical applications of pharmacokinetics and adaptive control. IEEE Trans. Biomed. Eng. 34:624–632
Jelliffe R. W., A. Schumitzky, Van Liu G. M. M., Hu L., Maire P., Gomis P., Barbaut X., and Tahani B. (1993) Individualizing drug dosage regimens: roles of population pharmacokinetic and dynamic models, Bayesian fitting, and adaptive control. Ther. Drug Monit. 15:380–393.
Sheiner L.B., Rosenberg B., Melmon K.L. (1972) Modelling of individual pharmacokinetics for computer-aided drug dosage. Comput. Biomed. Res. 5:411–459
Karlsson M.O. and Sheiner L.B. (1993) The importance of modeling interoccasion variability in population pharmacokinetic analyses. J. Pharmacokinet. Biopharm. 21:735–750
S. Retout, S. Duffull, and F. Mentré. (2001) Development and implementation of the population Fisher information matrix for the evaluation of population pharmacokinetic designs. Comput. Methods Programs Biomed. 65:141–151.
Retout S. and Mentré F. (2003) Further developments of the Fisher information matrix in nonlinear mixed effects models with evaluation in population pharmacokinetics. J. Biopharm. Stat. 13:209–227
Yano Y., Beal S.L., and Sheiner L.B. (2001) Evaluating pharmacokinetic/pharmacodynamic models using the posterior predictive check. J. Pharmacokinet. Pharmacodyn. 28:171–192
Food and Drug Administration. Guidance for industry – Population pharmacokinetics. Available from: http://www.fda.gov/cder/guidance/1852fnl.pdf. Last Accessed: July 4 2006.
F. Mentré and M. E. Ebelin. Validation of population pharmacokinetic/pharmacodynamic analyses: review of proposed approaches. In: COST B1. European cooperation in the field of scientific and technical research. The population approach: measuring and managing variability in response, concentration and dose, L. Aarons, L. P. Balant, M. Danhof, M. Gex-Fabry, U. Gundert-Remy, M. Karlsson, F. Mentre, P. Morselli, M. Rowland, J. L. Steimer, S. Vozeh, and F. Rombout, eds. pp. 148–160, Office for official publications of the European Communities, Brussels.
Ette E.I., Williams P.J., Kim Y.H., Lane J.R., Liu M.J., and Capparelli E.V. (2003) Model appropriateness and population pharmacokinetic modeling. J. Clin. Pharmacol. 43:610–623
Tod M. and Rocchisani J.M. (1996) Implementation of OSPOP, an algorithm for the estimation of optimal sampling times in pharmacokinetics by the ED, EID and API criteria. Comput. Methods Programs Biomed. 50:13–22
Tod M. and Rocchisani J.M. (1997) Comparison of ED, EID, and API criteria for the robust optimization of sampling times in pharmacokinetics. J Pharmacokinet. Biopharm. 25:515–537
Matthews I., Kirkpatrick C., and Holford N. (2004) Quantitative justification for target concentration intervention–parameter variability and predictive performance using population pharmacokinetic models for aminoglycosides. Br. J. Clin. Pharmacol. 58:8–19
Bonate P.L., Craig A., Gaynon P., Gandhi V., Jeha S., Kadota R., Lam G.N., Plunkett W., Razzouk B., Rytting M., Steinherz P., and Weitman S. (2004) Population pharmacokinetics of clofarabine, a second-generation nucleoside analog, in pediatric patients with acute leukemia. J. Clin. Pharmacol. 44:1309–1322
Jolling K., Ruixo J.J., Hemeryck A., Piotrovskij V., and Greway T. (2004) Population pharmacokinetic analysis of pegylated human erythropoietin in rats. J. Pharm. Sci. 93:3027–3038
Bruno R., Washington C. B., Lu J. F., Lieberman G., Banken L., and Klein P. (2005) Population pharmacokinetics of trastuzumab in patients With HER2+ metastatic breast cancer. Cancer Chemother. Pharmacol.
Lindbom L., Pihlgren P., and Jonsson E.N. (2005) PsN-Toolkit–a collection of computer intensive statistical methods for non-linear mixed effect modeling using NONMEM. Comput. Methods Programs Biomed. 79:241–257
Beal S. L. and Sheiner L. B. (1989). NONMEM User’s Guide - Part I. Users Basic Guide. University of California, San Francisco
Pinheiro J. C. and Bates D. M. Mixed-Effects Models in S and S-PLUS. Springer-Verlag, New York, 2001.
Delyon B., Lavielle M., Moulines E. (1999) Convergence of a stochastic approximation version of the EM algorithm. Ann. Stat. 27:94–128
Kuhn E. and Lavielle M. (2004) Coupling a stochastic approximation version of EM with a MCMC procedure. ESAIM P&S: 115–131
D. Spiegelhalter, A. Thomas, N. Best, and D. J. Lunn. WinBUGS User Manual Version 1.4. Available from: http://www.mrc-bsu.cam.ac.uk/bugs/winbugs/contents.shtml. Last Accessed: July 4 2006.
Ette E.I. (1996) Comparing non-hierarchical models: application to non-linear mixed effects modeling. Comput. Biol. Med. 26:505–512
M. Davidian and D. M. Giltinan. Nonlinear Models for Repeated Measurement Data. Chapman & Hall, London, 1995.
Davidian M. and Giltinan D.M. (2003) Nonlinear Models for Repeated Measurement Data: An Overview and Update. J. Agr. Biol. Envir. St. 8:387–419
Lindstrom M.J. and Bates D.M. (1990) Nonlinear mixed effects models for repeated measures data. Biometrics 46:673–687
J. I. Myung and D. J. Navarro. Information matrix. In: Encyclopedia of behavioral Statistics, B. Everitt and D. Howel, eds., 2004.
Retout S., Mentré F., Bruno R. (2002) Fisher information matrix for non-linear mixed-effects models: evaluation and application for optimal design of enoxaparin population pharmacokinetics. Stat. Med. 21:2623–2639
S. L. Beal and L. B. Sheiner. NONMEM User’s Guide – Part II. Users Supplemental Guide. University of California, San Francisco, 1988.
White H. (1982) Maximum likelihood estimation of misspecified model. Econometrica 50:1–25
Kuhn E. and Lavielle M. (2005) Maximum likelihood estimation in nonlinear mixed effects model. Comput. Stat. Data Anal. 49:1020–1038
B. Efron and R. Tibshirani. An Introduction to the Bootstrap. Chapman & Hall, New York, 1993.
Ette E.I. (1997) Stability and performance of a population pharmacokinetic model. J. Clin. Pharmacol. 37:486–495
Das S. and Krishen A. (1999) Some bootstrap methods in nonlinear mixed-effect models. J. Stat. Plan. Inference 75:237–245
Wu C.F.J. (2006) Comment on jackknife, bootstrap and resampling methods in regression analysis. Ann. Stat. 14:1261–1350
Gelman A., Carlin J. B., Stern H. S., and Rubin D. B. Bayesian Data Analysis, 2nd ed. Chapman & Hall, London, 2004.
Gelman A. and Rubin D.B. (1992) Inference from iterative simulation using multiple sequences (with discussion). Stat. Sci. 7:457–511
Brooks S.P. and Gelman A. (1998) Alternative methods for monitoring convergence of iterative simulations. J. Comput. Graph. Stat. 7:434–455
Best N.G., Tan K.K., Gilks W.R., and Spiegelhalter D.J. (1995) Estimation of population pharmacokinetics using the Gibbs sampler. J. Pharmacokinet. Biopharm. 23:407–435
Lunn D.J., Best N., Thomas A., Wakefield J., and Spiegelhalter D. (2002) Bayesian analysis of population PK/PD models: general concepts and software. J. Pharmacokinet. Pharmacodyn 29:271–307
Pouillot R., Albert I., Cornu M., and Denis J.-B. (2003) Estimation of uncertainty and variability in bacterial growth using Bayesian inference. Application to Listeria monocytogenes. Int. J. Food. Microbiol. 81:87–104
Roe D.J. (1997) Comparison of population pharmacokinetic modeling methods using simulated data: results from the population modeling workgroup. Stat. Med. 16:1241–1262
A. J. Boeckmann, L. Sheiner, and S. L. Beal. NONMEM User’s Guide – Part VIII. Help Guide. University of California, San Francisco, 1996.
Tod M., Mentré F., Merlé Y., and Mallet A. (1998) Robust optimal design for the estimation of hyperparameters in population pharmacokinetics. J. Pharmacokinet. Biopharm. 26:689–716
M. Tod, F. Mentré, Y. Merlé, and A. Mallet. Introduction to POPOS (optimization of sampling times for population parameter estimation in pharmacokinetics). In POPOS 1.0. 2000, pp. 4–5.
Gisleskog P.O., Karlsson M.O., Beal S.L. (2005) Use of prior information to stabilize a population data analysis. J. Pharmacokinet. Biopharm. 29:473–505
White D.B., Walawander C.A., Tung Y., and Grasela T.H. (1991) An evaluation of point and interval estimates in population pharmacokinetics using NONMEM analysis. J. Pharmacokinet. Biopharm. 19:87–112
Laird N.M. and Louis T.A. (1987) Empirical Bayes confidence intervals based on bootstrap samples. J. Am. Stat. Assoc. 32:739–757
Wade J.R., Kelman A.W., Howie C.A. and Whiting B. (1993) Effect of misspecification of the absorption process on subsequent parameter estimation in population analysis. J. Pharmacokinet. Biopharm. 21:209–222
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Dartois, C., Lemenuel-Diot, A., Laveille, C. et al. Evaluation of Uncertainty Parameters Estimated by Different Population PK Software and Methods. J Pharmacokinet Pharmacodyn 34, 289–311 (2007). https://doi.org/10.1007/s10928-006-9046-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10928-006-9046-9