Abstract
Background
Coronary artery disease (CAD) risk prediction tools are useful decision supports. Their clinical impact has not been evaluated amongst Asians in primary care.
Objective
We aimed to develop and validate a diagnostic prediction model for CAD in Southeast Asians by comparing it against three existing tools.
Design
We prospectively recruited patients presenting to primary care for chest pain between July 2013 and December 2016. CAD was diagnosed at tertiary institution and adjudicated. A logistic regression model was built, with validation by resampling. We validated the Duke Clinical Score (DCS), CAD Consortium Score (CCS), and Marburg Heart Score (MHS).
Main Measures
Discrimination and calibration quantify model performance, while net reclassification improvement and net benefit provide clinical insights.
Key Results
CAD prevalence was 9.5% (158 of 1658 patients). Our model included age, gender, type 2 diabetes mellitus, hypertension, smoking, chest pain type, neck radiation, Q waves, and ST-T changes. The C-statistic was 0.808 (95% CI 0.776–0.840) and 0.815 (95% CI 0.782–0.847), for model without and with ECG respectively. C-statistics for DCS, CCS-basic, CCS-clinical, and MHS were 0.795 (95% CI 0.759–0.831), 0.756 (95% CI 0.717–0.794), 0.787 (95% CI 0.752–0.823), and 0.661 (95% CI 0.621–0.701). Our model (with ECG) correctly reclassified 100% of patients when compared with DCS and CCS-clinical respectively. At 5% threshold probability, the net benefit for our model (with ECG) was 0.063. The net benefit for DCS, CCS-basic, and CCS-clinical was 0.056, 0.060, and 0.065.
Conclusions
PRECISE (Predictive Risk scorE for CAD In Southeast Asians with chEst pain) performs well and demonstrates utility as a clinical decision support for diagnosing CAD among Southeast Asians.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
BACKGROUND
Risk prediction tools aid physicians to objectively evaluate the probability of coronary artery disease (CAD) among patients presenting with chest pain. Such decision support is particularly useful at a clinical setting where the actual disease prevalence is low, such as at the primary healthcare setting1.
The pre-test probability of CAD reflects a continuum of risk. When reaching a shared decision to refer a patient for further cardiac investigations, one should take into account individual risk appetite and also consider the trade-offs, namely between correctly diagnosing disease versus unnecessary added tests in the otherwise healthy.
However, conventional methods used to evaluate and compare various prediction models, namely the discrimination and calibration statistics, are not intuitive enough to aid decision-making in routine clinical practice.
The Duke Clinical Score2 (DCS), CAD Consortium Score3 (CCS), and Marburg Heart Score4 (MHS) are commonly used prediction models for CAD diagnosis. To date, the clinical implications of using these risk scores have not been compared in a primary care setting. It is also not known which tool is best calibrated for use in an Asian population.
OBJECTIVE
We aimed to develop and validate a diagnostic prediction model for CAD in Southeast Asians using clinical parameters readily available in primary care, and to compare the performance and clinical utility of three existing prediction tools (DCS, CCS, and MHS) against our new model.
METHODS
We report our study in accordance with the TRIPOD statement5.
Study Sites
Singapore is an urbanized island-state in Southeast Asia with a multi-ethnic population of 5.7 million6. SingHealth Polyclinics (SHP) is a provider of subsidized primary healthcare services. Its network of eight polyclinics covered 1.8 million attendances in 20187. Its affiliate, the National Heart Centre Singapore (NHCS), is the largest local tertiary referral center for cardiovascular (CV) diseases.
Participants
A prospective cohort study was conducted on consecutive patients who attended all SHP branch clinics for chest pain. They were stable clinically and were subsequently referred for cardiac evaluation at NHCS between July 2013 and December 2016. Those with (a) existing or prior history of CAD, (b) acute coronary syndrome such as unstable angina and evolving acute myocardial infarction, and (c) age below 30 years were excluded. Ethical approval was obtained from SingHealth Centralised Institutional Review Board (CIRB2018/2851). All participants provided informed consent.
Study Procedure
Patients completed an interviewer-administered questionnaire and underwent resting electrocardiogram (ECG). Electronic medical records (EMR) were accessed to determine clinical history and laboratory test results. Patients without investigations in the preceding year had fasting blood tests taken upon enrolment to determine their lipid and glucose levels.
The patient and his attending doctors (primary care physician and cardiologist) were blinded to the CAD pre-test probability (PTP) results, computed using the various models tested. All subsequent cardiac investigations at NHCS were determined at the clinical discretion of the reviewing cardiologist.
Definition of Predictors
Chest pain was classified clinically as typical, atypical, or non-anginal8. Patients indicated on a diagram the region of their chest discomfort. To determine if chest pain was reproducible upon palpation, a trained investigator palpated the same area in which the discomfort was reported.
Diagnoses of type 2 diabetes mellitus (T2DM), hypertension, or dyslipidemia were retrieved from the EMR. Newly diagnosed T2DM or dyslipidemia was also based on fasting blood glucose ≥ 7 mmol/L (126 mg/dL) or fasting total cholesterol ≥ 5.2 mmol/L (200 mg/dL) respectively. Smoking was defined as current (tobacco product use within the last 6 months), former, or never. Ethnicity was self-reported and categorized as Chinese, Malay, Indian, or “others.”
ECGs were reviewed by an investigator who was blinded to the case records. Results were classified according to the Minnesota code (Appendix in the Supplementary Information).
Outcome Measures
The primary outcome was diagnosis of significant CAD, defined as (a) ≥ 70% luminal stenosis of at least one major coronary artery or ≥ 50% left main stenosis (based on either catheter-based or CT coronary angiography), or (b) clinical diagnosis of CAD in patients without coronary angiography. All clinical diagnoses were independently adjudicated by an investigator who was blinded to the diagnosis of the attending cardiologist. Discrepancies in diagnoses were arbitrated independently by another cardiologist in the study team.
At 1 year of follow-up, matching was done at the respective national registries (Appendix in the Supplementary Information) for mortality and major adverse cardiovascular events (MACE). MACE includes non-fatal myocardial infarction (MI), non-fatal stroke, and coronary revascularization (coronary artery bypass grafting and/or percutaneous coronary intervention). Data on revascularization was obtained from EMR and phone interviews conducted using standardized scripts.
Risk Instruments Tested
The Duke Clinical Score (DCS) predicts critical ischemia (≥ 75% coronary stenosis, or significant left main stenosis). The score was derived from a single tertiary center cohort of patients presenting with chest pain, and who underwent cardiac catheterization between 1969 and 19792.
The CAD Consortium Score (CCS) was developed from a pre-existing database, sourced from a consortium of 18 hospitals across Europe and the USA. It comprises basic (CCS-basic) and clinical (CCS-clinical) models, which can be used in a stepwise manner to predict risk as more clinical information unfolds. CAD was defined as ≥ 50% luminal stenosis3.
The Marburg Heart Score (MHS) was derived from a multicenter primary care cohort in Germany. Patients presented to their general practitioner with chest pain. CAD status was determined by expert panel consensus at 6 months follow-up4.
Refer to details in Appendix Table 1 in the Supplementary Information.
Sample Size Calculation
We envisioned our clinical prediction rule to contain about 10 predictors. Using the rule of thumb of 10 events per predictor, 100 cases of CAD would be needed. CAD prevalence was estimated to be 10%. A 35% dropout was factored. The calculated sample size was 1350. We aimed to recruit an additional 600 patients to allow for internal validation.
Statistical Analysis
The cohort’s baseline characteristics were summarized as frequencies and percentages for categorical variables and mean ± standard deviation (SD) for continuous variables.
Model Development
We developed the PRECISE (Predictive Risk scorE for CAD In Southeast Asians with chEst pain) risk score for CAD in Southeast Asians. Independent t test and chi-square test were used to determine variables associated with CAD. Clinician input and p < 0.05 determined the variable retention in the final regression model. Odds ratios (OR) and their 95% CI were calculated. Univariate and multivariate logistic regression analyses were performed to determine the final independent predictors of CAD. We used complete-case analysis to handle missing data for predictors and outcome.
Values predicted by the PRECISE score belong to a range from 0 to 1, and represent the probability of CAD. The model is:
where p is the predicted probability of CAD, e is a base of natural logarithm, and y is a linear combination of variables (xi) and their estimators (bi) included in the model:
For binary predictors (e.g., dyslipidemia status), xi = 1 if present, and 0 if absent. For ternary predictors (e.g., type of chest pain, in which categories are mutually exclusive), if xi = 1 (e.g., typical pain), then xi + 1 = 0 (e.g., atypical pain).
We created two independent risk calculators:
-
PRECISE-S, a simple model comprising demographic and clinical variables, and
-
PRECISE-C, a clinical model that included additional variables from resting ECG.
Model Discrimination, Validation, and Calibration
Model discrimination (the ability to differentiate between those with and without disease) was measured by the AUC and its 95% confidence interval (95% CI).
When evaluating an association within any given data set, the apparent strength of that association may be overestimated because of idiosyncrasies of the data. To correct for this over-optimism (so-called internal validation), bootstrapping9 was used as a resampling technique.
Here, 829 (50%) new data sets were used by randomly sampling selected subjects from the main data set, with replacement. Next, stepwise multivariable logistic regression analysis was performed on each of these 829 data sets, by considering the models with significant univariable p values (p < 0.1). Internal bootstrap validation (829 bootstrap samples) was used to provide optimism-corrected estimates. It was applied to each of the imputed data sets. The optimism is the decrease in model performance between the bootstrap and the original samples, which can adjust the developed model for overfitting. The corrected calibration slope was used as a shrinkage factor for the regression coefficients, and AUC with 95% CI corrected for over-optimism was estimated.
Model calibration (the agreement between predicted and observed outcomes) was expressed graphically, with observed risks plotted on the y-axis against predicted risks on the x-axis. The corresponding calibration intercept and slope were calculated. The calibration slope evaluates the spread of the estimated risks and has a target value of 1. The calibration intercept is an assessment of calibration-in-the-large, and has a target value of 0. Perfect calibration shows predictions lying on the 45° line of the calibration plot (i.e., a slope of 1 and intercept of 0).
Performance of Comparator Models
We quantified the predictive performance of DCS, CCS (basic and clinical models), and MHS using the original equations (as published)2,3,4. The respective AUC and calibration plots were presented.
Risk Stratification
We stratified the cohort into low, intermediate, and high CAD risk groups, using empirical risk thresholds of 5% and 50% respectively. In clinical practice, patients at low risk are managed expectantly, and those at intermediate risk should be referred for further cardiac investigations, while those at high risk may receive invasive diagnostic tests (e.g., cardiac angiography) from the outset10.
To offer insights to the use of each predictive model in clinical practice, we performed reclassification analysis and calculated the net benefit.
Reclassification Analysis
Reclassification was performed by cross-tabulating the probability classification of patients using PRECISE-C against other risk models. Reclassification was regarded correct if the predicted probability using PRECISE-C was closer to the observed probability of CAD compared to DCS, or CCS-clinical.
Net reclassification improvement11 (NRI) quantifies changes in risk classification when using different models.
where
- NRIcase:
-
= P (up | case) − P (down | case)
- NRInon-case:
-
= P (down | non-case) − P (up | non-case)
NRI reports differences in proportions of patients moving “up” and “down” for cases and non-cases separately. “Up” refers to moving to a higher risk category. “Down” refers to moving to a lower risk category. “Case” refers to patient with CAD; “Non-case” refers to patient without CAD. NRI indicates changes to a patient’s risk category across the stipulated cutoffs, which translates into modification in treatment recommendations. It is interpreted as the percentage reclassified, adjusted for the reclassification direction. In conclusion, NRI values were applied to compare the reclassification capacities of the various models across our pre-determined risk thresholds.
Net Benefit Analysis
We used a threshold probability (Pt) of 5% as the cutoff between low- and intermediate-risk groups. For any chosen threshold probability, there is an associated number of “true positives” and “false positives.” The trade-off between benefits (of detecting CAD among “true positives”) versus harm (of unnecessary cardiac tests among “false positives”) is expressed in terms of net benefit (NB)12. The unit of NB is “true positive.”
We calculated and compared NB for each risk model. The model with the highest NB demonstrates the highest clinical value.
Statistical analysis was performed using IBM SPSS version 25.0 and SAS version 9.4. A p value of < 0.05 was taken to be statistically significant.
RESULTS
Participants
A total of 1858 patients were recruited, of which 1658 had complete outcome data (mean age 56.7 ± 11.1 years, 809 males, 1374 Chinese). Among those excluded were 179 who had dropped out and 21 who were withdrawn.
Prevalence of CAD in our cohort was 9.5% (n = 158). Eighty-six (54.4%) patients had evidence of stenosis on catheter-based angiography, 2 (1.3%) had CT evidence of stenosis, 55 (34.8%) had positive stress test and deemed to have clinically significant CAD, and 14 (8.9%) were diagnosed clinically to have CAD by their cardiologist and started on treatment.
Demographic characteristics are summarized in Table 1. The participant flow diagram is presented in Figure 1. Figure 2 summarizes cardiac investigations at specialist follow-up.
1-Year Follow-up
At 1 year, 1 (0.6%) out of the 158 patients classified as “CAD positive” had died of CV cause, and 81 (51.3%) developed MACE. In comparison, only 1 (0.1%) out of the 1500 patients classified as “CAD negative” had died of CV cause, and 13 (0.9%) developed MACE. Of the 179 dropouts, 1 (0.6%) died of non-CV cause and 1 (0.6%) developed MACE.
Predictors of Coronary Artery Disease
After multivariable analysis, significant predictors of CAD were older age (OR 1.03 (1.02–1.05), p = 0.001), male (OR 5.75 (3.63–9.11), p < 0.001), diabetes mellitus (OR 1.82 (1.20–2.76), p = 0.005), hypertension (OR 1.64 (1.12–2.42), p = 0.012), smoker (OR 2.08 (1.30–3.34), p = 0.002), typical chest pain (OR 3.95 (2.40–6.50), p < 0.001), chest pain radiating to neck (OR 3.18 (1.50–6.74), p = 0.003), Q waves on ECG (OR 2.77 (1.54–5.01), p = 0.001), and ST-T changes on ECG (OR 1.74 (1.07–2.83), p = 0.027) (Table 2).
The final equation for the simple model (PRECISE-S) is:
The final equation for the clinical model (PRECISE-C) with resting ECG parameters is:
The prediction models are available as an online probability calculator. (https://webapps.duke-nus.edu.sg/tools/PRECISE; Appendix Figure 1 in the Supplementary Information)
Performance of Risk Scores
Discrimination and Validation
AUCs for PRECISE-S and PRECISE-C are 0.808 (95% CI = 0.776–0.840) and 0.815 (95% CI = 0.782–0.847) respectively. In the bootstrapped validation cohort, PRECISE-S and PRECISE-C resulted in AUC of 0.825 (95% CI = 0.782–0.868) and 0.841 (95% CI = 0.799–0.883) respectively.
AUCs for DCS, CCS-basic and CCS-clinical models, and MHS were 0.795 (95% CI = 0.759–0.831), 0.756 (95% CI = 0.717–0.794), 0.787 (95% CI = 0.752–0.823), and 0.661 (95% CI = 0.621–0.701) respectively (Table 3).
Calibration
The calibration intercept and slope are 0.025 and 0.503 for PRECISE-S, and − 0.044 and 2.00 for PRECISE-C. The calibration intercept and slope were − 0.037 and 0.313 for DCS, 0.014 and 0.400 for CCS-basic, and 0.013 and 0.382 for CCS-clinical (Fig. 3a–e).
Pre-test Probability Scores
Using DCS, 60.9% of patients were classified as intermediate risk, for which guidelines recommend further cardiac investigations10. Using the CCS-basic and CCS-clinical, 76.5% and 70.2% of patients were grouped as intermediate risk respectively. In comparison, the use of PRECISE-S resulted in 51.0% of patients being classified as intermediate risk, while 47.8% were classified as low risk. PTP ranged between 0 and 67% with PRECISE-S. With PRECISE-C, 48.8% of patients were classified as intermediate risk and 49.8% were classified as low risk. PTP ranged between 0 and 78% with PRECISE-C (Appendix Table 2 in the Supplementary Information).
Reclassification Analysis
73.1% of patients were classified into a different risk category when PRECISE-C was used instead of DCS. 32.3% of patients were classified into a different risk category when PRECISE-C was used instead of CCS-clinical. The NRIcase and NRInon-case for PRECISE-C versus DCS were − 75.0% and 72.9% respectively. The NRIcase and NRInon-case for PRECISE-C versus CCS-clinical were − 26.7% and 30.4% respectively (Table 4; Fig. 4; Appendix Tables 3 and 4 in the Supplementary Information).
Net Benefit Analysis
At the pre-determined threshold probability of 5%, NB of PRECISE-S and PRECISE-C was 0.061 and 0.063 respectively. Taking PRECISE-C as an example for illustration: The primary care physician is willing to refer 20 “at-risk” patients for tertiary evaluation in order to find 1 patient with CAD (i.e., 5% threshold probability). He decides to use PRECISE-C as a clinical decision support tool to identify patients with ≥ 5% PTP of CAD for referral. With the aid of PRECISE-C, he refers a total of 1000 patients for tertiary evaluation, out of which a net of 63 patients are “true positive” for CAD (i.e., a net of 1 “true positive” out of every 16 patients referred).
In comparison, NB for DCS, CCS-basic, and CCS-clinical was 0.056, 0.060, and 0.065 respectively.
DISCUSSION
We developed and validated a clinical decision support tool (PRECISE: Predictive Risk scorE for CAD In Southeast Asians with chEst pain) for the diagnosis of CAD in Southeast Asians presenting with chest pain. Our methods conformed to the TRIPOD5 reporting framework. We also evaluated the performance of existing CAD risk calculators in our population.
The American College of Cardiology/American Heart Association13 and European Society of Cardiology10 guidelines strongly recommend PTP calculation to select at-risk patients for further investigations. Patients with low pre-test risk do not benefit from routine additional testing, while those with intermediate pre-test risk are most likely to benefit from an initial non-invasive test.
PRECISE has demonstrated good discriminatory ability (C-statistic > 0.8 for both PRECISE-S and PRECISE-C), and these results were consistent upon internal validation (bootstrap C-statistic > 0.8 for both models).
Overall disease prevalence can affect the probability of CAD. Latest ESC guidelines publish PTP estimates (based on age, gender, and chest pain type) at approximately one-third of those reported in the 2013 guidelines, in part due to lower CAD prevalence in recent cohorts14,15,16. For our cohort using PRECISE-C, PTP ranged between 0 and 78%, comparable to data from Foldyna et al.14 The latter found a PTP range of between 2 and 48% in patients with stable chest pain referred for non-invasive testing. Their prevalence of CAD was 13.9% (defined as ≥ 50% coronary stenosis) and 6.1% (if defined as ≥ 70% coronary stenosis), similar to the CAD prevalence of 9.5% in our study. Our results also mirror the CAD prevalence (10.5–11.2%) reported on populations in Europe and the USA17.
External Validation of Comparator Models
Based on our results, the DCS, CCS-basic, and CCS-clinical models demonstrate acceptable discrimination (C-statistic 0.795, 0.756, and 0.787 respectively) in our population, while MHS shows poor discrimination (C-statistic 0.661). However, DCS, CCS-basic, and CCS-clinical models are poorly calibrated for Southeast Asians. The calibration plots for the abovementioned models show that they tend to systematically over-estimate the probability of CAD in our primary care population. Over-estimation of PTP will result in increased and unnecessary cardiac investigations.
Prevalence of CAD in the tertiary-based Duke cohort was 66%2. Hence, it is likely that DCS would over-estimate the CV risks18 of patients in the community. Indeed, Cheng et al.16 externally validated DCS in a contemporary population (2003–2010) who had undergone CT angiography. They observed a substantially lower CAD prevalence than what was predicted by DCS (10% actual vs 42% predicted risk of CAD, defined as 70% coronary stenosis).
The predictive accuracy of CCS was found to be good, when externally validated in predominantly white populations19. In our study, although CCS demonstrated acceptable discrimination, it was found to be poorly calibrated for Southeast Asians.
CAD prevalence among Germans in the MHS4 cohort was 15%. Although MHS was developed from a primary care population, it showed poor discriminatory value (C-statistic 0.661) when validated in our Southeast Asian setting. The Marburg Heart study was limited by incomplete tertiary evaluation for the entire study population, resulting in uncertainty in those labeled as “CAD negative.” Furthermore, differing magnitudes and interaction of genetic, environmental, and cultural, as well as socioeconomic risk factors for CAD may result in differences between Western and Asian cohorts20, 21.
Clinical Implications
When PRECISE was used (instead of CCS-clinical or DCS), a greater proportion of patients were classified as low CAD risk (defined empirically as < 5%); 49.8% were classified as low risk using PRECISE-C, versus 22.9% and 6.3% respectively when CCS-clinical or DCS was used. These findings are consistent with the ESC 201910 report of PTP over-estimation by existing risk models.
A 5% risk threshold was chosen as the cutoff between low- and intermediate-risk groups. At the primary care level, patients with low PTP (< 5% risk) are managed expectantly, while those at intermediate (or higher) risk should be referred for further investigations and management at the tertiary setting. In patients with low CAD risk, deferment of routine investigations can reduce healthcare cost, and has also been shown to be safe (< 1% annual risk of CV death or MI)14, 15.
We quantified the clinical impact of choosing PRECISE over DCS or CCS-clinical using NRI and NB. This would make it more intuitive for the practicing clinician to judge the implications of choosing one model over another.
Reclassification analysis showed that PRECISE-C was able to correctly reclassify 100% of patients compared to CCS-clinical. The bulk of these were reclassified to a lower risk category (Appendix Table 4b in the Supplementary Information), likely due to lower CAD prevalence in our cohort (9.5% vs 29% in CCS cohort)3, and lower diagnostic thresholds used in the CCS cohort (≥ 50% coronary stenosis vs ≥ 70% coronary stenosis or ≥ 50% left main stenosis). Our threshold was selected as it better corresponds to myocardial ischemia.
At the pre-determined threshold probability of 5%, both PRECISE-S and PRECISE-C demonstrate superior NB when compared with DCS and CCS-basic. However, CCS-clinical was found to have higher NB than PRECISE-C; We calculated a net of 65 “true positives” out of every 1000 patients referred for tertiary evaluation (with CCS-clinical), versus a net of 63 “true positives” out of every 1000 patients (with PRECISE-C).
Clinical Significance
Patients presenting with chest pain need to be managed safely, and in a timely and objective manner. Although clinical judgment still remains critical, PRECISE can provide useful clinical decision support. With the use of PRECISE CAD risk calculator, patients at intermediate CAD risk can be referred for routine cardiac evaluation, while those at high CAD risk can be given earlier access to a specialist. Finally, patients assessed to have low CAD risk can continue to be managed expectantly at the primary care level.
PRECISE calculator incorporates readily available clinical variables and aligns to current ACC and ESC guidelines10, 13. Substantial literature evidence underpins the independent associations between variables in this model and presence of CAD20, 22,23,24. PRECISE assesses CAD risk with or without ECG data. This broadens its utility in resource-poor primary care settings within Southeast Asia, where ECG facilities or expertise in ECG interpretation may be lacking.
Other proofs of value in using PRECISE, such as cost-savings and patient safety, can be evaluated in a randomized controlled trial.
Limitations
CAD diagnosis was based on variable but valid investigations in this real-world pragmatic study. Although only 11.6% of all patients underwent coronary angiogram, it is noteworthy that a further 68.6% received stress imaging and 8.4% underwent exercise treadmill testing. Our findings reflect the real-world setting, where the majority of patients are in the intermediate PTP strata, and hence undergo stepwise assessment: first with non-invasive functional testing, followed by invasive coronary angiography if needed. Non-invasive functional tests for ischemia typically have better rule-in power10 for CAD than CT coronary angiogram among patients with intermediate PTP.
Our independent adjudication of the CAD diagnosis was a deliberate measure to reduce risk of misclassification. Furthermore, the composite 1-year mortality and MACE among those classified as “CAD negative” was verified to be very low (< 1%).
As expected of a primary care population, CAD prevalence in our study was low. Due to the relatively small number of “CAD positive” patients (n = 158), only internal validation was performed. PRECISE is well calibrated to predict CAD at lower levels of risk (< 20% CAD risk). However, at higher CAD risk levels, there is poor concordance between the predicted and observed risks. These results reflect the inherent nature of our primary care patients.
The inclusion of only patients who were referred to cardiology for further evaluation may have led to selection bias. Bias may also have been introduced by missing data for predictors and outcome (Appendix Table 5 in the Supplementary Information). Another limitation is that our calibration method (i.e., calculation of the calibration intercept and slope) is considered a “Level 2” method in the calibration hierarchy25.
PRECISE has been developed for use in primary care practice. In high prevalence settings, our model may underestimate the actual CAD risk. Further external validation studies are needed to calibrate the performance of PRECISE for use in higher prevalence settings (e.g., outpatient clinic of a tertiary hospital).
This study focused mainly on Chinese, Malay, and Indian ethnic groups. While it did not cover other Asian ethnicities, these aforementioned ethnic groups represent significant proportions of populations living in Malaysia, the vast Indonesian archipelago, minority groups in Thailand, and Indochina in Southeast Asia and beyond to mainland China and India.
CONCLUSIONS
Existing chest pain risk scores have limitations when applied in an Asian population. The novel PRECISE score, based on readily available clinical information, has been developed and validated to predict CAD in Southeast Asians who report of chest pain. PRECISE demonstrates clinical benefits when used as a decision support tool at the primary care setting.
SUMMARY BOX
What is already known on this topic
-
Current ACC/AHA and ESC guidelines for stable coronary artery disease strongly recommend the calculation of pre-test probability to select at-risk patients for further investigations.
-
Existing diagnostic prediction models have been found to over-estimate the risk of coronary artery disease.
What this study adds
-
We developed the PRECISE calculator using readily available clinical variables such as age, gender, type of chest pain, underlying metabolic risk factors, and ECG data.
-
PRECISE is the first diagnostic prediction model for CAD in Southeast Asians that is designed for use in a primary care setting.
Change history
21 February 2024
A Correction to this paper has been published: https://doi.org/10.1007/s11606-024-08684-z
References
Kwok WK, Tang HC, Wee SL, Tai UM, Tan GP, Chua SJ. Pattern and outcome of subsidised referrals to cardiology specialist outpatient clinics. Ann Acad Med Singapore 2008;37:103-8
Pryor DB, Harrell FR Jr, Lee KL, Califf RM, Rosati RA. Estimating the likelihood of significant coronary artery disease. Am J Med 1983;75:771-80
Genders TSS, Steyerberg EW, Hunink MGM, Nieman K, Galema TW, Mollet NR, et al. Prediction Model to Estimate Presence of Coronary Artery Disease: Retrospective Pooled Analysis of Existing Cohorts. BMJ 2012;344:e3485 https://doi.org/10.1136/bmj.e3485
Bösner S, Haasenritter J, Becker A, Karatolios K, Vaucher P, Gencer B, et al. Ruling Out Coronary Artery Disease in Primary Care: Development and Validation of a Simple Prediction Rule. CMAJ 2010; 182;1295-1300 https://doi.org/10.1503/cmaj.100212
Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD Statement. Ann Intern Med. 2015;162:55-63. https://doi.org/10.7326/M14-0697
Department of Statistics Singapore. Population and population structure [Internet]. Singapore: Government of Singapore; 2020 [updated 2019 Sep 25, cited 2020 Jul 20] Available from https://www.singstat.gov.sg/find-data/search-by-theme/population/population-and-population-structure/latest-data
Overview 2018/2019; SingHealth group overall key figures and statistics [Internet]. Singapore: SingHealth Duke-NUS Academia Medical Centre; 2019 [updated 2019 Oct 11; cited 2019 Oct 24] Available from https://www.singhealth.com.sg/about-singhealth/newsroom/Documents/SingHealth-AR18-Insert.pdf
Diamond GA. A clinically relevant classification of chest discomfort. J Am Coll Cardiol 1983;1:574–75
Harrell FE Jr., Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat med 1996;15:361-87.
Knuuti J, Wijns W, Saraste A, Capodanno D, Barbato E, Funck-Brentano C, et al. 2019 ESC Guidelines for the Diagnosis and Management of Chronic Coronary Syndromes. The Task Force for the Diagnosis and Management of Chronic Coronary Syndromes of the European Society of Cardiology (ESC). Eur Heart J 2019;00;1-71 https://doi.org/10.1093/eurheartj/ehz425
Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008 27:157-72; discussion 207-12.
Vickers AJ, Calster BV, Steyerberg EW. Net Benefit Approaches to the Evaluation of Prediction Models, Molecular Markers, and Diagnostic Tests. BMJ 2016;352:i6. https://doi.org/10.1136/bmj.i6
Fihn SD, Gardin JM, Abrams J, Berra K, Blankenship JC, Dallas AP, et al. 2012 ACCF/AHA/ACP/AATS/PCNA/SCAI/STS Guideline for the Diagnosis and Management of Patients with Stable Ischemic Heart Disease. A Report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines, and the American College of Physicians, American Association for Thoracic Surgery, Preventive Cardiovascular Nurses Association, Society for Cardiovascular Angiography and Interventions, and Society of Thoracic Surgeons. Circulation 2012;126:e354-e471. 10.1161/CIR.0b013e318277d6a0
Foldyna B, Udelson JE, Kara´dy J, Banerji D, Lu MT, Mayrhofer T, et al. Pretest Probability for Patients with Suspected Obstructive Coronary Artery Disease: Re-Evaluating Diamond–Forrester for the Contemporary Era and Clinical Implications: Insights from the PROMISE Trial. Eur Heart J Cardiovasc Imaging 2019;20:574-581. https://doi.org/10.1093/ehjci/jey182.
Reeh J, Therming CB, Heitmann M, Højberg S, Sørum C, Bech J, et al. Prediction of Obstructive Coronary Artery Disease and Prognosis in Patients with Suspected Stable Angina. Eur Heart J 2019;40:1426–1435 https://doi.org/10.1093/eurheartj/ehy806
Cheng VY, Berman DS, Rozanski A, Dunning AM, Achenbach S, Al-Mallah M, et al. Performance of the Traditional Age, Sex, and Angina Typicality-Based Approach for Estimating Pretest Probability of Angiographically Significant Coronary Artery Disease in Patients Undergoing Coronary Computed Tomographic Angiography: Results from the Multinational Coronary CT Angiography Evaluation for Clinical Outcomes: an International Multicenter Registry (CONFIRM). Circulation 2011;124:2423-32, 1-8. https://doi.org/10.1161/CIRCULATIONAHA.111.039255. Epub 2011 Oct 24.
Ebell MH. Evaluation of chest pain in primary care patients Am Fam Physician 2011;83:603-5.
Sox HC Jr, Hickam DH, Marton KI, Moses L, Skeff KM, Sox CH, et al. Using the patient’s history to estimate the probability of coronary artery disease: a comparison of primary care and referral practices. Am J Med 1990;89:7-14
Bittencourt MS, Hulten E, Polonsky TS, Hoffman U, Nasir K, Abbara S, et al. European Society of Cardiology–Recommended Coronary Artery Disease Consortium Pretest Probability Scores More Accurately Predict Obstructive Coronary Disease and Cardiovascular Events than the Diamond and Forrester Score; The Partners Registry. Circulation 2016;134:201–211. https://doi.org/10.1161/CIRCULATIONAHA.116.023396
Ueshima H, Sekikawa A, Miura K, Turin TC, Takashima N, Kita Y, et al. Cardiovascular Disease and Risk Factors in Asia: a Selected Review. Circulation 2008;118: 2702–09. https://doi.org/10.1161/CIRCULATIONAHA.108.790048.
Sasayama S. Heart Disease in Asia. Circulation 2008;118:2669-71. https://doi.org/10.1161/CIRCULATIONAHA.108.837054
Kotecha D, Flather M, McGrady M, Pepper J, New G, Krum H, et al. Contemporary Predictors of Coronary Artery Disease in Patients Referred for Angiography. Eur J Cardiovasc Prev Rehabil 2010;17:280-8. https://doi.org/10.1097/HJR.0b013e3283310108.
Menotti A, Mulder I, Kromhout D, Nissinen A, Feskens EJ, Giampaoli S. The association of silent electrocardiographic findings with coronary deaths among elderly men in three European countries. The FINE study. Acta Cardiol 2001;56:27-36.
Rautahariu PM, Kooperberg C, Larson JC, LaCroix A. Electrocardiographic Abnormalities that Predict Coronary Heart Disease Events and Mortality in Postmenopausal Women. The Women’s Health Initiative. Circulation 2006;113:473–80. https://doi.org/10.1161/CIRCULATIONAHA.104.496091
Calster BV, Nieboer D, Vergouwe Y, Cock BD, Pencina MJ, Steyerberg EW. A Calibration Hierarchy for Risk Models Was Defined: from Utopia to Empirical Data. J Clin Epidemiol 2016;74:167-176. https://doi.org/10.1016/j.jclinepi.2015.12.005
Acknowledgements
The authors would like to thank our research coordinators (Siti Maryam, Syarafina, and Chris Goh), the polyclinic nurses, and support staff from SingHealth Polyclinics Department of Research (Patricia Kin, Caris Tan, Usha Sankari) as well as support staff from the National Heart Centre Singapore Specialist Outpatient Clinics for their invaluable help in this project.
Funding
This study was supported by the Lee Foundation; funds were administered via the SingHealth Foundation Grant (SHF/14/GMC(2)/014(RP)).
Author information
Authors and Affiliations
Contributions
WZS and YJ contributed equally as corresponding authors. WZS, YJ, NNV, GSC, CSJ, and TNC contributed to the study concept and design. WZS and TNC obtained funding. WZS, YJ, and KYL had full access to all the data in the study and take responsibility for the integrity of the data. ATW, SKH, YKK, GSC, LCS, TLL, OCW, and CSJ provided administrative support. KYL and CSY performed the statistical analysis. WZS and YJ interpreted the findings and drafted the manuscript. All the authors contributed to the critical revision of the manuscript for important intellectual content. All authors approved the final version of this paper and the authorship list. WZS and YJ are study guarantors. The corresponding authors attest that all listed authors met authorship criteria and that no others meeting the criteria have been omitted.
Corresponding authors
Ethics declarations
The lead authors (WZS and YJ) affirm that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.
Ethics Approval
The study protocol was approved by the SingHealth Centralised Institutional Review Board (CIRB2018/2851). All patients provided informed consent before taking part.
Conflict of Interest
All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare the following: support from the Lee Foundation and SingHealth Foundation; YKK holds research grants sponsored by Amgen, Astra Zeneca, and Holmusk; YKK also receives consulting fees from Abbott Vascular, Boston Scientific, and Medtronic; YKK is a speaker for Shockwave Medical, Abbott Vascular, Boston Scientific, Medtronic, Philips, and Alvimedica; No other relationships or activities that could appear to have influenced the submitted work.
Disclaimer
The funders were not involved in the research and preparation of the article, including study design; collection, analysis, and interpretation of data; writing of the article; or in the decision to submit it for publication.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Zhen Sinead Wang and Jonathan Yap are co-first authors.
Supplementary Information
ESM 1
(DOCX 288 kb)
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, Z.S., Yap, J., Koh, Y.L.E. et al. Predicting Coronary Artery Disease in Primary Care: Development and Validation of a Diagnostic Risk Score for Major Ethnic Groups in Southeast Asia. J GEN INTERN MED 36, 1514–1524 (2021). https://doi.org/10.1007/s11606-021-06701-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-021-06701-z