Introduction

Comparisons of Markov cohort modeling (MM) and discrete event simulation (DES) methods for cost-effectiveness analysis (CEA) in health are dominated by non-empirical assessments based on authors’ experiences and their understanding of these modeling techniques [1]. From a recent systematic review, only two studies were identified that compared MM and DES empirically, both assessing the cost-effectiveness (CE) of drug therapies; one for early breast cancer and one for HIV [2, 3]. Neither study found that the different modeling techniques altered CE results enough to change decision-making; however, these studies did not consider the impact capacity constraints, competition for limited resources or queuing had on CE results [2, 3]. The use of these different modeling techniques in settings that have significant capacity constraints and high demand has the potential to elicit different CE results and lead to different resource allocation decisions and resultant clinical practice.

Hospital outpatient services are part of the health care system where capacity constraints (e.g., specialist staff, theatre access) are a major feature of the system. Under these conditions, it is hypothesized that MM and DES could produce differing CE ratios. This study investigates the differences in CE estimates generated by MM and DES methods in hospital outpatient orthopedic services when usual orthopedic care (UC) and an orthopedic physiotherapy screening clinic and multidisciplinary service (OPSC) are compared. UC relies on medical specialists to assess and screen all new patients entering the service and directing them to the most appropriate (conservative or surgical) care. This model of service delivery has led to significant delays in service provision due to the limited availability of funding for medical specialists to conduct this screening [4–7]. By using physiotherapists to direct patient care, OPSC seeks to alleviate these delays in low-risk orthopedic patients in triage urgency category 2 (semi-urgent) or 3 (non-urgent) with benign musculoskeletal conditions (primarily of the low back, knee, or shoulder) where serious pathology is not suspected, immediate surgery is not indicated, and the patient is likely to benefit from non-surgical management. These types of service models have been shown to reduce waiting times and staff costs while maintaining quality of service in Canada, the UK, and Australia [6, 8–11]. Previous research has demonstrated the CE of OPSC over UC; however, it is unknown what impact the use of a MM over a DES in this evaluation may have had on the results of this analysis [12].

This study has a number of novel objectives. While previous studies have empirically compared DES and MM for pharmaceutical therapies, this is the first study to empirically compare a MM and DES to determine the CE of health services provision per se. Second, this study aims to compare the results of a MM with a constrained-resources DES model (herein described as a dynamic queuing (DQ) DES model), where queuing time is simulated as a function of the supply of, and demand for, limited resources [13]. To the author’s knowledge, this is the first CE study in health care to attempt such a comparison. Third, this is the only study to provide an empirically based comparison of a MM and all types of DES (i.e., with and without dynamic queuing).

Methods

A previously published MM that compared OPSC and UC was used as the template for the DES model development [12]. Data for the original MM were obtained from hospital administrative sources, published studies, and a retrospective chart audit of 980 patients attending an OPSC with a primary diagnosis involving the knee, shoulder, or lumbar spine [12]. The MM was developed in TreeAge Pro 2014 software and the DES was developed using the Simul8© 2014 software package. The MM included waiting times currently associated with provision of UC in Australia. These waiting times were modeled in the MM by applying transition probabilities to the patient cohort at each fixed-length cycle in the model. To relax the Markovian assumption of memorylessness, ‘tunnel states’ were employed to allow the probabilities of the cohort transitioning to the health states of interest (e.g., drop out, surgery) to vary by health state over the modeled period. Full details of the methods used in the development of the MM have been published previously [12]. As the DES model is an individual patient simulation (IPS) technique it was not necessary to use ‘tunnel states’ to circumvent any Markovian modeling limitations. Instead, each simulated individual carries its own history which can inform the future resource use and pathways that a simulated individual may take through the model structure.

Three DES models were developed in order to fully compare and contrast the outcomes from DES to MM. Two of the DES models had no DQ. The first of these was developed without restriction (DES-no-DQ). The second model was a modification of the first, designed to generate results that calibrated back to those of the MM (DES-CAL). This second model served to validate the initial DES-no-DQ model and demonstrate why the results of the DES-no-DQ and MM differed. The third model incorporated DQ (DES-DQ).

Waiting times to various events such as surgery, drop out, or time to all-cause mortality in the DES-no-DQ were sampled from a series of time-to-event distributions using fixed probabilities at discrete time-points. The time-to-event distributions were sampled at the appropriate point in the modeled analysis (e.g., after an event) to determine when this next event may occur. Other events in the DES-no-DQ model occur instantaneously. The time for each event in the DES-no-DQ model was then compared, and the next event for the simulated patient determined. In this way, the DES-no-DQ model generates a series of formally instantaneous discrete events that occur at varying points in time in the modeled analysis (i.e., not at fixed time points).

In a DES, events are formally instantaneous; this contrasts with the MM where a change of health state takes one full cycle (in this case 3 months). This can lead to the introduction of artificial time delays in the MM. For example, in the MM, the cohort transferring from an orthopedic review to the beginning of the surgical waiting list requires a change in health state, which takes an additional 3 months more than the actual surgical waiting times measured in practice. This is an artificial delay introduced through the application of the MM methods (i.e., an artefact of the MM methods). The DES-CAL model was then developed by simply inserting a number of 3-month delays between the appropriate events into the DES-no-DQ model to mirror those that were implicitly present in the structure of the MM due to its fixed 3-month cycle length (i.e., artefacts of the MM design). The details of these delays and their placement in the DES-CAL model are presented in Table 1. The disaggregated expected number of events, costs, effects, incremental cost, incremental effects, incremental cost-effectiveness ratios (ICERs), cost-effectiveness acceptability curves (CEACs), and response to sensitivity analyses generated by the DES-CAL and the MM were then compared in detail to determine if their results calibrated appropriately.

Table 1 Base case input parameters by model, model arm, and parameter type
Full size table

While the DES-no-DQ and the DES-CAL versions of the model provided a useful starting point for the comparison of the results of each of the modeling methods they did not explore the impact the inclusion DQ may have on CE in this setting. Therefore, after this comparison, the DES model was altered such that queuing times were generated dynamically as a function of the demand (e.g., patients requiring orthopedic assessment) and the capacity of the health care service to assess these patients (e.g., driven by the availability of orthopedic specialists).

Full details of the input parameters applied in the MM and DES economic models, by parameter type (e.g., costs, probabilities), model arm (i.e., UC, OPSC) and model type (i.e., MM, DES-no-DQ, DES-CAL, DES-DQ) are presented in Table 1. The table also presents cross-references between the input probabilities and the structure of the economic model presented in Fig. 1. The distributions used for the probabilistic sensitivity analyses (PSA) were fitted to the data using the method-of-moments [14].

Fig. 1
figure 1

a UC arm of the DES-DQ model. b OPSC arm of the DES-DQ model. DES-DQ discrete event simulation with dynamic queuing, GP general practitioner, NR non-responder, MDS multidisciplinary service, OPSC orthopedic physiotherapy screening clinic, Q queue, RES responder, UC usual orthopedic care

Full size image

The structure of the UC and OPSC arms of the DES-DQ economic model are presented in Fig. 1a, b, respectively. The reader is referred to the previous publication by Comans et al. [12] for additional details regarding the conceptual design of the economic analysis and the MM.

In the UC arm of the model, patients are initially assessed by an orthopedic specialist, they may then be: directed back to their general practitioner for continuing management; referred to conservative management, which could include physiotherapy or other medical management (e.g., corticosteroid injections); placed on a surgical wait list; or continue to be monitored for the need for surgery and called back for review at a later date.

In the OPSC arm of the model, patients are initially screened by a physiotherapist who has postgraduate qualifications in musculoskeletal physiotherapy. Following this assessment, patients may be referred for co-ordinated multidisciplinary non-surgical management. This can include physiotherapy, occupational therapy, dietetics, psychology, and pharmacy as well as other allied health intervention when indicated (e.g., podiatry). In addition to services tailored to individual patients, group-based programs are used to support ongoing self-management, knowledge, and skill development. Alternatively, following screening, the patient may be referred back to an orthopedic specialist. This may occur if issues are identified that indicate the need for urgent medical attention or suggest a strong need for surgical review.

In both arms of the economic model, patients may drop out of care while waiting for an assessment or treatment or die due to all-cause mortality.

At the time of these analyses, the demand for OPSC assessment did not exceed supply and, therefore, queue development was not a feature of this health service. As such, a queue was not dynamically modeled for OPSC assessment. Instead, a brief fixed wait period was applied prior to OPSC assessment in the model to reflect that no assessment occurs instantaneously.

To reduce stochastic variability and parameter uncertainty, the number of simulated patients and PSA iterations were increased until such time as the ICERs generated by the model stabilized. Around 5000 simulated individual patients and 1000 PSA iterations were required to generate stable ICER estimates from the DES models. It was not necessary to reduce stochastic variability in the MM as it is a cohort analysis that generates expected values. To further test the stability of the results, the pseudorandom number generator was seeded with different starting values and the ICERs generated assessed to ensure that the ICER values remained stable. In the DES-DQ version of the model, a range of ‘run-in’ times (i.e., the time the economic model is initially run prior to the initiation of data collection to allow the model to be populated and queues to develop) were explored to allow the queues in the model to develop and to determine the impact these differences in initial conditions have on the CE results.

All versions of the economic model had a time horizon of 5.25 years. The time horizon of the original model was guided by clinical experts who identified the minimum duration in which the main differences in costs and effects between treatment arms were likely to be realized. Given that some patients currently wait up to 3 years for care, 5 years was considered an appropriate time horizon for the modeled analysis. A longer time frame has the potential to introduce more uncertainty in the results as the type of care may change over time. Orthopedic treatment is primarily expected to affect patient quality of life and not life span, therefore, this time horizon was considered appropriate. The DES-DQ model employed queues with exponential inter-arrival times. Each queue used a simple single server structure to reflect the total throughput of the health service being modeled (e.g., specialist orthopedic assessment). The queues used in the DES-DQ model employed a first-in, first out (FIFO) scheduling discipline. The DES queues allowed baulking or reneging due to patient compliance (i.e., drop out) or patient mortality. The DES-DQ model employed a run-in period of 1 year and a total simulation time of 10 years. Only simulated patients that entered the model after the run-in period and were present in the simulation for the designated time horizon (i.e., 5.25 years) had their data included in the cost-effectiveness and queuing analyses (see Fig. 2).

Fig. 2
figure 2

DES-DQ run-in and total simulation time, time horizon, and data collection methods. Pt patient

Full size image

The run-in time applied in the base-case analysis was selected so that the number of simulated patients that had entered the model and were processed by orthopedic specialists was equivalent to that recorded in the data collection used to populate this queue. The total simulation time of the DES-DQ model was then programmed to collect data until such time as the DQ model estimated a mean time to orthopedic specialist review (in the UC arm) that equaled that applied in the MM (i.e., approximately 1.5 years; see Fig. 6). This simply provides a convenient starting point for the comparison of the MM and the DES-DQ models. In practice, mean waiting times would be generated by the modeled analysis based on assumptions around the supply and demand of health services. In this way, the DES-DQ allows the modeler to explore the implications of altering the supply of, and demand for, resources over time and the resultant queuing periods. The impact of altering the mean time to an orthopedic specialist appointment on the cost-effectiveness of OPSC in the DES-DQ analysis is presented in Fig. 7.

The results of the four models (i.e., MM, DES-no-DQ, DES-CAL, and DES-DQ) are compared by presenting the costs, effects, incremental cost and effects, ICERs and CEAC. To provide more detailed insights into the differences generated by each of the economic models, the expected number of events are also compared in a disaggregated form. Further, the dynamically generated queuing times produced by the DES-DQ model were compared with those applied in the other economic models to provide further insight into the differences between these modeling methodologies.

Results

The MM and the DES-CAL models generated almost identical results in all analyses and are not discussed further, however, the results of the DES-CAL model are presented in each of the pertinent outcome comparisons for completeness.

Figure 3 presents the expected number of modeled events predicted to occur in the UC arm of the four economic models. The expected number of modeled events generated by the MM and the DES-no-DQ models were similar. However, the DES-no-DQ model predicted a slight increase in the expected number of modeled events for surgery, ongoing monitoring reviews, discharge to a GP, and a slight reduction in the expected number of drop out events. The most pronounced difference between the DES-DQ model and the other models in the UC arm was that DES-DQ predicted a reduction in the expected number of surgical procedures.

Fig. 3
figure 3

Disaggregated expected number of modeled events for the UC arm by modeling method. DES-CAL discrete event simulation: calibration version, DES-no-DQ discrete event simulation without dynamic queuing, DES-DQ discrete event simulation with dynamic queuing, DO drop out, GP general practitioner, NR non-responder, RES responder, UC usual orthopedic care, MM Markov cohort model

Full size image

Figure 4 presents the expected number of modeled events predicted to occur in the OPSC arm of the four economic models. The expected number of modeled events generated by the MM and the DES-no-DQ models were similar. The DES-no-DQ model predicted a slightly higher expected number of modeled events for the OPSC intervention, being referred back to the waitlist to undergo reassessment by the OPSC, and a slight reduction in the expected number of monitoring reviews or drop out events. The most pronounced difference in the results of the DES-DQ model compared to the other models in the OPSC arm was that DES-DQ predicted an increase in the expected number of surgical procedures per patient.

Fig. 4
figure 4

Disaggregated expected number of modeled events for the OPSC arm by modeling method. DES-CAL discrete event simulation: calibration version, DES-no-DQ discrete event simulation without dynamic queuing, DES-DQ discrete event simulation with dynamic queuing, DO drop out, GP general practitioner, NR non-responder, OPSC orthopedic physiotherapy screening clinic and multidisciplinary service, RES responder, MM Markov cohort model

Full size image

Table 2 presents the cost, effect (quality-adjusted life-years; QALYs) and ICERs (cost per additional QALY) results generated by each of the four economic models. As expected, the MM and the DES-CAL models generate very similar cost, effect, and cost-effectiveness estimates.

Table 2 Cost-effectiveness results by model type
Full size table

While the QALYs predicted by the MM and the DES-no-DQ models are similar, the DES-no-DQ model predicts higher costs in the UC arm and slightly higher costs in the OPSC arm than the MM. This led to OPSC dominating UC in this analysis. The DES-DQ model predicted lower costs in the UC arm and higher costs in the OPSC arm compared to the other models. Similar QALY results were recorded in the UC arm of the DES-DQ model and slightly higher QALYs were generated in the OPSC arm of the DES-DQ model. These cost-and-effect results led to the DES-DQ model generating the highest ICER of the four models at $2342 per additional QALY. The DES models without DQ took substantially longer than the MM to run (10.0 h versus 4.4 s, respectively).

A series of univariate sensitivity analyses were conducted for each of the four economic analyses to compare how these changes affected the ICER results generated in the economic models (Table 3). In all models, decreasing the disutility associated with waiting reduced the incremental difference in QALYs between arms and worsened the ICER for OPSC versus UC slightly. Increasing the disutility of waiting had the opposite effect in all analyses. Similarly, decreasing the utility of responders to surgery and other interventions decreased the incremental difference in QALYs between arms and worsened the ICER for OPSC versus UC in all analyses and vice versa. Doubling the cost of OPSC led to an increase of around $1630–$1700 in the ICER for OPSC versus UC. The different models behaved similarly in respect to univariate sensitivity analyses.

Table 3 Univariate sensitivity analyses
Full size table

Figure 5 presents the cost-effectiveness acceptability curves (CEAC) generated by the PSA conducted with each of the four economic models. The DES-no-DQ CEAC predicts that OPSC is more CE compared to UC than predicted by the MM. In fact, the DES-no-DQ model suggests that around 54 % of the PSA-iterations of this model predict OPSC will cost less and provide more benefit than UC (i.e., OPSC dominating UC). Of the four models, the DES-DQ model predicts the highest ICER for OPSC versus UC with less than 3 % of the PSA-iterations predicting that OPSC would dominate UC.

Fig. 5
figure 5

Cost-effectiveness acceptability curves. NB. All costs presented in Australian dollars (AUD); 1.00 AUD = 0.7630 USD at the 2nd July 2015 (http://www.federalreserve.gov). The results of the MM and the DES-CAL model are almost identical and, as such, their cost-effectiveness acceptability curves are superimposed on each other. CE cost-effective, DES-CAL discrete event simulation: calibration version, DES-no-DQ discrete event simulation without dynamic queuing, DES-DQ discrete event simulation with dynamic queuing, OPSC orthopedic physiotherapy screening clinic and multidisciplinary service, QALY quality-adjusted life-year, UC usual orthopedic care, MM Markov cohort model

Full size image

Figure 6a, b present the mean waiting times to see an orthopedic specialist, or to receive orthopedic surgery, respectively. Each figure compares the wait times applied in the non-dynamic queuing models (i.e., MM, DES-no-DQ and DES-CAL) with those generated by the DES-DQ model over various total simulation run times. The figures also present the maximum individual patient queuing times generated by the DES-DQ model over this period. The non-dynamic queuing models have fixed mean queuing times, whereas the mean queuing times generated by the DES-DQ model increase dynamically over time as the demand for orthopedic specialist appointments and surgery exceeds supply over the model period.

Fig. 6
figure 6

a Waiting time for orthopedic assessment by total simulation run time (UC arm only). b Waiting time for orthopedic surgery by total simulation run time. DES-CAL discrete event simulation: calibration version, DES-no-DQ discrete event simulation without dynamic queuing, DES-DQ discrete event simulation with dynamic queuing, ind individual, max maximum, OPSC orthopedic physiotherapy screening clinic and multidisciplinary service, pt patient, UC usual care, MM Markov cohort model

Full size image

As discussed previously, the base case run-in and total simulation time were set such that the mean time to see an orthopedic specialist generated by the DES-DQ model was equal to that applied in the MM (i.e., 1.5 years; see Fig. 6a).

The last individual to successfully traverse both the orthopedic specialist appointment queue and the orthopedic surgical queues in the UC arm had spent a total of 5.25 years (i.e., the total time horizon of the economic model) waiting for specialist assessment and surgery. Of this waiting time, approximately 1.60 years were spent waiting for orthopedic assessment and 3.65 years (see Fig. 6b) were spent in the surgical queue.

Figure 7 shows the impact-increased mean queuing time for a specialist orthopedic appointment has on the ICER estimates generated by the DES-DQ model. The figure also presents the incremental difference in the percentage of surgical events predicted in each of the model arms (i.e., percentage surgery incremental = percentage surgery OPSC − percentage surgery UC). At a queuing time of around 0.9 years, OPSC dominates UC. As the orthopedic assessment queuing time increases, the ICER increases to a maximum of around $3060 per additional QALY at 1.95 years and then declines to around $2300 per QALY at around 2.8 years. The change in the incremental percentage of surgical procedures with queuing time closely matches the changes in ICER.

Fig. 7
figure 7

Incremental cost-effectiveness and percentage of surgical procedures by orthopedic specialist assessment wait time (DES-DQ). NB. All costs presented in Australian dollars (AUD); 1.00 AUD = 0.7630 USD at the 2nd July 2015 (http://www.federalreserve.gov). DES-DQ discrete event simulation with dynamic queuing, OPSC orthopedic physiotherapy screening clinic and multidisciplinary service, QALY quality-adjusted life-year, UC usual care

Full size image

Discussion

This study is the first to compare MM and DES CE estimates in a capacity-constrained system. In this example, the choice of MM or DES modeling (with or without dynamic queuing) would be unlikely to result in a different resource allocation decision as OPSC was predicted to be highly cost-effective compared to UC in all analyses. However, the results of these analyses did differ, with the MM estimating that OPSC was very cost-effective at around $495 per QALY, the DES-no-DQ predicting that OPSC would dominate UC (i.e., cost less and provide more benefit), and the DES-DQ model predicting a higher ICER estimate at $2342 per QALY.

The two previously published studies comparing DES and MM empirically did not model resource constraints and, therefore, most closely resemble the DES-no-DQ and MM comparison presented herein [2, 3]. In line with previous studies, the DES-no-DQ model generated only slightly different outcomes compared to the MM. In contrast, in the DES-DQ model, which incorporated DQ of capacity constraints, a number of important model outputs differed from those generated by the MM.

The primary reason for the differences in the ICERs predicted by the MM and the DES-no-DQ model was that the MM had fixed cycle lengths of 3 months, which led to some implicit and artificial delays of this duration being incorporated into its structure that were not present in the DES-no-DQ model or in the system being modeled. When these artificial delays were added back into the DES-no-DQ model to produce the DES-CAL model, the outcomes generated by the DES-CAL and MM were nearly identical. Importantly, these implicit time delays present in the MM did not reflect true delays present in clinical practice, rather they are artefacts of the MM process. In contrast, DES models have no fixed cycle length and therefore no such implicit artificial delays were present. This demonstrates one of the advantages of DES over the MM method. However, as other authors have discussed, it would be possible to mitigate this inaccuracy in the MM by employing shorter cycle lengths [15, 16]. Nevertheless, it may be argued that DES incorporates time more explicitly and therefore it is potentially less likely that such implicit time delays from the use of fixed cycle lengths will be introduced during the construction of DES compared to a MM analysis. This is also an advantage DES without DQ has compared to techniques such as Markov modeling with individual patient simulation which, like a Markov cohort modeling has fixed cycle lengths. Further, DES without DQ also differs from Markov modeling with individual patient simulation methods as it manages the sequencing of events by generating a future events list and selects the next closest time-to-event to ascertain which event occurs next in the process. This process is then repeated and any impact the updated patient history may have on future events is captured. In contrast, in Markov modeling with individual patient simulation, a transition probability is calculated for each mutually exclusive competing health state and the individual patient moves into the appropriate health state.

The DES-DQ model has a number of advantages over the other analyses, particularly for economic models of health service provision. In both arms of the MM, DES-no-DQ and DES-CAL models average waiting times are assumed to be fixed over the course of the modeled period (see Fig. 6). The assumption that queues have reached a steady state simplifies such economic analyses and the interpretation of their results. However, in some situations, these assumptions do not adequately reflect the reality of the system being modeled. For example, it is common that demand exceeds supply in health services delivery and, in reality, this leads to dynamically increasing queuing times for patients over the period for which the analyst wishes to measure CE. Further, analyses that assume that queuing has reached a steady state often take these queuing times from historical data and are, in effect, modeling the past. In contrast, the DES-DQ model may use current demand and resourcing levels to dynamically project future queuing times and their impact on CE.

In Australia, the number of people on waiting lists for public hospital orthopedic outpatient clinics, and the length of time they wait, has steadily increased over recent years [7]. This indicates that demand for specialist orthopedic resources in these settings exceeds supply. Often, constraints on health care funding make it unfeasible for a hospital to employ enough specialist resources to review and treat these non-urgent outpatients. Therefore, in the base case of the economic model, it is assumed that orthopedic specialist and surgical throughput is not particularly flexible and remains static over time. When combined with excess demand for these resources, this results in dynamically growing queues over the modeled period. In situations where the nature of the resource in question is highly flexible, then resources could fluctuate rapidly to meet demand, and in some cases, this could be approximated using static waiting times. However, the advantage of using DES-DQ over a static waiting time approach is that it allows the modeler to explore the CE of the intervention of interest under different supply and demand scenarios to assist the decision maker to make informed decisions about the appropriate level of resourcing and the cost-effectiveness of altering such resourcing.

Waiting times for surgery are a function of the number of people seen in screening clinics and the referral patterns from these clinics. The models with fixed average waiting times do not capture the impact the introduction of new capacity such as the OPSC has on waiting times for surgery. DES-DQ allows the model to dynamically predict the change in surgical queuing times in line with the altered demand for these procedures with the introduction of OPSC. In the base case, this results in a decrease in the mean waiting times for patients in the OPSC arm requiring orthopedic surgery (see Fig. 6) and a resultant increase in the percentage of patients that undergo surgical procedures (Fig. 4). This in-turn results in higher costs being accrued in this arm of this model than seen in the other analyses (Table 2).

Unlike the models with fixed queuing times, the DES-DQ model does not assume that the queues in the model have reached a steady-state. For instance, while the waiting times for surgical procedures generated in the OPSC arm of the model appear shorter than recorded in clinical practice for UC, these waiting times are dynamic and are increasing over the modeled time-horizon as the demand for surgery out-strips supply. Therefore, a patient that enters the model early is more likely to gain access to an orthopedic assessment or a surgical procedure than one that enters the model later (see Fig. 6). Consequently, as the duration of the modeled analysis is extended, on average, the proportion of all patients that gain access to surgery in the OPSC arm decreases over the given time-horizon. In the UC arm, this effect is more pronounced, as patients have to traverse two queues, one for orthopedic assessment, and the wait list for the surgical procedure itself. In the base case, the dynamic increases in these queuing times over the modeled time horizon lead to a decrease in the average number of patients that have a surgical procedure in the UC arm of the DES-DQ model compared to the other non-dynamic analyses. This in-turn leads to a decrease in the average cost accrued in the UC arm of the model.

In the DES-DQ model, at relatively short total simulation times, the waiting times generated by the model are also short, allowing quicker access to surgery and thereby increasing the proportion of the patient population accessing these expensive procedures over the modeled time-horizon. In analyses with short waiting times, more patients receive surgery in the UC arm of the model than the OPSC arm leading to lower incremental costs and OPSC dominating UC (Fig. 7). As the total simulation time is extended, and the queuing times further increase, the proportion of total patients in the UC arm receiving surgery falls more rapidly than in the OPSC arm, as these patients have to traverse two dynamically growing queues. Eventually, the queuing times in the UC arm increase to such an extent that the simulated patients cannot traverse both orthopedic specialist assessment and surgical queues and receive surgery within the 5.25-year time horizon of the model, and, as such, these surgical results are not recorded in the results of the analysis. This leads to a higher proportion of patients undergoing surgery in the OPSC arm than the UC arm driving the incremental cost and ICER higher for OPSC compared to UC (Fig. 7). As the total simulation time is further extended, the queues into surgery in the OPSC arm grow to such an extent that the incremental difference in the percentage of surgical procedures per patient predicted in each model arms narrows, reducing incremental costs and reducing the ICER for OPSC versus UC (Fig. 7).

The DES-DQ model captures these dynamic increases in queuing times and their effects on patient morbidity and the CE of the interventions being investigated whereas the MM and the DES models without DQ do not. In this way, the DES-DQ model makes any assumptions about the nature of queue development in the clinical setting explicit. In practice, this makes the interpretation of the modeled results more complex. However, these results are potentially more accurate and may provide the decision-maker with deeper and more nuanced insights into the research question being analyzed.

This study is not without its limitations. Due to data availability, the data used to populate the dynamic queues in the DES-DQ models were derived from a more recent data-set taken from one of the clinical audit study sites, whereas the non-DQ models used older queuing data from multiple hospital sites. This could lead to different modeled results beyond those expected due to modeling methods alone. However, this difference further demonstrates the aforementioned advantage of the DES-DQ analysis over the MM as it is truly modeling the future CE of the intervention by explicitly taking into account current demand, staffing levels and throughput, whereas the MM is relying on a static depiction of historical queuing times. Furthermore, while not the primary focus of this research, it should be acknowledged that the comparison of UC and OPSC presented herein is based on non-randomized evidence and, therefore, the possibility that these data may be affected by confounding cannot be entirely ruled out. Nevertheless, any confounding will affect all four models in a similar manner.

In summary, the MM and DES without DQ can produce nearly identical results when the cycle length used in the MM is accounted for. As such, these models are likely to elicit the same resource allocation decisions. Similarly, in this setting, the use of the DES-DQ model would be unlikely to change the resource allocation decisions surrounding the use of OPSC, as it appears to be highly cost-effective compared to UC. However, in this analysis, the queues of interest were formed outside of the hospital in a low-cost environment where patient disutility due to waiting was assumed to be low. In another setting where queues are formed and where costs or disutility are high, the use of a modeling method with dynamic queuing may alter resource allocation decisions. Furthermore, in situations where the cost-effectiveness estimates of the proposed intervention are closer to decision-makers’ WTP threshold per QALY the use of DES-DQ over a MM (or a DES model without DQ) may lead to a change in reimbursement priorities and resultant clinical practice. By making any assumptions around the development of queues explicit, the DES-DQ model may more fully inform the decision-makers’ understanding of the interventions CE and thereby better support their resource allocation decisions than the comfortable fiction of invariant queuing times. However, if the mean queuing time in the research question of interest is known, and has reached a steady state, both MM and DES-DQ models are likely to generate similar results. Where mean queuing times are changing dynamically over the model period, a DES-DQ model would be preferred. DES with DQ is likely to become increasingly valuable when modeling research questions where multiple queues interact and generate emergent results that are otherwise intractable to simple calculations with a MM. However, these insights come at a cost, with the time required to develop, verify, validate, and run the DES analyses substantially exceeding those of the MM.