Introduction

Decisions about specific healthcare choices are frequently framed in the context of the trade-off between harms and benefits [1]. For apical prolapse repair, surgical options include traditional vaginal native tissue (NT) repair, or mesh-augmented repair via vaginal or abdominal routes. Compared to abdominal mesh repair, traditional vaginal NT repair is less costly, with shorter operative and recovery times, but carries a higher risk of prolapse recurrence with a significant number of patients undergoing reoperation for repair of recurrent prolapse [2]. In an effort to reduce the risk of prolapse recurrence while maintaining shorter operative and recovery times, vaginal repair with mesh augmentation (transvaginal mesh, TVM) was developed and has become a commonly utilized alternative [3, 4]. However, mesh augmentation introduces an additional risk of mesh-related complications that may require surgical treatment [2, 5, 6].

Patients and clinicians choosing an approach to vaginal surgery for apical prolapse repair must weigh the potential harm of additional surgery for mesh-related complications and the potential benefit of a reduced risk of recurrent prolapse. Although the best evidence for these estimates would be obtained from large randomized trials, such data are not currently available. Until then, we are able to model estimates based on pooled data from observational studies. In 2009, Diwadkar et al. performed a systematic review of the complications and reoperation rates after apical prolapse surgical repair including traditional NT repair and TVM repair [7]. Our objective was to estimate the risk of reoperation for mesh exposure/erosion and reoperation for recurrent prolapse from these data. Additionally, as the average follow-up time for vaginal NT repair studies was significantly longer than that for TVM repair studies [7], our second objective was to examine how various ways of incorporating follow-up time affected our outcome estimates.

Materials and methods

This study used only published literature or publically available aggregated data and was declared exempt from review by the Duke University Heath System Institutional Review Board.

Markov model

We constructed a Markov state-transition model using TreeAge Pro 2014 software (TreeAge Pro, Williamstown, MA) to estimate the cumulative 24-month probabilities of reoperation for mesh-related complications (exposure or erosion), and/or reoperation for recurrent prolapse, after either traditional vaginal NT repair or TVM repair of vaginal apical prolapse (Fig. 1). Markov models are widely used in health care and are particularly well suited to the estimation of the likelihood of events over time regardless of whether the risk is constant or changing [8]. Simplifying assumptions included: (1) no competing risks such as reoperation for other indications or death, (2) all states considered mutually exclusive (e.g., recurrent prolapse symptoms do not develop during an episode of mesh erosion, and vice versa), (3) post-operative event rates are constant over time (e.g., an event is as likely to occur in the 6th month after the procedure as in the 16th month), and (4) probabilities not adjusted for individual patient or procedure characteristics, such as age, body mass index, preoperative prolapse stage, concomitant procedures, or tobacco use, that might affect outcome [6, 9, 10]. Although the model includes transitional states such as perioperative complications and medical management of erosion, we assumed that these had no effect on the ultimate likelihood of surgical management of recurrent symptoms or mesh-related complications The model also can incorporate more complex sequences of events (for example, a patient could undergo NT repair, have a recurrence and undergo TVM repair, and subsequently have a mesh-related complication), although for the purposes of this analysis we focused only on the outcomes of the initial procedure.

Fig. 1
figure 1

Schematic of Markov model. After the immediate postoperative period, patients with symptomatic relief are at risk of recurrent symptoms or a mesh-related complication, with the monthly probability of an event depending on the choice of initial procedure (e.g., the monthly probability of a mesh-related complication after NT repair is 0 %)

Full size image

The model uses monthly cycles and we limited this analysis to 24 months after the procedure because of variability in the length of follow-up across studies; reports of mesh complications in particular tended to have shorter follow-up time.

All parameter estimates were characterized as probability distributions, with the type of distribution noted below. The model was run using Monte Carlo simulation: for each analysis, the model was run 100,000 times, drawing the value for a specific parameter from its distribution during each simulation (for example, for a normal distribution, the majority of values will be close to the mean value, but 2.5 % will below the lower 95 % confidence bound, and 2.5 % above the upper 95 % confidence bound). This probabilistic approach allows assessment of the impact of uncertainty in the values of specific parameters on confidence in the model estimates by providing confidence intervals (CIs) around the model output.

The model was validated by comparing model-based estimates of event probabilities for NT and TVM repair using the mean follow-up times reported by Diwadkar et al. (32 and 17 months, respectively); results were identical for surgery for recurrent prolapse (3.9 % and 1.3 %, respectively), and nearly so for overall mesh events (5.8 % for the paper, 5.3 % for the model) [7].

Parameter estimates

For this analysis, the two outcomes of interest were (1) reoperation for mesh exposure/erosion and (2) reoperation for recurrent prolapse. We estimated the cumulative probability of these outcomes after NT or TVM repair using several methods.

Published pooled estimates

One source was the published estimates from the systematic review by Diwadkar et al. [7]. This review summarized complication and reoperation proportions after apical prolapse surgical repair via both vaginal and abdominal routes, based on articles and abstracts published from January 1985 through January 2008 with a minimum of 50 subjects and a minimum follow-up period of 3 months. The majority of subjects undergoing NT repair underwent either uterosacral ligament suspension or sacrospinous ligament fixation, but other less common techniques were also reported. For the TVM group, a variety of vaginal mesh kits were included for purposes of anterior/apical repair, apical/posterior repair and combined anterior/apical/posterior repair. The authors calculated weighted averages and 95 % CIs of the proportion of subjects experiencing a variety of outcomes (unadjusted for follow-up time) following apical prolapse repair. Although the mean follow-up time across all studies was reported, only crude proportions, not rates per person-time, were reported.

We used two different estimates for outcome probability based on the review of Diwadkar et al. First (method 1), we used the reported pooled proportion of outcomes (number of operations for recurrent prolapse or mesh exposure/erosion divided by number of subjects) for NT and TVM repair, characterized as beta distributions (beta distributions cannot go below 0 or above 1.0, and are typically used for parameters such as probabilities or proportions which are similarly bound). Second (method 2), we converted these proportions to monthly rates by dividing the proportion by the follow-up time in months. We characterized follow-up time as log-normal distributions (which, like follow-up time, cannot go below 0), based on the reported mean and standard deviation in the paper. These monthly rates were then converted to monthly probabilities using standard methods [8].

Direct estimation of pooled estimates

We reviewed all studies included in the review by Diwadkar et al., and abstracted all data related to mesh exposure/erosion, mesh infection, reoperation for mesh exposure/erosion or mesh infection, and reoperation for recurrent prolapse of any type (i.e. anterior, apical, posterior or a combination; Appendix 1 and 2). We also abstracted data on the follow-up period for each study. Two independent reviewers assessed eligibility and abstracted data. In cases of discordance, differences were discussed until a consensus was reached. If unable to reach a consensus, a third reviewer intervened to make the final decision. Data were abstracted using a standard form. Of note, we did not include studies that did not directly report on our outcomes. Thus, as is shown in Table 1, there were fewer studies included in our subsequent analysis (33 NT repair studies and 22 TVM repair studies) than in the original 2009 review paper (48 NT repair studies and 24 TVM repair studies).

Table 1 Model inputs
Full size table

We then performed a random effects meta-analysis to generate summary estimates of proportions of outcomes (number of outcomes divided by number of patients), and rates of outcomes (number of outcomes divided by person-time of follow-up), using a published and validated method using Excel [11] based on the method of DerSimonian and Laird [12]. Because studies typically did not report time-to-event analyses, we assumed that all subjects experienced the reported mean follow-up time [13].

From the meta-analysis results, we generated two additional estimates of monthly probability of outcomes. For one set of estimates (method 3), we converted the summary estimate of proportion of events (characterized as a beta distribution) to a monthly rate by dividing by mean follow-up time per study, then converting the rate to a probability. Finally (method 4), we derived summary estimates for monthly outcome rates directly from the meta-analysis of study-specific events per patient-time, and converted these to monthly probabilities characterized as beta distributions.

In studies in which overall mesh exposure or erosion was reported but the number of surgical interventions to treat it were not, we followed the approach of Diwadkar et al. to estimate operations for mesh-related complications, and, based on a systematic review by Abed et al., we assumed that half of patients with mesh exposure/erosion would be treated with reoperation involving mesh removal in the operating room [7, 14].

Outcomes

We estimated the mean (with 95 % CI) cumulative probabilities of additional surgery for recurrent prolapse (NT repair) or recurrent prolapse plus mesh-related complications (TVM repair). For TVM repair, we also estimated an incremental harm/benefit ratio, defined as

$$ \frac{Surgeries\;for\; Mesh\; Complications}{Surgeries\;for\; Recurrent\; Prolaps{e}_{NT}- Surgeries\;for\; Recurrent\; Prolaps{e}_{TVM}} $$

For each analysis, we also estimated the probability that TVM would result in fewer operations for recurrent prolapse than NT, defined as the proportion of simulations where the 24-month cumulative probability of this outcome was lower for TVM than for NT.

Results

Table 1 shows model parameter values for each of the four approaches. The “aggregated” values include proportions only (method 1) or rates (method 2) based on total numbers of patients, events, and follow-up time, based either on the values reported by Diwadkar et al. or our abstraction of relevant citations, while the “meta-analysis” values were derived from the random effects meta-analyses. Tests for heterogeneity showed substantial variability in estimates across studies, with I 2 values ranging from 0.65 to 0.88.

Table 2 shows the results of the four different analytic approaches. Method 1 (the aggregate proportions alone without accounting for follow-up time) resulted in a mean harm/benefit ratio of 1.4, with surgery for recurrent prolapse being more common after NT repair in 100 % of the simulations. Note that, as shown Table 1, the proportions derived from the meta-analysis differed somewhat from the aggregate value; however, the estimated ratio of mesh exposure/erosion operations incurred to operations for recurrent prolapse prevented by TVM was quite similar (1.7).

Table 2 Results by analytic method
Full size table

Using method 2 (monthly rates derived from reported aggregate proportions and distribution of follow-up time), the 24-month cumulative probability of surgery for mesh exposure/erosion or for recurrent prolapse after TVM repair was substantially higher, showing the effect of adjusting for follow-up time. CIs were quite wide, because the monthly incidence rate was calculated based on both the proportion of events and the duration of follow-up, and the impact of uncertainty in both of these components on the combined measure was magnified. When this uncertainty was incorporated, the CIs for the probability of surgery for recurrent prolapse overlapped and in approximately 30 % of simulations NT repair resulted in fewer operations for recurrent prolapse over 24 months than TVM repair. The estimated mean harm/benefit ratio is 6.6.

With method 3 (rates based on proportions derived from the meta-analysis, using fixed mean values of follow-up time), the estimated probability of surgery for recurrent prolapse was lower for both NT repair and TVM repair, as was the probability of surgery for mesh exposure/erosion after TVM repair. The smaller absolute difference between the estimated surgical recurrence probabilities resulted in a higher harm/benefit ratio (12.3), and NT repair resulted in fewer operations for recurrent prolapse in approximately 30 % of the simulations.

Method 4 provided the most “direct” estimate of probability, using rates derived from the meta-analysis, which synthesize rates per person-time from each included study. Using this approach, the mean estimated 24-month probability of recurrent prolapse was higher for TVM repair (with recurrent prolapse higher in 72 % of simulations). In this scenario, TVM repair would be “dominated” (excess harms with fewer benefits) compared to NT repair.

Discussion

In the absence of direct comparisons with either randomized trials or large-scale observational studies that adjust for potential confounding, synthesis of smaller observational studies is often the only alternative for generating estimates for patients and clinicians faced with choices about different treatment options. Based on the best available current evidence, there is a substantially greater likelihood that a woman undergoing mesh-augmented repair for apical prolapse will undergo reoperation for mesh exposure or erosion than avoid reoperation for recurrent prolapse. Based on this evidence, when reported follow-up time is taken into account, there is at least a 30 % likelihood that TVM repair would result in a greater likelihood of undergoing surgery for recurrent prolapse over 24 months, with this likelihood increasing to 70 % using our most robust analysis method (method 4).

These models are based on the best available evidence. However, the quality of this evidence is low: all of the studies had substantial risks of bias because of confounding, there was variability in the procedures used as well as in definitions and completeness of reporting, there was considerable statistical heterogeneity across studies even for specific types of procedures and outcomes, and the simulation model methods introduced additional uncertainty. In addition, because we were interested in comparing results across a variety of methods for synthesizing postoperative event probabilities, we limited studies providing data for our parameter estimates to those included in a recent high-quality review, which only included published data on mesh devices and techniques in use prior to 2009. During our review, we noted that the utility of many relatively large studies of prolapse surgery was limited by sometimes sparse and inconsistently reported outcomes. For instance, while recurrent anatomical prolapse after repair was often described, whether or not symptomatic patients required surgery and, if so, by what method surgery was performed was frequently not stated. Details about the management of mesh exposure/erosion were also often not clearly specified. For the purposes of this specific analysis, the lack of data describing the time postoperatively that a mesh complication occurred was particularly challenging. As a hypothetical example, a study reporting the finding of a 10 % mesh erosion rate with a mean postoperative follow-up of 12 months may be indicating that 10 % of patients experienced erosion in the first 6 weeks or that the erosions accumulated gradually over 12 months.

Due to the lack of time-to-event data, our model was limited in that all postoperative event rates were constant over time and thus may not accurately reflect the time of erosion if indeed the risk of mesh exposure/erosion or prolapse recurrence is not equal at all time points after surgery. This distinction has obvious significance for the impact of complications on patients as well as implications for the method of erosion itself. This lack of consistency across studies, and our focus on a few specific outcomes rather than the total range of complications of surgical approaches, likely explains some of the quantitative differences between our results for numbers of studies, patients, and events, compared with those of Diwadkar et al. [7]. However, even with these limitations, all three analytic approaches which incorporated follow-up time in the estimate of subsequent surgical probabilities resulted in substantially lower differences in the estimated benefit of TVM repair compared to NT repair, illustrating the importance of using time-to-event as a primary method of analysis for all future studies of postoperative outcomes.

Time-to-event analyses are important not just for generating valid comparisons, but because the specific time when a postoperative complication occurs has clinical significance: two procedures might have identical 5-year recurrence probabilities, but most patients and clinicians would likely prefer one where most of those recurrences occurred during the 5th year to one where there was a constant risk of failure. The lack of time-to-event data forces the assumption of a constant risk in estimating cumulative probabilities, which may lead to overestimation or underestimation of the incidence. As time-to-event analyses are incorporated into future studies, it will be important to consider the potential impact of competing risks on estimates [15, 16]. Recurrent prolapse and mesh exposure/erosion are potential competing risks that are not independent of the choice of procedure, and failure to account for this lack of independence in traditional survival analysis can lead to biased estimates; Markov models are one approach to dealing with this problem [17, 18].

The main strengths of this study included the use of a widely accepted modeling approach, and formal meta-analytic methods for generating estimates of probabilities. Models are especially useful research tools when, as was the case here, the available evidence is limited, since they can quantify the clinical impact of uncertainty in the data available to providers. Models are also very useful when more than one outcome is relevant to clinical decision-making, since variable rates of outcomes can be assessed in relation to one another. By performing Monte Carlo simulations we were able to examine and account for variability of follow-up times after the incident vaginal surgery for apical prolapse repair with limited published outcome data. Although the quality of the underlying data creates substantial uncertainty about the true estimate of event rates, we believe the use of formal meta-analytic methods using estimates of incidence from individual studies is the most appropriate approach where feasible; the ready availability of a validated, user-friendly Excel spreadsheet for performing these analyses should facilitate future efforts [11].

The limitations of our study, in addition to those inherent in the available data, include the necessary simplifying assumptions associated with any model. These included a lack of adjustment for individual factors that may have affected postoperative reoperation rates, and the assumption that each outcome was mutually exclusive. Mesh exposure/erosion as well as recurrence of prolapse following vaginal surgery for apical prolapse repair are multifactorial processes involving individual patient characteristics (i.e. age, weight, genetics, preoperative stage of prolapse, tobacco use), environmental factors (i.e. infection, use of vaginal estrogen) and surgery-specific characteristics (i.e. graft characteristics, surgical technique) that may each contribute to the overall risk of each outcome [6, 9, 10]. As there is relatively little known about the specific influence each of these characteristics has on the outcomes of interest, we simplified the model to examine the two specific outcomes that are assumed to be highly relevant to a general population of women undergoing apical prolapse repair. When considering management of mesh exposure/erosion, it appears that operative intervention is commonly used, and may even be the favorable option [6, 19–24]. However, mesh resection, depending on the amount removed or the extent of dissection, may increase the subsequent risk of prolapse recurrence and, thus, place the patient at additional risk of repeat surgery [20, 25]. Currently, we do not have a clear understanding of which outcomes are the most important to patients considering surgery for prolapse repair. The importance of the recent focus on patient-centered outcomes is particularly relevant to this analysis. One advantage of simulation modeling is that the potential impact of these factors on the probability of relevant patient outcomes, and thus the harm/benefit trade-off, can be explicitly included to help identify which factors are potentially most important to help guide future research. Thus, further research into patient preferences related to prolapse surgery and perceptions surrounding reoperation for mesh-related complications or recurrent prolapse are needed and, if these aspects can be clarified, will be extremely beneficial for future modeling studies.

Based on our review and synthesis, there is considerable uncertainty about the balance of benefits and harms of traditional NT versus TVM for repair of apical prolapse, making decisions regarding surgical treatment options difficult for patients and clinicians. The Pelvic Floor Disorders Registry should ultimately help provide higher quality evidence (http://pfdregistry.augs.org/. Accessed 19 November 2014), and the use of models such as this one may help future planning for both the Registry and other studies. Synthesis of future studies would be greatly facilitated by establishing standards for reporting, which would be used by investigators and encouraged by journal editors, and by focusing on time-to-event analyses for long-term postoperative outcomes.