Inadequate pain management remains a frequent problem after surgery. Despite the introduction of new techniques and analgesics aimed at reducing the severity of acute pain, a recent survey found that approximately 70% of patients experience moderate, severe, or extreme pain following surgery [2]. The consequences of inappropriately managed acute pain can be serious: increased morbidity and mortality and chronic pain, increased hospital stay and healthcare costs, and decreased quality of life [4, 7, 9, 25, 29–31, 34].

Enduring problems that may contribute to poor standards of acute pain management include inadequate information and awareness, as well as persisting misconceptions [3, 11]. Clinical guidelines, which aim to overcome these difficulties and help physicians to use hospital resources effectively to provide consistent and adequate healthcare, can improve patient outcomes [16]. Guidelines that are based on the highest quality evidence are important because translation of scientific evidence into clinical practice is extremely slow [21]. Such guidelines in acute pain management may be of most value when they consider the particular characteristics of pain associated with different operative procedures [15, 20]. Analyses of published data have demonstrated that analgesic efficacy can vary significantly between procedures [15, 24]. Extrapolation or pooling of data from studies in various types of surgery to provide assessments of analgesic efficacy may generate results that are not clinically intuitive for different operative settings where the intensity, type, character, and location of pain differ. Generalized guidelines based on such extrapolations consequently do not go far enough toward promoting optimal postoperative pain treatment. Moreover, generalized guidelines do not provide sufficient recommendations regarding the use of certain perioperative interventions that may apply only to particular surgical procedures, such as many regional analgesic techniques.

With the aim of overcoming these problems, we have developed the procedure-specific postoperative pain management (PROSPECT) initiative to provide a novel Web-based clinical decision support program [20, 32]. PROSPECT presents procedure-specific recommendations for postoperative pain management, formulated by the consensus of an international panel of experienced surgeons and anaesthesiologists. We derive the PROSPECT recommendations by interpretation of the evidence from a procedure-specific systematic review of the literature while considering the balance of risks and benefits for each intervention in the context of clinical practice. Analgesic, anaesthetic, and surgical techniques are reviewed for their effects on postoperative pain. Supporting evidence for each recommendation is provided on a website, allowing the end user to make informed clinical decisions on a procedure-specific basis.

Successful implementation of evidence-based recommendations for clinical practice depends upon the user’s degree of confidence in the evidence and recommendations. Building on experience from previous PROSPECT reviews, we have implemented changes to increase the rigor and transparency of the systematic review process and of the formulation of the consensus recommendations [28]. We present here the new methodology, which is first applied in full for the 2006 update of the PROSPECT review of postoperative pain management for laparoscopic cholecystectomy.

Materials and Methods

Methodology for PROSPECT reviews was developed and refined by discussion of the Working Group. Recognized methods for performing systematic reviews, for assessments of study quality and level of evidence, and for formulation of consensus recommendations [8, 12, 13, 17–19] were considered and used or adapted according to the specific requirements of PROSPECT.

Results

Methodology for procedure-specific systematic reviews of postoperative pain management

The PROSPECT review process begins with a systematic review of the literature, designed to answer the question “what is the evidence from randomized controlled trials that the different perioperative interventions available for use in procedure “X” are of benefit for the management of postoperative pain?” The methodology used is based on that recommended by the Cochrane Collaboration [18] (Fig. 1). A Subgroup, consisting of two or three Working Group members (and occasionally a clinician external to the PROSPECT group) with particular expertise in the operative procedure to be reviewed, is closely involved in designing the review strategy, together with a medical writing team.

Fig. 1.
figure 1

Procedure-specific systematic literature review of postoperative pain management.

Full size image

Literature search strategy and inclusion criteria

Literature search terms include words or phrases related to pain and possible interventions, in addition to procedure-specific terms. An information specialist performs the literature search in both the Embase and Medline databases. Two members of the medical writing team independently review each abstract and select papers that appear relevant to the systematic review. Selected papers are then examined in detail and included or excluded according to pre-set inclusion criteria: randomized controlled trial of perioperative interventions in the specific procedure, reporting pain on a verbal or numerical rating scale or visual analogue scale, and in English. Relevant systematic reviews are identified from the Cochrane Collaboration library [10], and secondary searches of reference lists from the systematic reviews are carried out to identify any additional studies.

Study quality assessment

Our process for assessment of study quality has recently been refined. The level of evidence for each included study is now indicated in the review text, so that the PROSPECT Working Group can see the overall quality of evidence when formulating recommendations. Study quality assessments are also made available to end users of the reviews to enable the members to judge the strength of the evidence-based recommendations.

Two members of the medical writing team independently assess study quality according to the following systems that focus on different aspects of study reports (Table 1):

  • Statistical analyses and patient follow-up assessment: indicates whether statistical analyses were reported, and whether patient follow-up was greater or less than 80%.

  • Allocation concealment assessment: indicates whether there was adequate prevention of foreknowledge of treatment assignment by those involved in recruitment (A adequate, B unclear, C inadequate, D not used).

  • Numerical scores (total 1–5) for study quality: assigned using the method proposed by Jadad et al. 1996 [19] to indicate whether a study reports appropriate randomization, double-blinding, and statements of possible withdrawals.

The two reviewers discuss discrepancies between their assessments to agreed final scores. Subgroup members validate the assessments and assign a level of evidence (LoE) to each study accordingly (Table 1). All randomized controlled trials are assigned LoE 1 or 2, depending on the source and quality of evidence. Failure to use adequate allocation concealment may distort estimates of effect [23], and so allocation concealment is important for determining the LoE (Table 1). For a study with allocation concealment score B (“unclear”) and a low numerical quality score 1–2, the Subgroup judges the LoE based on overall study quality (including an assessment of how closely the study report meets the requirements of the CONSORT statement [26]).

Table 1. Relationship between quality and source of evidence, levels of evidence and grades of recommendation in PROSPECT reviews
Full size table

Data collection and analyses

A member of the medical writing team extracts information from each included study, with like comparisons grouped together in tables for ease of analysis. The following information is always recorded if included in the published study:

  • Interventions compared and patient numbers in each group

  • Analgesia given to all patients

  • Qualitative outcomes for pain scores, supplementary analgesic use, time to first analgesia, postoperative nausea and vomiting

  • Other qualitative outcomes that are pertinent to the procedure-specific review

Qualitative analyses are performed for each group of studies reporting similar treatment comparisons and then reported in a summary Outcomes Document (Fig. 1). Quantitative analyses (meta-analyses) are also performed where possible (i.e. where study designs are homogeneous and outcomes are reported in a suitable manner), using Review Manager 4.2.2 software, which has been developed for Cochrane Collaboration systematic reviews [18].

Subgroup review

The Subgroup evaluates and discusses the evidence reported in the Outcomes Document (Fig. 2) and then agrees to include supplementary evidence from other procedures and clinical practice information, as necessary, before formulating a first draft of the recommendations, as described below.

Fig. 2.
figure 2

Subgroup evaluation of systematic literature review outcomes and drafting of the Review Document.

Full size image

Transferable evidence

Where there is limited procedure-specific evidence regarding the analgesic effects of a particular intervention, the Subgroup may agree to include supplementary “transferable” evidence from other procedures with a comparable pain profile (derived from other systematic reviews, where possible, or identified by comprehensive literature searching). Transferable evidence may also include published information about potential adverse effects of the intervention that is not covered by the procedure-specific review.

Clinical practice information

The Subgroup provides expert interpretation of the evidence in the context of current clinical practice, and pertinent information from clinical practice is included in the PROSPECT review. The following factors are considered [6]:

  • Ethical constraints

  • Clinical expertise

  • Patient preferences

  • Strength of results and consistency of evidence

  • Clinical relevance

  • Pathophysiology and clinical plausibility

  • Applicability to patient group

  • Practicality

  • Side effects

Recommendations

The Subgroup drafts procedure-specific recommendations for each available intervention, based on the procedure-specific evidence from the systematic review as well as transferable evidence and experience from clinical practice. The recommendations are graded to take account of the level and source of evidence (Table 1).

Preparation of Review Document

After the Subgroup meeting, a Review Document is prepared, containing the evidence and draft recommendations to be reviewed by the whole PROSPECT Working Group before formulation of the final consensus recommendations (Fig. 3). Each statement of evidence or clinical practice information is presented with a tick or cross to indicate whether it supports or does not support the use of that particular intervention. Each recommendation is supported by a summary of the evidence on which it is based and by the Grade of Recommendation to indicate the strength of recommendation. An Overall Recommendation is formulated for perioperative management of postoperative pain in the procedure under review, based on an overview of the balance of benefits and risks, and is presented as a flow diagram or table.

Fig. 3.
figure 3

Formulation of the final Review Document by consensus of the PROSPECT Working Group.

Full size image

Working Group review

The Review Document is circulated to each member of the PROSPECT Working Group for review. We have implemented a new, more structured process for the formulation of consensus recommendations (Fig. 3), based on established methods for achieving group consensus [12, 13], as described below. This process is designed to minimize potential bias toward the views of any one member of the Working Group and to encourage the expression of novel comments and ideas about the draft recommendations during the consensus process.

Collating comments: the Delphi Method

Members of the Working Group first comment on the review using the Delphi Method, which aims to obtain the most reliable consensus of opinion by individual questioning of the experts without direct confrontation [12]. Comments are forwarded only to a member of the medical writing team and not to the whole Working Group. This method avoids the potential for one Working Group member’s views to be adopted by the rest of the Group without full consideration of the data. Individual comments are then collated for discussion at a round-table meeting.

Formulating consensus recommendations by group discussion

At the round-table meeting, which is moderated by the medical writing team, the members of the Working Group discuss the draft recommendations. Collated comments from the Delphi Method round are shared and discussed until consensus is achieved. However, if consensus is not arrived at by discussion alone, a modified nominal group process is used to finalize the recommendations, whereby:

  • Each Working Group member expresses his or her comments/concerns about each recommendation, one after the other. At this stage, there is no discussion, agreement or disagreement from the other members

  • Comments are discussed

  • Each Working Group member votes to accept or reject individual comments

  • Further rounds of comments, discussion and voting continue until a consensus is achieved

  • Where full consensus of the Working Group is not achieved, a majority decision is taken based on a vote, and this is noted alongside the recommendation

During the meeting, the recommendations are presented as slides, and the moderator adjusts the wording of each recommendation in full view of the Working Group until the final consensus is reached. Transferable evidence and clinical practice statements may also be modified by agreement of the Working Group.

Preparation of final Review Document

The updated Review Document is circulated to the Working Group after the round-table discussions for final comments using the Delphi Method; these are incorporated according to the consensus of the Working Group (Fig. 3).

PROSPECT at http://www.postoppain.org

The final version of the Review Document is presented on the PROSPECT website (http://www.postoppain.org). Evidence and graded recommendations for perioperative interventions are contained within folders, with procedure-specific evidence, transferable evidence, clinical practice information, and recommendations clearly separated. A summary of the recommendations and details of the systematic literature review are also presented, including criteria for study inclusion as well as lists of included and excluded studies. The Web-based format offers a user-friendly way to present the large amount of information contained within each review, and it encourages users to submit feedback to the Working Group via the website. Every 2–3 years following a procedure review, the PROSPECT Working Group performs an assessment of newly published trials in that procedure to determine whether a change to the recommendations is warranted. If so, the review is updated accordingly.

Discussion

The PROSPECT initiative has developed a novel process for providing the medical community with accessible and clinically relevant evidence-based recommendations for postoperative pain management. Several surgical procedures have been reviewed to date (laparoscopic cholecystectomy [22], total hip arthroplasty [14], abdominal hysterectomy, open colonic resection, herniorraphy, and thoracotomy), with further reviews underway.

Potential bias in PROSPECT recommendations has been reduced by the enhanced rigor and transparency of the systematic review and consensus processes. However, some limitations remain. Owing to time and resource constraints and the breadth of the systematic reviews undertaken, we do not include papers published in non-English languages or unpublished studies, which could potentially introduce publication bias. It is also possible that studies published in journals not indexed for EMBASE or MEDLINE will not be identified. The analyses that we generate depend solely on published data; no attempt is made to retrieve patient data for additional meta-analyses. As a result, the reviews rely heavily on qualitative analyses, because outcome data suitable for meta-analyses are limited. Despite these limitations, we believe that the PROSPECT initiative offers a useful tool and a comprehensive reference source for clinicians.

To have clinical relevance, recommendations for health care must take into account the overall balance of risks and benefits. PROSPECT recommendations are formulated by an international group of both surgeons and anaesthesiologists who provide expert interpretation of the evidence from the different perspectives of their clinical roles. Although expert opinion on such factors as clinical practicalities, ethics, and safety issues may be subject to greater bias than evidence from randomized controlled trials, these issues are often not sufficiently covered in the literature, and they may be of overriding importance in determining the recommendations. PROSPECT users are made aware of the Grades of Recommendation and may judge for themselves the reliability of the evidence that is presented.

International dissemination of evidence-based recommendations is important if they are to have an impact on clinical practice, and peer-reviewed publication remains indispensable (for example the PROSPECT Working Group [22] and others, including Bisgaard [5] and Neudecker et al. [27], have published such recommendations on aspects of patient care in laparoscopic cholecystectomy). However, the availability of recommendations online further increases accessibility. Other evidence-based postoperative pain guidelines developed by working groups of experts, and also available on the Internet, include the “Acute pain management: scientific evidence” 2005 report by the Australian and New Zealand College of Anaesthetists and Faculty of Pain Medicine [1] and the “Management of postoperative pain clinical practice guideline” from the United States Veteran’s Health Administration in collaboration with the US Department of Defense and the University of Iowa [33]. The Australia/New Zealand guidelines cover a broad range of acute pain issues but are not procedure-specific, and although the US guidelines are procedure-specific and embrace a wide range of surgical settings, they provide incomplete details of the evidence used to formulate those guidelines. Although PROSPECT has reviewed only a limited number of operative procedures, details of all included references are provided, allowing users to consider the evidence and its relevance to their own practice. Thus these different approaches to development of guidelines and recommendations may be complementary, each offering valuable opportunities for improvement of postoperative pain management.

The primary aim of PROSPECT in postoperative pain management is to provide procedure-specific evidence and recommendations to support rational decision making, rather than to impose standards. Individual patient factors such as disease status, comorbidities, and psychological factors (e.g., anxiety, depression) affect postoperative pain and recovery, and they must be taken into account in choosing appropriate analgesia. Furthermore, practices and resources differ so widely between hospitals in different regions and countries that improvements to standards of care inevitably differ in nature and occur at different rates. The international nature of the PROSPECT Working Group is valuable because it provides a broad perspective on clinical practice. However, the presentation of detailed supporting evidence from a systematic review along with the recommendations is crucial as it enables the clinician to make informed decisions whether or not they implement the recommendations. We suggest that the PROSPECT initiative may provide a useful template for developing recommendations for clinical practice in other fields.