Introduction

Patient-reported outcomes (PROs) have evolved and recently gained their acceptance in health outcome research and clinical practice [1–8]. Although the definition of PRO is still evolving, it generally covers concepts, from the purely symptomatic (disease activity) to functional (physical and psychological) status to satisfaction with the therapy and adherence to treatment [7]. It is broadly defined to include any endpoint derived from patient reports, whether collected in the clinic, in a diary, or by other means, including single-item outcome measures, event logs, symptom reports, formal instruments to measure health-related quality of life (QOL), health status, adherence, satisfaction with treatment, and work productivity [9].

The proper use of PRO measurement in clinical settings has become increasingly important to inform clinical decision-making and guide treatment planning and management. PRO data may potentially improve the quality of patient care and reduce the health care costs by: (1) detecting patients’ problems with daily functioning and well-being early; (2) guiding therapeutic intervention and management; and (3) leading to patients’ improved QOL and satisfaction with care [10, 11] Morris et al. [12] report that 80% of surveyed oncologists believed PRO data should be collected prior to the commencement of treatment. Bezjak et al. [13] also report that 93.5% of surveyed oncologists would plan to incorporate PRO data in their practice. Furthermore, the need to know the patient’s perspective about treatment effectiveness is also reflected in the US Food and Drug Administration (FDA) draft guidance on PROs [7].

However, incorporating PRO information in research and clinical settings poses challenges [14]. Despite these potential benefits, PRO data are neither routinely collected nor utilized in clinical practice due to technological and logistical constraints [12, 13, 15]. It also calls for a range of reliable and valid instruments or measurement tools for a multitude of diseases or conditions so that the impact of diseases, efficacy of new pharmaceuticals or medical devices, and effectiveness of treatments or intervention programs can be appropriately evaluated.

Fundamental to high-quality health outcomes measurement and management is the notion that well-developed and validated assessment tools and systematic, yet efficient, disease and symptom monitoring and management programs will enable health care providers to improve the care quality and reduce the cost. The major unmet need is in the area of user-friendly, time-efficient, and cost-effective ways to reliably collect and accurately report clinically relevant data about a patient’s health status. Also needed is a psychometrically robust indicator of PRO that can be easily interpreted and understood by both physicians and patients. Well-designed and targeted PRO instruments, coupled with the state-of-the-art methodologies and cutting-edge technologies, can potentially become innovative clinical tools to make significant improvements in the delivery of high quality care possible. These efforts depend heavily on modern measurement theory and information technology.

In order to fully realize the potential benefits of PRO assessment in clinical practice, research needs to be undertaken to empirically examine the psychometric properties of the PRO instruments, their feasibility for clinical use, and the impact of routine PRO assessment on the quality of patient care. This article specifically addresses the challenges and solutions for accurate, time-efficient, and cost-effective PRO data acquisition and dissemination in clinical practice.

Challenges in incorporating PROs data into clinical practice

It is now commonly acknowledged that patients’ reports of their health and QOL, and their satisfaction with the quality of care and services, are as important as many clinical health measures. In spite of the potential applications and benefits of PRO data, incorporating this information for routine use in clinical practice has been challenging [16–19]. Major issues and barriers remain to be resolved, including feasibility, clinical relevance, cost and clinicians’ resistance. PRO information is not routinely collected or used because of lack of time, personnel, and infrastructure resources to collect and/or analyze the data; and perceived lack of an appropriate questionnaire [12]. Frequent complaints and concerns about current PRO measures is that they do not provide clinically relevant results [12, 20] the results are not readily available at the time of the consultation with the patients [15] and formal assessment of PRO will consume time that is not reimbursable [21]. Some physicians have even expressed the concern that including PRO makes a treatment decision more difficult for the patients [20, 21] and that such discussions may create an expectation that the clinician can influence PRO [10]. Further, physicians are concerned that routine PRO assessment may lead to discussions of topics on which they have received little training [21].

The promise of electronic medical records (EMRs) or personal health records (PHRs) to improve medical systems has been documented in many settings [22] but the potential for computerized information systems to lead to problems has also been seen [23]. In order for PRO assessment to be put into routine operation during clinical encounters in busy clinics, the measures must be brief and easy for the patient to complete, impose little or no burden on clinic staff to collect and analyze, and provide critical clinical information during the clinical encounter [10, 24]. Computerized PRO assessment and management could address many of these barriers, but only if the system is well designed, tested, and implemented. Computerized adaptive PRO assessment based on item response theory (IRT) [25] offers promises and has created a new practical way for enhanced PRO measurement and management. Six major challenges, identified from existing literature and our own work, to the implementation of a practical PRO assessment in clinical practice are briefly discussed. Some potential solutions to overcome these barriers and challenges are also discussed. Table 1 lists and summarizes the challenges and their respective solutions.

Table 1 Challenges of Implementing patient-reported outcomes (PROs) in clinical practice and their potential solutions
Full size table

Challenge 1: inadequate use of computerized delivery platforms

Despite the growing interest in applying a variety of cutting-edge technologies to capture PRO data (e.g., interactive voice response (IVR), touch-screen, etc.), most PRO measures are still often being administered in traditional paper-and-pencil (P&P) format. In principle, each technology or device has its pros and cons and no single PRO survey delivery platform is suitable for all situations. However, the slow adaptation of a broad range of proven technologies reduces the potential for providing a more personalized environment in which patients can report their health outcomes or participate in outcomes studies at any time and place using their preferred data delivery and acquisition platform.

Web-based surveys are becoming more popular as an alternative to conventional surveys done by papers or face-to-face interviews, however, their uses are limited to only those who can gain access to it [26, 27]. Although a phone-based IVR system would be a more acceptable way for most patients to collect PRO data either before, during or after the office visit, only limited information can be collected due to time constraints [28, 29]. For inpatients who do not possess sufficient physical strength or mobility to answer PRO questionnaires, a portable computing device such as a Tablet PC with voice recognition capacity would be a good alternative. These scenarios and circumstances highlight the importance of supporting multiple delivery platforms because of the heterogeneous patients populations and clinical settings.

Challenge 2: lack of clinical use of translated PRO measures for non-English-speaking patients

With the increasing culturally diverse patient populations in the US, having a data collection medium or platform that uses the language the patients speak or understand could increase the chance to identify their problems/issues and therefore proper care can be provided. In addition, as globalization of clinical research has increased, more studies include multiple countries, whose people speak various languages and have different cultural norms. In order to eliminate language barriers and to increase the chance of identifying essential problems, it is therefore a requirement that a PRO data acquisition system have the capability to display the survey items and allow for data entry in the languages of patients’ choice.

Given that PRO data are assessed by obtaining patients’ perspectives of the effects of their disease and treatment on their health, PRO instruments must be translated first and culturally validated from their original source language into the local languages participating in studies. However, some, but not all, PRO questionnaires have been translated into languages for use in non-native-English speaking patient populations (e.g., SF-36 [30], FACT-G [31] and EORTC QLQ-C30 [32]). Furthermore, an often ignored, but critical task for multi-lingual support is a rigorous translation process (e.g., forward translation, backward translation, reconciliation, and pilot testing) [33–35]. Translation tasks are often done in an ad hoc fashion, resulting in unreliable quality and/or delivery delays. The extent to which items in a questionnaire perform similarly across languages is of critical importance in comparative studies when determining whether data collected using translated questionnaires can be used an unbiased basis for comparing clinically relevant groups, once the data from all languages have been pooled.

Challenge 3: lack of comprehensive yet clinically relevant PRO measures

Physicians often desire clinically relevant questions that can be summarized and interpreted in a meaningful way for use during the medical encounter. However, many clinical issues and concerns (e.g., disease management, treatment side effects, treatment compliance, and patient satisfaction) are often assessed by very specific PRO measures that often encompass very limited topics. As such, comprehensive assessment can be time consuming for the health care professional and effort-intensive for the patient.

Integrative disciplines such as geriatrics and palliative care have particularly championed the importance of comprehensive assessments in patient care. Very sick populations often experience illness-related circumstances in which the mental, social and spiritual domains need to be considered in addition to the standard physical domain. However, the need to use multiple questionnaires for comprehensive assessment introduces extra challenges in PRO data collection and analysis in order to ascertain a comprehensive view of a patient. The lack of coordinated efforts to collect and standardize the PRO instruments has potentially hampered the adoption and integration of health outcomes assessment into mainstream health care practices.

Challenge 4: impractical patient and staff burden

Administering lengthy PRO questionnaires is time consuming and labor intensive and can potentially overburden patients. Lengthy instruments often frustrate and disinterest patients to respond because of the time needed for completion and may further irritate them because of content redundancy. Also, lengthy instruments can cause resource contention issues when administered at clinics (e.g., a patient may need to stay in the exam room longer to complete the assessment). Unfortunately, most existing fixed-length PRO instruments have more than 20 items (e.g., SF-36 [30]) and they typically overburden patients and may lead to their disinterest in taking assessments on a regular basis. The problem of patient overburden is exacerbated when a battery of questionnaires or multiple forms are administered in attempts to capture a wide range of PRO information in one session or on a regular basis. Also, many existing measures have significant overlap in their item content, which causes a more frustrating experience for patients when answering repetitive questions.

In addition, very sick patients are often ill enough that stamina is limited, and obtaining a comprehensive history and physical assessment is challenging. Although healthier populations might be able to tolerate a large number of questions about health status and other variables, a very sick population is less likely to have the physical and mental energy for such a task. In addition, extensive questioning can lead to a number of problems with research, including missing or inaccurate data due to fatigue or refusal to participate in subsequent data collection sessions.

Challenge 5: lack of clinically meaningful analyses and recommendations

It is not an uncommon perception that current PRO instruments, if not all, do not usually provide clinically relevant data [12, 13]. This is in part due to how the information about these measures is disseminated, often based on psychometric jargon that is not geared toward clinical audiences or patients. Clinical guidance is still greatly limited to that based on expert consensus rather than on firm evidence. Practice patterns still show wide variation and standards of care need to be set and systematized.

It is unsatisfactory to simply collect PRO data, present the results to clinicians and then expect them to adequately interpret and respond to such data in making clinical decisions. The challenges are enormous as to how to carry out clinically relevant analyses and provide succinct, yet critical, information or recommendations to facilitate better informed clinical decision-making.

Challenge 6: Health Insurance Portability and Accountability Act (HIPAA) compliance

Patients have become increasingly aware of the privacy issues concerning their PHRs. The recent HIPAA legislation has specifically addressed the privacy issue and has broad implications on most medical information systems [36–38]. Many of these HIPAA rules impose not only technical but also procedural requirements that would affect staffing and operational decisions for business. HIPAA compliance is especially important in the areas of patient privacy and data security and requires special system architecture considerations.

Solutions to the challenges of PRO implementation

An integrated computerized PRO measurement and management information system should meet the necessities of simplification and standardization of data collection and monitoring via well-developed methodologies and proven technologies. A system that is clinically useful, psychometrically sound and technically robust needs to integrate PRO data, critical clinical information, evidence-based medicine, and predictive modeling to provide individualized treatment guidelines on-demand to facilitate clinical decision making. We offer potential solutions below in hopes to overcome the above-mentioned challenges and to achieve the goal of fully integrating PRO measurement and management into clinical practice.

Solution 1: multi-channel delivery platforms in an integrated environment

It is vital to construct an integrated, yet expandable, PRO collection, analysis and reporting system (see Fig. 1) that supports multiple delivery platforms to allow for maximum flexibility in where, when and how to administer PRO assessments under a variety of settings and patient populations. It is of critical importance to construct a “fully” integrated back-end server system that can support a wide range of front-end administration platforms or devices for a number of reasons. First, from a usability perspective, a single, integrated solution allows a user to switch easily between different platforms when unanticipated situations arise. For instance, when a computer network is down unexpectedly at a clinic, patients need to be able to switch to phones (wired or wireless) to continue their PRO assessments without interruptions. Second, from a system development perspective, an integrated solution allows for maximum reusability of already developed critical system components, such as IRT-based engine, to potentially reduce the re-development cost. Third, an integrated solution provides a uniform, centralized process of streamlined secured access control to a single-point of entry, therefore maximizing protection of patient data and system integrity.

Fig. 1
figure 1

System architecture and functionality of a PRO measurement system

Full size image

A “multi-channel” PRO information system can also offer an array of delivery and access platforms to provide a much more flexible data acquisition environment where patients can choose when, where and how to report their health outcomes. For example, outpatients can use the Internet or dial into an “automated” IVR system to complete the PRO questionnaires. A health care professional can use a PDA to collect the responses from an inpatient at bedside and transmit the data back to the server either via synchronization cradles or wireless transmissions. The “multi-channel” approach, with a single generic application, can greatly reduce system development, deployment and maintenance costs and minimize device-specific implementations.

Solution 2: multi-language support to eliminate language barriers

A good PRO measurement-to-management solution must effectively deal with language issues to permit broader accessibility, particularly for culturally diverse patient populations who speak little English. Having a valid and reliable tool for measuring components of PRO in different languages adds value to cross-cultural outcomes research. One cannot assume that a translated PRO questionnaire to be cross-culturally valid by virtue of its translation without rigorous validation. The establishment of cross-cultural measurement equivalence between different language versions using both traditional psychometric testing and modern measurement theory, therefore, can provide evidence as to whether it is appropriate to pool the multi-lingual PRO data collected from a study for the purpose of comparative analysis of treatment effects.

Ideally, the availability of PRO instruments in different languages via rigorous translation methodology would potentially address disparities in health care. Also, a system can theoretically be designed to support the display and input of characters in different languages, dependent on the availability of the translated versions and the font support of certain devices, to allow patients to use their preferred languages to respond to the questions.

Solution 3: multi-level PRO item bank to broaden the PRO assessment to diverse diseases and medical conditions

A comprehensive PRO item bank that covers all the relevant domains (e.g., physical, mental, social, and spiritual) is a much more effective and economical approach that could benefit both patients and physicians. For example, different types of pain manifest differently, therefore, it is likely to have different sets of questions for customized assessment and management. The simplification and standardization of PRO data monitoring via computers in a way that is clinically useful, psychometrically sound and technically robust requires a comprehensive bank of PRO items shown to be relevant and sensitive to the concerns of the patients. That is, a well-designed PRO item bank should be fine-tuned to suit the needs of different disease sites, instead of being overly generic, although these site-specific item banks could share certain items that are of common concern.

With the IRT-based computerized adaptive testing (CAT) technology, [39] it has become more feasible to assess a set of clinically relevant domains with a reasonable number of questions [40, 41]. However, this requires a comprehensive bank of PRO questions shown to be relevant and sensitive to the concerns of the patients and physicians. The success of an operational CAT platform relies on the breadth and depth of the available questions or items in the item banks for optimal test administration. More than just a collection of items; an item bank is comprised of IRT-calibrated and thoroughly evaluated questions that develop, define and quantify a common theme and thus provide an operational definition of a latent trait (e.g., physical health). The items in the bank are concrete manifestations of positions along the continuum that represent differing amounts of the latent trait being measured. Not only does this allow for tailored, adaptive testing, it also allows one to compare the outcomes of patients across studies that have used different questionnaires.

A PRO item bank would also provide a basis for designing the best possible set of questions (a “test”) for any particular applications. The development of an Internet-based item bank management environment can potentially foster the collaboration among academic researchers, industry practitioners and government agencies in producing and maintaining PRO items.

Solution 4: adaptive PRO assessment to reduce patient burden and improve measurement precision

Physicians frequently request the minimum-required number of questions to obtain sufficiently relevant information about the patients. This is especially of great consequence for patients with serious illnesses, physical limitations or limited energy to answer all the required questions. The intelligent and dynamic selection of targeted questions based on a patient’s prior responses to reduce the required time is the key to making PRO assessment less burdensome. This can be accomplished via CAT that make use of IRT to estimate a respondent’s latent trait level (e.g., pain) after each response and to choose the next “most informative” item based on the current trait level estimation and prior responses. Each patient only needs to answer a subset of targeted items without sacrificing measurement precision and this can avoid unnecessarily long questionnaires. In fact, the capacity to measure all patients on the same latent trait continuum, even if they have not been administered any items in common, gives rise to the possibility of a PRO assessment that can be individually tailored to each patient. Since item and test difficulties are tailored dynamically to the level of each individual patient in CAT environment, it should not only yield measures that promote accurate selection and classification decisions, but also reduce patient boredom and frustration with floor (too easy) and ceiling (too challenging) items, respectively.

Solution 5: justifications of the benefits of routine collection and utilization of PRO data in clinical practice

Since the routine collection and utilization of PRO data in clinical practice is still a new frontier, it is reasonable to expect that top administrative executives and decision makers of health care organizations may be hesitant to invest in PRO measurement and management, especially when taking into account the time involved before, during, and after the clinical encounter. Also, clinicians are not customarily trained to utilize PRO information in their treatment planning. PRO data will be pertinent to various evidence-based clinical approaches to PRO management, while other source data will be related to the social, economic, and psychological concerns of patients who must face other treatment decisions. Having a readily available summary report that can be easily understood is essential to provide reminders or clinical advice specific to a given patient based on information entered into the system about that patient.

From clinical perspectives, these issues need to be evaluated: (1) whether the collected PRO data provide the clinician with new information about the patient; (2) whether the new information leads to changes in therapy; (3) whether the changed therapy results in better patient outcomes; and (4) whether the PRO data can be used for monitoring purposes and be predictive of conditions that warrant clinicians’ attention. From administrative perspectives, the topics that need to be carefully examined are: (1) whether the use of PRO information improves patient satisfaction about the care delivered; (2) whether physicians can work more efficiently and effectively when PRO data are available during clinical encounters; and (3) whether the collected PRO data can optimize the utilization of resources in the clinics.

Solution 6: HIPAA-compliant to safeguard patient privacy and enhance data and systems security

HIPAA legislation has broad implications on most medical information systems, and it is especially important when PRO data are used by health care providers for diagnosis or treatment decisions. Many of these HIPAA rules impose not only technical but also procedural requirements that would affect staffing and operations decisions. Among the various HIPAA requirements, the Privacy Rule and the Security Rule are the most relevant to outcome assessment and need to be compliant, from both technical and procedural perspectives, to safeguard patient privacy.

A PRO information system should be designed in compliance with HIPAA privacy and security requirements. For instance, an adequate access control mechanism, such as role-based, must exist to determine if the user has the privilege to execute an action on a particular set of data. Individual-identifiable data (e.g., names, street addresses, social security numbers, etc.) should be de-identified and clearly separated from other medical data and under strict access control, both technically and procedurally. Data transmissions between the front-end client and back-end server machines must be encrypted and protected using firewall technologies.

Components of a system for interactive assessment and management of PROs research and practice

We envision the PRO measurement and management information system to be a synthesis of measurement sciences, statistical modeling, evidence-based medicine, and informatics that allows research findings to guide clinical decision making in real time (see Fig. 2). It is designed to collect patient assessment information and make immediate comparisons against population data and practice guidelines for the purpose of offering clinical guidance. It also allows for a continuous accrual of information from all patients whose care is filtered through the system so that it automatically builds the population database.

Fig. 2
figure 2

Components of an integrated clinical decision support system for patient-reported outcomes

Full size image

This type of system, commonly known as a Clinical Decision Support System (CDSS) [42] for PRO measurement and management, should be designed to ultimately aid clinicians in making diagnostic and therapeutic decisions in patients’ care and to encompass at least four key components: (1) a CAT component that administers tailored assessments; (2) a statistical modeling component that develops predictive models from large databases; (3) an evidence-based medicine component that incorporates clinical practice guidelines; and (4) an informatics component that guides optimal clinical decision making.

A practical PRO measurement-to-management system that is patient-centered, methodology-derived, evidence-based, database-driven, technology-assisted and informatics-guided—and intended for use in diverse patient populations—is the essence of a CDSS for PRO measurement and management (the CDSS for PROs system or CDSS-PRO for short). Figure 3 further depicts how multiple sources of information, from the clinicians, patients and caregivers, can feed into the CDSS-PRO system through the use of CAT.

Fig. 3
figure 3

Infrastructure of an integrated patient care and research system

Full size image

The design of the CDSS-PRO allows questions/items and responses to be transmitted by whichever front-end assessment medium or device is available–telephone, Tablet PC, iPod, cell smart phone—with the information input being completed by multiple parties. For instance, a patient may enter data by telephone and a clinical staff member or nurse may then transfer those data into a computer terminal that interfaces with CDSS-PRO. Data can then be securely transferred to a central back-end server or data warehouse through mobile and fixed telecommunications technologies. These data can then be analyzed, for research purposes or with sophisticated statistical modeling to develop predictive models. The CDSS-PRO system interprets the patient-related data in real-time and matches it to clinical practice guidelines based on research performed using the predictive models to create advisory output to assist clinical decision-making among clinicians, caregivers, or patients.

Advanced computer technologies allow health outcomes data to be securely transmitted, summarized, and accessed by patients and their treating clinicians for timely clinical decision making at the point of service. The CDSS-PRO system can blend state-of-the-art automation with expert scripting (narrative) and customized (tailored) guideline-directed treatment recommendations. Information can be made available to the clinician in real time at the point of care. Physicians can easily access shared, up-to-date data (specially designed databases) needed to make treatment decisions, provide reminders and prompts at the time of a patient encounter, assist in establishing a diagnosis and in entering appropriate orders, and alert clinicians when new patterns in patient data are recognized. The well-organized PRO information derived from computerized assessment can be useful to the physicians by: (1) reminding the physician of relevant issues to bring up for discussion; and (2) providing critical information about prognoses, disease progression, etc. Conversely, the system may be helpful for alerting patients to possible topics that can be discussed with their physicians, and letting them know that it is appropriate to do so.

Decision support systems that present patient-specific recommendations in a form that can save clinicians time have been shown to be highly effective, sustainable tools for changing clinician behavior [42]. These technologies and methodologies are crucial for developing an effective telemedicine program, and such a program is especially well-suited to the underserved and other groups that lack access to health care resources.

However, designing and implementing an integrated PRO information system is challenging in clinical settings because of the required computing infrastructure, the need for reliable input and patient data, and the changes to existing clinic workflow that may result. Such technologies depend upon databases that integrate medical knowledge with patient characteristics, and then generate patient-specific recommendations or patient profiles. As computer-based records and order-entry systems become more common, automated decision support systems will be used more broadly [42].

Discussion

The creation of an integrated PRO measurement and management information system that utilizes methodologies and technologies requires innovative interdisciplinary efforts [43–45]. Experts from a variety of disciplines (e.g., psychometrics, informatics, medicine, statistics, outcomes research, etc.) who are knowledgeable in every aspect of PRO research and clinical practice need to be presented and to work together to design such a system that can combine multiple sources of information within proper methodological rules to enhance clinical decision making and PRO management.

However, given the limited range of clinical settings in which they have been tested, such systems must be evaluated rigorously before widespread introduction into clinical practice. A multidisciplinary group of experts then will be needed to evaluate these data sources and evidence derived from them, and make decisions on what evidence should be included in populating the core database. Decisions will be made on the basis of whether the various data sources appear to be valid and helpful in modeling patient factors that are dictated by the conceptual model, and appear to be associated with better uses of pain management resources, including healthcare and other types of services. Several principles will guide the process of constructing the core database, and evidence-based approaches will be used to the full extent possible. Several issues remain to be addressed:

  • Is an integrated PRO measurement-to-management system a useful supplement for physicians and patients who are (or could be) dealing with clinical decision-making?

  • Is this type of system acceptable to the clinicians and/or patients?

  • Can the processes of care be improved, from a professional perspective?

  • Are patients and physicians satisfied with the improved PRO management?

Summary

Patient-reported outcomes have taken center stage as the primary means of measuring the effectiveness of health care delivery. Proper measurement of PRO is essential. Multi-channel capability gives patients the freedom of choosing when, where, and how they wish to report their health outcomes and participate in outcomes studies. Multi-language capacity has the potential to increase the participation of non-native-English speaking patients in clinical trials and research projects. Multi-level PRO item banks can facilitate in compiling clinically relevant and psychometrically sound multidimensional PRO questions into a single standard repository to greatly simplify future PRO measurement tasks and improve the quality and comprehensiveness of PRO questionnaires. Adaptive testing capability allows the dynamic selection of fewer targeted questions to retain same level of measurement precision.

Although there remain several technical issues to overcome (e.g., user interaction, survey adaptation, data analysis, internationalization, data storage, etc.), the continuing implementation of such an integrated system as described above has the potential to improve the quality of patient care. Through the integration of adaptive testing, predictive models, clinical practice guidelines, informatics, and patient profiles, the integrated CDSS-PRO system can bring research and patient management together in real-time to help overcome some existing barriers to quality health outcomes management. This includes: (1) well-designed plans for data collection, warehousing, and mining; (2) proper analyses using IRT and computer-intensive methods; (3) user-friendly CAT and clinical decision support interfaces; and 4) a centralized information system. However, buy-in from clinicians who contribute to and use the CDSS-PRO system is another obvious requirement. Without additional resource supports at the institutional level, this CDSS-PRO system would not likely to succeed.

Through the use of sophisticated methodologies and proven technologies and input from the interdisciplinary team efforts, similar to those implemented in the PROs Measurement Information System (PROMIS; http://www.nihpromis.org/), it is envisioned that such an integrated system can: (1) establish cost-effective methods for capturing PRO data at a time and place convenient for the patients; (2) improve PRO measurement with adaptive capacity; (3) organize and present critical PRO and clinical information to both treating physicians and patients close to the time of decision-making action and in a manner that is sensitive and responsive to clinical needs, so increasing the impact of clinical guidelines; and (4) enhance disease and symptom management.