Introduction

The theme of the 10th Quality in the Spotlight Conference was the “NEXUS Vision.” NEXUS is the Latin word for cohesion, coherence, or context, and Dr. Henk Goldschmidt, one of the conference coordinators, suggested in his opening comments that in the context of “NEXUS” quality must “go beyond getting the ‘right’ answer on the ‘right’ patient that can be interpreted against ‘right’ reference values. Quality ultimately means being sure the patient is treated correctly.” For the NEXUS Vision to come to fruition, we must focus on “ensured quality” in clinical laboratories. We can no longer just hope that quality will happen as a result of following a series of prescribed protocols – rules, regulations, good laboratory practices, etc.

In the US, all clinical laboratory testing is regulated by the Clinical Laboratory Improvement Amendments (CLIA) [1]. CLIA links test quality and adherence to a body of testing regulations intended to ensure accurate, reliable, and timely patient test results. These regulations actually specify minimum requirements for personnel, quality control, quality assurance, and proficiency testing (external quality assessment). Testing sites also are inspected every 2 years to assess and ensure, through threat of fines and penalties, compliance.

The goal of the CLIA legislation was to ensure a minimum, fundamental level of quality – independent of the site (e.g., large reference laboratory, hospital, physician office) performing the testing. Despite the admirable intent of CLIA, 13 years later testing problems still abound. For example, a woman in Minnesota underwent a double mastectomy only to be told that her amputated breast tissue contained no malignant cells [2]. Her “normal” breast biopsy was switched with specimens taken from another woman. The Pennsylvania Department of Health found that several patients in a skilled nursing facility died as a result of being overdosed with Coumadin [3]. The test site used the wrong international sensitivity index (ISI) to calculate and report international normalized ratio (INR) values. Maryland General Hospital reported as many as 500 questionable HIV and hepatitis test results despite quality control values being outside of established tolerance limits [4]. The analysts simply altered the quality control results so that they were within acceptable tolerance limits. The US Institute of Medicine (IOM) estimates that 44,000 to 98,000 hospitalized Americans die each year due to “medical errors” and Newsweek, a popular weekly news magazine, reported that errors may actually result in as many as 195,000 deaths each year [5, 6]. Although errors due to laboratory testing were not specifically enumerated in the IOM report, laboratory results certainly play a role. More than 7 billion laboratory tests are performed in the US each year and the results generated provide approximately 70% of information used in health care.

The CLIA regulations and error prevention

CLIA regulations with specific minimum, performance requirements, or safeguards, are designed to prevent testing errors. For example, CLIA mandates that laboratories are to follow written policies for specimen labeling and to have a system in place to ensure patient data are reliable and accurate from order entry to final report. Laboratories must have written policies addressing quality assurance practices and require documentation and protocols evaluating new reagents before placing them into routine use. At a minimum, sites are required to assess the results of quality control materials with lot changes and major maintenance, review patient test results for inconsistencies with diagnosis, or pertinent clinical data, and examine the overall distribution of patient test results. CLIA regulations clearly state that quality control results must be acceptable or within established tolerance limits before reporting patient test results. This requirement implies a three-step process: (1) quality control limits are meaningfully set by the laboratory, (2) quality control specimens are run concurrently with patient samples, and (3) quality control results are assessed, evaluated as being correct and documented before reporting patient test results. Finally, CLIA in its quality standard §493.1407 insists that the laboratory director is responsible for overall operation and administration of the laboratory, which includes ensuring adequate laboratory staffing and adequate training of testing personnel and having testing systems that provide quality laboratory services for all aspects of test performance – pre-analytic, analytic, and post-analytic phases of testing [1].

Perception versus reality: What have we learned about quality under CLIA?

Despite the serious and inexcusable testing errors sited above, all three laboratories previously discussed passed inspections for CLIA compliance! As a profession, we want to believe, a priori, that following quality-based regulations such as CLIA improves laboratory test quality and positively impacts patient outcomes. This view is demonstrated by the proliferation of national, international, and professional “laboratory” standards. However, regulations do not absolutely ensure quality! While following regulations (e.g., adhering to basic quality control and quality assurance practices as opposed to doing nothing at all) MAY foster quality, regulations are not guarantees of quality. Moreover, inspections do not ensure quality. Testing sites clearly can have up-to-date, quality control practices in place, routinely analyze control materials, and dutifully record results. However, if the analysts consciously circumvent the intent of the requirements or ignore the quality assessment data, compliance can still be achieved while patients are poorly served. To achieve the NEXUS Vision in the Deming sense, “quality” is essential throughout the entire testing process [7] (Fig. 1).

Fig. 1
figure 1

Impact on patient test result quality

Full size image

Is technology the key to NEXUS?

The US Institute of Medicine found that testing processes fail as a result of human error, lack of documentation and lack of test management [5]. As a start, the report recommends error reduction through technology – computerized physician order-entry, barcoded patient wristbands and samples, analytical automation, smart technology to ensure analytical quality, intra-hospital computer systems that “talk” to each other, and direct physician interface with patients’ electronic records. A recent article in Clinical Laboratory News, recommends improving quality through “people power” – defined as skilled and dedicated laboratory professionals who undergo continuous education, training and competency assessment, work in a testing environment focused on patient safety, and adhere to quality management principles and on-going quality improvement plans [8]. While we concur with the concept of people power, the reality today in the US is that laboratory professionals are retiring, fewer students are entering the profession, more non-laboratorians are performing testing, and, in general, the laboratory staff has less knowledge of the “science” behind quality laboratory testing.

EQC: Minimum quality and minimum quality control

It is important to draw a distinction between US CLIA requirements and those in ISO documents, for example ISO 15189 [9]. Both are based on the concept that adherence to established quality processes is key to producing a product or service of superior quality. However, CLIA and ISO have a different emphasis. While the latest revisions to CLIA take a quality systems approach (like ISO 15189) to emphasize quality requirements that encompass the entire testing process – pre-analytical, analytical, post-analytical – CLIA seeks the lowest common denominator by mandating adherence to stated minimum quality practices [10]. On the other hand, ISO standards focus on excellence, requiring followers to be the best possible by striving for perfection. This difference is acutely evident with CLIAs latest attempt to lower the assessment standards of analytical quality through “equivalent” quality control (EQC).

In the latest (2004) interpretations of CLIA regulations, the minimum quality control requirement continues to be analyzing at least two external, liquid quality control materials per test per day [11]. However, these 2003–2004 revised rules and interpretations also open the door for routine use of “equivalent” or “alternative” quality control approaches – from electronic to using sophisticated internal quality algorithms. Testing sites using instruments with alternative, “built-in” (electronic, procedural, or internal) controls can choose to continue to analyze at least two external liquid controls per test per day OR qualify the “built-in” controls under one of three EQC evaluation options. For example, when the instrument's built-in control system evaluates, in the judgment of the laboratory director, the entire analytical process, the site, using EQC option 1, must analyze two external quality control materials daily for ten consecutive days. If the test site (the director) judges the “built-in” and external QC results to be “equivalent” or within acceptable limits and yielding similar information, the test site can then reduce the frequency of external quality control analysis from daily to once every 30 days. The key word in the CLIA regulations is “acceptable,” and unfortunately, CLIA offers NO insight. The January 2004 Interpretive Guidelines simply leave this decision to the test site director [11].

In our view, the major flaw in the EQC process is that during the 10-day evaluation period, absent an actual failure of the “built-in” control, we learn nothing about the instrument's quality assessment capabilities! The final CLIA stipulation for EQC is that if the 30-day external quality control check fails, the site must reevaluate patient results from the previous 30 days. Since the patient specimens are no longer available or viable and the patient has been treated based on the original results as reported, this requirement is illogical and/or unattainable.

For those laboratories choosing to implement EQC, we can offer some erudite, Web site comments from an “authority” in the laboratory field on ways to protect the test site choosing to use EQC [12]. Suggestions include relabeling EQC “equivocal” not “equivalent QC” and adding, “in God we trust” or “in George W. Bush we trust” on all laboratory reports. We concur with others that if we embrace the current EQC concept, we must accept the idea that “quality has nothing to do with science.” Instead, it is conceding that EQC is about saving money and appeasing analysts, since many do not want to, or have the time to, run quality control; many simply do not understand quality control. In addition, EQC may please some instrument manufacturers by giving their sales staff the additional selling point of cost savings through the decreased need for external quality control materials. It appears that the US government may have ascribed to the view that routine use of EQC is better than no quality control at all.

Could the EQC concept work?

Indeed yes! We proposed in 1995 that responsibility for achieving quality test results shifts from the sole purview of the laboratory director to an “alliance” of laboratory professionals, manufacturers, and regulators [13]. The EQC concept as proposed is a positive step in achieving this alliance, but not in the present form. With the obvious lack of scientific and statistical robustness, EQC falls far short of ensuring quality.

First to ensure quality we, as a profession, need to define the level of quality needed. Rather than relying on a vague statement referencing the arbitrary judgment of the laboratory director, quality goals should be based on medical (clinical, patient care) or quality needs. From these, we need to define laboratory performance requirements. Secondly, the manufacturers of an instrument system must be allowed to design the quality assurance algorithm. The manufacturer best knows the system's performance characteristics as well as how specific features mitigate possible sources of error. The manufacturer needs to be freed from the “traditional” quality control, calibration, calibration verification, etc., concepts included in the current CLIA regulations. These were designed to evaluate mid-20th century, first or second-generation testing technologies. Manufactures must design statistically robust experiments to test quality control systems, with appropriate measures, as a part of the system development process. This involves collecting extensive data to validate their alternative approaches. Third, regulators have to be prepared to accept, as they are apparently willing to do, alternative (equivalent or even better) quality control practices. Regulators are correct in demanding rigorous proof of an alternative quality control approach. EQC, as proposed today, falls far short in all respects.

But how can these alternatives be validated? Regulators claim to be understaffed (a euphemism for unqualified, unwilling, unable to judge) evolving test technologies and their alternate approaches to quality control. The answer is deceptively simple. A team of manufacturers and professional experts (perhaps chosen with input and/or participation from the regulators) could evaluate manufacturers’ data and render judgment. This approach is used for efficacy studies with proposed drugs.

Once approved for an instrument or system, the responsibility for quality control could be totally out of the hands of the analysts; meeting ensured quality requirements would become a matter of following manufacturers’ directions. Manufacturers, through design, can control the testing process to ensure full regulatory compliance. If defined quality requirements are met fully, the instrument would report patient data. If the requirements are not met, no patient data would be released. Several currently available test systems already meet this description. With these advanced test systems, analyzing the mandated external quality control materials each day, week, or month adds little if any additional value to the patient test result. EQC is a “right” idea; the current implementation concept is hopelessly flawed!

Finally, the confusion and concerns that manufacturers and professionals have with EQC are being addressed. In March 2005, the U.S. Centers for Medicare and Medicaid Services (CMS) and the Clinical Laboratory Standards Institute (CLSI formerly NCCLS) convened a “QC for the Future” conference that brought all concerned parties together. In response to the concerns, CLSI formed a committee to develop guidelines for EQC evaluation option 4 [14]. These guidelines will be for manufacturers to follow in validating their alternative quality control approaches and, if approved by the U.S. Food and Drug Administration (FDA), test sites would meet the CLIA quality control requirements by following the manufacturers’ directions. In many ways, the newly proposed EQC option parallels our “alliance” proposal and also sounds much like the original, and now defunct, FDA “clearance” provision specified in the original CLIA (1992) regulations and deleted in the 2003 CLIA revision [15]. In the meantime, the current Director of the CMS Division of Laboratory Services has assured CLIA inspected testing sites that inspection citations regarding EQC will continue to be “educational” until the government's quality control policy is clarified [16]. This suggests that laboratory directors are still free to choose to implement the current EQC protocols. The bottom line, in our view at least for now, is that testing sites inspected for CLIA compliance should follow manufacturers’ suggested quality control protocols including those for systems employing alternative procedures.

Conclusions

Achieving the “NEXUS Vision” for quality laboratory testing will not come solely from laboratory professionals. They will no longer be responsible for choosing the quality control protocol (algorithms, rules, criteria, etc.); performing the actual quality control testing and interpretations; or creating the documentation. These functions can be assigned to manufacturers and built into the test system to ensure an absolute level of defined quality in the test results. Professionals, however, will need to define the level of quality necessary and ensure the right test on the right patient at the right time with the right interpretation. The NEXUS, in 2005 and beyond, is not about how to quality control. It is about how to ensure that the full quality assessment of the testing process – pre-analytical, analytical, post-analytical – is in fact achieved.