The closeout phase starts with the final follow-up visit of the first participant enrolled and lasts until all analyses have been completed. It is evident that well before the scheduled end of the trial, there needs to be a detailed plan for this phase if the study is to be completed in an orderly manner. Importantly, one must be prepared to implement or modify this plan prior to the scheduled termination since unexpected trial results, either beneficial or harmful, may require the trial to be stopped early.

This chapter addresses a number of topics on the closeout process. Although many of them relate primarily to large single-center or multicenter trials, they also apply to smaller studies. The topics discussed include technical procedures for the termination of the trial, cleanup and verification of data, dissemination of trial results, storage of study material, and post-study follow-up. Obviously, the details of the closeout plan have to be tailored to each particular trial.

Fundamental Point

The closeout of a clinical trial is usually a fairly complex process that requires careful planning if it is to be accomplished in an orderly and effective fashion.

Termination Procedures

Planning

Many details of closeout will depend on factors that only become known once the trial is underway or participant enrollment is completed. Nevertheless, general planning for the closeout ought to start early. There are arguments for initiating this process on day one of the trial. Data management processes can be optimized for rapid database finalization at the end the trial. One major issue is that the trial may not continue through its scheduled termination. Greater-than-expected benefit or unexpected harm may lead to early termination. A more subtle reason is that developing plans for closeout after the trial is well underway may be interpreted by the blinded investigators as a signal of imminent trial termination. Thus, another recommendation is to develop the general closeout plans prior to the first meeting of the data monitoring committee [1].

The closeout phase needs its own written protocol or operating procedures with respect to termination activities, dissemination of results, and data cleanup and storage. The literature on the topic of closeout is scant but there are a few good descriptions of the process [2].

Scheduling of Closeout Visits

If each participant in a clinical trial is to be followed for a fixed period of time, the closeout phase will be of the same duration as the enrollment phase. If recruitment took 2 years, the closeout phase would last 2 years. This fixed follow-up design may not be desirable, since terminating the follow-up of some participants while others are still being actively followed can create problems. In some blinded trials, the code for each participant is broken at the last scheduled follow-up visit. If the unblinding must occur over a span of many months or years, there is the possibility of the investigator learning information that could suggest the identity of the drugs taken by participants still actively followed in the trial. This may happen even if drug codes are unique for each participant. The investigator may start associating a certain symptom or constellation of symptoms and signs with particular drug codes.

An alternative and frequently used plan involves following all participants to a shortened closeout period to avoid the problems described above. Another advantage of this design is the added power of the trial and more information about the effects of longer intervention. The follow-up period is extended beyond the minimum time for all but the last participant enrolled. In a trial with 2 years of uniform recruitment, the additional follow-up period would increase by an average of up to 1 year. In addition, this approach might be more cost-efficient when clinic staff is supported solely by the sponsor of the trial. With all participants followed to a shortened closeout period, full support of personnel can be justified until all participants have been seen for the last time. In trials where the participants are phased out after a fixed time of follow-up, an increase in the staff/participant ratio may be unavoidable.

Despite the problems with following all participants for a fixed length of time, this approach may be preferable in certain trials, particularly those with a relatively short follow-up phase and when the effect of the intervention is believed to be restricted to a short period of time. In such studies, there may be no realistic alternative. In addition, it may not be logistically feasible to conduct a large number of closeout visits in a short time. Depending on the extent of data collection at the last visit and availability of staff and weekly clinic hours, seeing 100–150 participants at a clinic may require a month or 2. A decision on the type of follow-up plan should be based on the scientific question as well as logistics.

Final Response Ascertainment

At trial termination, it is important in any trial to obtain, to the extent possible, response variable data on every enrolled participant. It is particularly so in trials where the main response variables are continuous, such as laboratory data or a performance measure. By necessity, the response variable data must be obtained for each participant at the last follow-up visit because it marks the end of treatment and follow-up. If the participant fails to show up for the last visit, the investigator will have missing data. When the response variable is the occurrence of a specific event, such as a nonfatal stroke or death, the situation may be different if the information can be obtained without having the participant complete a visit.

If a participant suffers an event after his or her last follow-up visit, but before all participants have been seen for the final visit, the study must have a firm a priori rule as to whether that response variable should be included in the data analysis. For the participants who complete their participation, the simplest solution is to let the last follow-up visit denote each participant’s termination of the trial. For participants who do not show up for the last visit, the investigator has to decide when to make the final ascertainment. If death is a response variable, vital status is usually determined as of the last day that the participant was eligible to be seen. The counting rule must be clearly specified in the study protocol or the manual of procedures.

Another approach is to have a common cut-off date (for example, the date of the first planned final follow-up visit). A common cut-off date—such as was used in the Apixaban for Reduction in Stroke and Other Thromboembolic Events in Atrial Fibrillation (ARISTOTLE) trial [3]—may be an advantage for clear definitions in the statistical analysis and for dealing with participants who may never appear for their final visits. In favor of a variable cut-off date (used in most trials) is that including all events until the last participant’s final visit allows maximal capture of the exposure period and therefore optimizes the precision of the estimate of the intervention effect. It is possible that the timing of the final visit could be affected by assignment to intervention or control and therefore could create bias when a variable cut-off date is used, although this would likely be small.

A number of means have been used to track participants and to determine their vital status. These include the use of a person’s identification number (e.g., Social Security number in the United States) or contact with relatives, employers, or health care providers. It has been discovered that participants have died through searching obituaries. Electronic medical records and various other electronic databases can be searched, with appropriate permissions. In countries with national death registries, including the United States, mortality surveillance is simpler and probably more complete than in countries without such registries. Agencies that specialize in locating people have been used in several trials. As has been used in many trials, the Digitalis Investigation Group trial [4] used a search agency, but the searches were limited to records only. It used directory assistance, credit header reports, property records, obituary searches, database mailing lists for magazine subscriptions, and other similar means. No personal contact was allowed. These constraints probably limited the success of finding participants lost to follow-up. This process can be sensitive, since a search may be looked upon as an intrusion into the privacy of the participant. The integrity of a trial and the importance of its results plus the participant’s initial agreement to participate in the trial have to be weighed against a person’s right to protect his or her privacy. Investigators should consider including in the informed consent form a sentence stating that the participant agrees to have her vital status determined at the end of the trial even if he or she has by then stopped participating actively or withdrawn general consent. It pays to initiate the process of obtaining information on vital status on inactive participants well in advance of the closeout phase.

The uncertainty of the overall results rises as the number of participants for whom response variable data are missing at trial termination increases. For example, assume that death from any cause is the primary response variable in a trial and the observed mortality is 15% in one group and 10% in the other group. Depending on study size, this group difference might be statistically significant. However, if 10% of the participants in each group were lost to follow-up, the observed outcome of the trial may be in question.

The Anti-Xa Therapy to Lower Cardiovascular Events in Addition to Standard Therapy in Subjects with Acute Coronary Syndrome–Thrombolysis in Myocardial Infarction 51 (ATLAS ACS 2–TIMI 51) trial of rivaroxaban following acute coronary syndromes [5] highlights the importance of completeness of follow-up data. The trial had a primary outcome of cardiovascular death, myocardial infarction, and stroke, using an on-treatment plus 30-day analysis with an intention-to-treat sensitivity analysis. In the original data from which the primary manuscript was published and the first United States Food and Drug Administration (FDA) filing was made [5], 1509 (or about 10%) of 15,526 participants had incomplete follow-up, and 799 participants had incomplete follow-up limiting the observation period of up to 30 days after early discontinuation. For the primary analysis, there was a 1.8% absolute and a 16% relative risk reduction with rivaroxaban (p = 0.008) that was counterbalanced by more major bleeding. There was a 0.8% lower mortality with rivaroxaban than placebo (p = 0.04) in the primary analysis. The FDA review raised important issues about missing data [6]. First, when the FDA declared that the 10% missing data made it impossible to interpret the mortality data, the sponsor was able to go back to the sites and establish vital status for 843 of 1338 patients with a missing status at the end of the trial, showing that it was possible to have more complete follow-up with more intense effort. When this was done, 22 additional rivaroxaban and 9 additional placebo participants were found to have died, and the p-value increased from 0.045 to 0.076 (for the main “stratum 2” with background thienopyridine therapy). Secondly, it was more common to have missing data in the rivaroxaban arm than in the placebo arm, raising further questions about interpretability. Third, there were three sites in India with questionable data that could neither be verified nor proven to be fraud, which raised additional questions about whether to exclude these data. To address the potential impact of the missing data with regard to mortality, the FDA presented a variety of imputation scenarios to address the missing data in stratum 2 [7] (see Table 19.1). Using the conservative approach of assuming the same mortality for all patients with missing data, the p-values increased from 0.076 to 0.100.

Table 19.1 ATLAS ACS 2–TIMI 51 trial mortality data with various imputations (adapted from an FDA slide presentation at a January 16, 2014 Cardiovascular and Renal Drugs Advisory Committee meeting [7])
Full size table

Another example of the problem of participants being lost to follow-up, and specifically of withdrawal of consent, comes from the Comparison of Medical Therapy, Pacing, and Defibrillation in Heart Failure (COMPANION) trial [8] of defibrillator versus pacemaker versus best medical care. Withdrawal of consent was four times higher in the medical care group than in the other two groups when the trial was terminated and follow-up ended. At a recommendation by the data safety and monitoring board, the investigators approached the participants who had withdrawn their consent and obtained their permission to collect data on vital status and hospitalizations retrospectively for the duration of the trial. This was done at a substantial extra cost and loss of time and stresses the importance of prevention of withdrawal of consent.

It is a mistaken concept that when a participant goes off study medication or intervention that he or she is out of the study and thus no longer followed, or at least not followed beyond some short period of time such as 7 days and 30 days. In the Adenomatous Polyp Prevention on Vioxx (APPROVe) study, participants who stopped their study medication (rofecoxib) due to adverse effects and other reasons were not followed beyond 14 days of going off drug [9]. In the re-analysis, the problem with this “informative censoring” was revealed and a full extra year of follow-up of all randomized participants after stopping study treatment was added. This analysis suggested that the excess of drug-induced major cardiovascular events observed during rofecoxib treatment continued to increase during the first year after treatment was stopped. The adjusted hazard ratio for the extra year was 1.41 (95% confidence interval [CI] 0.77–2.59), in addition to the hazard ratio of 2.12 (95% CI 1.20–3.74) on treatment and during the following 14 days.

The National Academy of Sciences, prompted by FDA officials, has published a comprehensive statement concerning missing data in clinical trials with a focus on phase III confirmatory trials [10]. The report states that “there is no ‘foolproof’ way to analyze data subject to substantial amounts of missing data; that is, no method recovers the robustness and unbiasedness of estimates derived from randomized allocation of treatments. Hence, the panel’s first set of recommendations emphasizes the role of design and trial conduct to limit the amount and impact of missing data” [10]. We stress the need to have systems in place from the beginning of the trial to minimize missing response variables, which for phase III trials begins with carefully structuring the informed consent to allow follow-up, at least for vital status, even if a participant otherwise withdraws from study procedures.

Transfer of Post-trial Care

Termination of a long-term study can be difficult due to the bonding that often develops between participants and clinic staff. The final visit needs to be carefully planned to deal not only with this issue, but also with the need in many trials to inform the participants of which medication they were on (in a blinded study), their individual study data, and the overall study findings (often at a later time). Referral of the participant to a regular source of medical care is another important issue (see Chap. 2).

If the closeout is extended over a long period, as it would be if each participant were followed for the same duration, any early recommendation to an individual participant would have to be based on incomplete follow-up data, which may not reflect the final conclusions of the trial. Moreover, any information given could “leak” to participants still actively treated, thus affecting the integrity of the trial. Although it is highly desirable to provide each participant with a recommendation regarding continued treatment, doing so may not be possible until the study is completely over and the trial results have been published. When unblinding occurs over a span of months or years, the investigator is in the uncomfortable position of ending a participant’s involvement in the trial and asking him or her to wait months before being told the study results and being advised about what to do. On the other hand, if the incomplete results are clear cut, it would be easy to arrive at such recommendations. However, in such an instance, the investigator would be confronted with an ethical dilemma. How can the investigator recommend that a participant start, continue, or discontinue a new intervention while keeping other participants active in the trial? For this reason, we generally prefer a shortened period of trial closeout.

Data and Other Study Material

Cleanup and Verification

Verification of data may be time-consuming, and it can conflict with the desire of the investigator to publish his findings as early as possible. While publication of important information should not be delayed unnecessarily, results should not be put into print before key data have been verified. Despite attempts to collect complete, consistent, and error-free data, perfection is unlikely to be achieved. Traditional monitoring systems are likely to reveal missing forms, unanswered items on forms, and conflicting data. In isolated cases, they may also uncover falsification of individual data [11, 12] and, in the worst cases, fabrication of all data on fictitious participants [13–15]. Data cleanup and verification typically continue for months after completion of the closeout visits, although the use of electronic records has substantially reduced the burden of this cleanup and verification. It is necessary to be realistic in the cleanup process. This means “freezing” and “locking” the files at a reasonable time after the termination of participant follow-up and accepting some incomplete data. Obviously, the efforts during cleanup should be directed toward the most critical areas—those crucial to answering the primary questions and serious adverse effects.

We strongly recommend that study forms and data be continuously monitored throughout a trial as pointed out in Chap. 11. Data editing should be initiated as soon as possible, because it is difficult to get full staff cooperation after a trial and its funding are over. Early monitoring may reveal systematic problems that can be corrected. Staff feedback is also important. Approaches for statistical process control audits are now available, and they have been shown to reduce the overall database error rates significantly [16].

Any clinical trial may be faced with having its results reviewed, questioned, and even audited. Traditionally, this review has been a scientific one. However, since regulatory and special interest groups may want to look at the data, the key results should be properly verified, documented, and filed in an easily retrievable manner. The extent of this additional documentation of important data will depend on the design of each trial. Electronic data provide verification opportunities that are more efficient than paper records, but storage remains important. Various models have been used. In one multicenter study, the investigators were asked at the end of follow-up to send a list of all deceased participants along with date of death to an office independent of the data coordinating center. In other trials, key data were independently audited before the results were published. Common to all models is an attempt to maintain credibility.

Procedures for data cleanup and verification in trials conducted for regulatory approval add substantially to the trial cost and complexity. Many such trials collect a large quantity of data. Final verification of these data is both time-consuming and costly [17, 18]. As noted in Chap. 11, investigators should, when designing such trials, both limit the amount of data and decide which data are essential and require full final verification.

Storage

Investigators should consider storing various kinds of material after a trial has ended. One set of documents—such as the trial protocol, manual of procedures, study forms, and the analytic material, including electronic records—should be kept by the investigator and sponsor. In addition, a list containing identifying information for all participants who enrolled in a trial ought to be stored at the institution where the investigation took place. Local regulations sometimes require that individual participant data such as copies of study forms, laboratory reports, electrocardiograms, and X-rays be filed for a defined period of time with the participants’ medical records. Storage of these data electronically clearly eases the problem of inadequate space. The actual trial results and their interpretation should be published and then can be retrieved through a library search, although it is all-to-common for trial results to remain unpublished [19]. As of 2012, less than two-thirds of National Heart, Lung, and Blood Institute (NHLBI)-sponsored clinical trials were published within 30 months [20]. Recognition of this major problem, pressure from sponsors and the clinical trial community to publish all trial results, and transparency and data sharing are all important steps to dealing with lack of publication (see Chap. 20).

In planning for a new trial, an investigator may want to obtain unpublished data from other investigators who have conducted trials in a similar population or tested the same intervention. Tables and figures in actual manuscripts seldom include everything that may be of interest. The situation is changing with online material available on journal websites. Many journals now publish full protocols, forms, manuals, and even raw data [21]. However, no uniform mechanism exists today for getting access to such study material from terminated trials. If information is available, it may not be in a reasonable and easily retrievable form. Substantial cooperation is usually required from the investigators originally involved in the data collection and analysis [22], and standards for data sharing and open access to trial data are evolving [23] (see Chap. 20).

The storage of biological material has raised new issues as it relates to genetic analyses. Biospecimens from well-characterized populations followed for long durations in clinical trials are in demand. These can be used to determine whether participant subgroups with a specific genotype are more likely to benefit or to experience serious adverse effects. The availability of these specimens for specific analysis depends on the wording of the informed consent (see Chap. 2). Patient privacy has to be considered, as always.

Storage of biomaterials may be costly. Freezers must be maintained, and a system for labeling and retrieving specimens or aliquots without damaging the remaining material must be implemented. Unlike with retrieval and distribution of data, many specimens may only be used once. Therefore, investigators need to develop a system for deciding when and how to use or distribute biospecimens. The cost and benefit, as well as the duration of storage must be considered. Central specimen repositories have been created to which investigators may be able to send their materials.

In summary, most trials collect an excess of study material, and it may not make sense to store everything. The investigator has to consider logistics, the length of the storage period, and cost. He also has to keep in mind that biological material, for example, deteriorates with time and laboratory methods change.

Dissemination of Results

The reporting of findings from a small single-center trial is usually straightforward. The individual participants are often told about the results shortly after the last follow-up visit, and the medical community is informed through scientific publications. However, there are situations that make the dissemination of findings difficult, especially the order in which the various interested parties are informed. Particularly in multicenter studies where the participants are referred by physicians not involved in the trial, the investigators have an obligation to tell these physicians about the conclusions, preferably before they read about them in the newspaper or are informed by their patients. In trials with clinics geographically scattered, investigators may have to be brought together to learn the results. In certain instances, the sponsoring party has a desire to make the findings known publicly at a press conference or through a press release. However, although an early press conference followed by an article in a newspaper may be politically important to the sponsor of the trial, it may offend the participants, the referring physicians, and the medical community. They may all feel that they have a right to be informed before the results are reported in the lay press. Companies may perceive a fiduciary responsibility to let the public know the “top line” results of a trial once they know them in order to control the risk of leaks.

We have had good experiences from the following sequence. First, the study leadership informs the other investigators who, in turn, inform the participants. Second, the private physicians of the participants are also told of the findings. Third, the results are then published in the scientific literature, after which they may be more widely disseminated in other forums. With journals now being available electronically, publication can be timed to coincide with presentation of the results at major scientific meetings.

However, there are sometimes unavoidable long delays between the presentation of trial findings at a scientific meeting and the publication of the full trial reports in peer-reviewed journals. The medical community may be placed in difficult positions by having to make treatment decisions if the lay press reports on elements of findings many months prior to the publication of the trial data in full. The messages released by the lay press are typically very simple. To minimize this problem, three recommendations have been made [24]: (1) “congress organizers should insist that published abstracts contain sufficient data to justify the conclusions of the presentation,” (2) “investigators should not present results of any study that is likely to influence clinical management until they are in a position to write a full paper,” and (3) “journal editors must be willing…to expedite the publication of such papers.” These recommendations are reasonable, but there may be exceptions.

In order to facilitate expedited translation of research results, the National Institutes of Health (NIH) introduced a data sharing policy in October 2003 [25] that has since been updated [26]. The agency’s position is that “Data should be made as widely and freely available as possible while safeguarding the privacy of participants, and protecting confidential and proprietary data.” The risk of wide dissemination of databases is that other investigators might analyze the data and arrive at different interpretations of the results. However, after a certain period of time has passed to allow for the trial investigators to analyze and publish, further analysis and discussion of various interpretations of trial data are usually scientifically sound and ought to be encouraged.

In special situations, when a therapy of public health importance is found to be particularly effective or harmful in a trial sponsored by the NIH, physicians and the public need to be alerted in a timely manner. The NIH would promptly post a release on its news website [27]. When the Adenoma Prevention with Celecoxib trial sponsored by the National Cancer Institute was terminated due to a 2.5-fold increased risk of major fatal and nonfatal cardiovascular events for participants taking celecoxib compared with those on a placebo, the release was issued the day after the decision was made to stop treatment [28]. Three months later, the results were published in The New England Journal of Medicine [29].

At the NIH, individual institutes may also issue their own press releases. These are generally released to coincide with the publication of an article in a medical journal. However, institutes, with journal permission, have issued brief press announcements prior to journal publication. To avoid criticism from physician groups, an institute may also notify the leadership of relevant medical societies before the release. The United States National Library of Medicine also releases timely scientific news on its MedlinePlus website [30]. These releases are not limited to NIH-sponsored research.

The FDA also informs physicians and the public about regulatory actions and news. FDA MedWatch Safety Alerts for Human Medical Products are posted on the website [31]. Included are brief summaries of products in question and FDA alerts. This and the general FDA drug website [32] provide recommendations and information for health care providers as well as information for patients to consider. If a serious adverse event has been uncovered by investigators in a trial, the FDA and other regulatory agencies or the trial sponsor may communicate this information to medical professionals, and thereby indirectly to the lay public, through a “Dear Healthcare Provider” letter.

Wide dissemination of trial findings to the public by investigators and study sponsors is increasingly common, even if the results are of modest scientific or public health importance. Press releases have become part of highly orchestrated marketing campaigns in both industry- and government-funded trials. We strongly support making trial results, and indeed data, widely available, with the expectation that broad discussion (and reanalysis as appropriate) will assist clinicians and the public in arriving at appropriate decisions as to the value of a trial’s intervention.

As emphasized in Chap. 1, clinical trials must be registered. Worldwide, there are a large number of registries [33–35]. Until the enactment of the FDA Amendments Act (FDAAA) in September 2007, the registration was limited to design information from the trial protocols [35]. The FDAAA expanded the scope to include a trial results database with information on participant demographics and baseline characteristics, primary and secondary outcomes, and statistical analyses. These data should be posted within 12 months of trial completion. The database should also be linked to publically available information from the FDA website. This would include summary safety and effectiveness data, public health advisories, and action packages for drug approval. Serious and frequent adverse effect data observed during a trial are to be added within 2 years.

Post Study Follow-up

There are three main reasons for short-term follow-up after completion of the intervention period. One is to find out how soon treatment-induced changes in laboratory values or symptoms return to pretrial level or status. The effect of the intervention may last long after a drug has been stopped, and abnormalities revealed by laboratory measurements or adverse drug effects may not disappear until weeks after the intervention has ended. Second, for certain drugs, such as beta-blockers and steroids, the intervention should not be stopped abruptly. A tapering of the dosage may require additional clinic visits. Third, clinical events may occur differentially in the study groups after the intervention is stopped due to lingering drug effects or to a hazard in switching patients back to standard of care [36]. Drug effects may be seen for weeks or months after treatment is stopped or there may be unfavorable withdrawal reactions [9]. These activities are separate from the moral obligation of the investigator to facilitate, when necessary, a participant’s return to the usual medical care system, to ensure that study recommendations are communicated to his or her private physician, and at times to continue the participant on a beneficial new intervention.

Long-term post-study follow-up of participants is a rather complex process in most, but not all, countries. The investigators and the sponsor have to decide what should be monitored. Mortality surveillance can be cumbersome globally but can easily be performed in selected regions, for example in Scandinavia. Usually, the justification for long-term post-study surveillance is based on a trend or unexpected finding in the trial or from a finding from another source. Since most clinical outcome trials of chronic therapies are relatively short in duration, extended follow-up can provide important additional information.

Obtaining information on nonfatal events is even more complicated and, in general, its value is questionable. However, a classical illustration that post-study follow-up for harm can prove valuable is the finding of severe adverse effects attributed to diethylstilbestrol. The purported carcinogenic effect occurred 15–20 years after the drug was administered and occurred in female offspring who were exposed in utero [37]. Similarly, use of unopposed estrogen has been reported to be associated with an increased risk of endometrial cancer 15 or more years after therapy was stopped [38]. One article reported an association between in utero exposure to valproate, an antiepileptic drug, and impaired cognitive function in offspring at 3 years of age [39].

In 1978, the results of a trial of clofibrate in people with elevated lipids indicated an excess of cases of cancer in the clofibrate group compared with the control group [40]. The question was raised whether the participants assigned to clofibrate in the Coronary Drug Project also showed an increase in cancer incidence. This was not the case [41]. Only 3% of deaths during the trial were cancer-related. Subsequently, a World Health Organization study of clofibrate reported that all-cause mortality was increased in the intervention group [42]. At the same time, Coronary Drug Project investigators decided that post-study follow-up was scientifically and ethically important, and such a study was undertaken. No increase in cancer incidence was noted in the clofibrate group [43]. A more recent example is the Women’s Health Initiative, which extended follow-up for 5 years after it reached its scheduled termination in 2005. This example brings up a question: should investigators of large-scale clinical trials make arrangements for surveillance in case, at some future time, the need for such a study were to arise? The implementation of any post-study surveillance plan has challenges. A key one is finding a way to keep participants’ names and addresses, or their Social Security or other national identification numbers, in a central registry without infringement upon the privacy of the individuals. The investigator must also decide, with little evidence, on the optimal duration of surveillance after the termination of a trial (e.g., 2, 5, or 20 years).

Another issue of post-study surveillance relates to a possible beneficial effect of intervention. In any trial, assumptions must be made with respect to time between initiation of intervention and the occurrence of full beneficial effect. For many drugs, this so-called “lag time” is assumed to be zero. However, if the intervention is smoking cessation, a lipid-lowering drug, or a dietary change, and if the response variable is coronary mortality, the lag time might be a year or longer. The problem with such an intervention is that the maximum practical follow-up may not be long enough for a beneficial effect to appear. Extended surveillance after completion of active treatment may be considered in such studies. At the scheduled termination of the Multiple Risk Factor Intervention Trial, the results favored the special intervention group over usual care but did not reach statistical significance [44]. Almost 4 years later, a statistically significant effect emerged [45]. A late benefit was also evident in a passive follow-up phase after stopping enalapril versus placebo in the Studies of Left Ventricular Dysfunction (SOLVD) [46].

The post-study surveillance in the Coronary Drug Project [43] showed unexpected benefit in one of the intervention groups. At the conclusion of the trial, the participants assigned to nicotinic acid had significantly fewer nonfatal re-infarctions, but no difference in survival was detected. Total mortality after an average of six-and-a-half years in the trial on drug, plus an additional 9 years after the trial, however, was significantly lower in the group assigned to nicotinic acid than in the placebo group. There are several possible interpretations of the Coronary Drug Project finding. It may be that this observation is real, and that the benefit of nicotinic acid simply took longer than expected to appear. Of course, the results may also be due to chance, a possibility that seems more likely with the lack of benefit and evidence of harm with niacin in the much larger Heart Protection Study 2–Treatment of HDL to Reduce the Incidence of Vascular Events (HPS2-THRIVE) trial [47]. A major difficulty in interpreting the data relates to the lack of knowledge about what the participants in the intervention and control groups did with respect to lipid lowering and other regimens in the intervening 9 years.

Knowledge of the response variable of interest for a substantial portion of participants is required if long-term surveillance after completion of regular follow-up is to be worthwhile. The degree of completeness attainable depends on several factors, such as the response variable itself, the length of surveillance time, the community where the trial was conducted, and the aggressiveness of the investigator. Many of the very large trials have successfully monitored participants (or subsets thereof) after closeout to determine whether behavioral effects of the study intervention have been sustained or participants have adhered to recommendations regarding continued treatment.