The Draft Guidance
In February 2023, CDER, CBER, and the Oncology Center of Excellence published a draft guidance titled “Considerations for the Design and Conduct of Externally Controlled Trials for Drug and Biological Products” (the “Draft Guidance”) to provide recommendations to those considering the use of externally controlled clinical trials to provide evidence of safety and effectiveness of a drug product.
An “externally controlled trial” is described in the Draft Guidance as a study in which “outcomes in participants receiving the test treatment according to a protocol are compared to outcomes in a group of people external to the trial who had not received the same treatment.” The external control arm can be either an “historical control” (from an earlier time period) or a “concurrent control” (from the same time period but in another setting). Because an external control can involve the use of real-world data (“RWD”), the Draft Guidance notes that this guidance is being issued to satisfy, in part, requirements from the 21st Century Cures Act on the use of real-world evidence (“RWE”) in regulatory decision-making, similar to other recent efforts in this arena that we have blogged about. This Draft Guidance focuses on the use of patient-level data from other clinical trials or from RWD sources. Notably, the Draft Guidance states that it is not intended to address other types of external controls, such as using summary-level estimates instead of patient-level data, nor does it discuss the use of external control data to supplement a control arm in a traditional randomized controlled clinical trial.
The Draft Guidance states that an external control group is potentially appropriate in a setting where the natural history is well-defined and the disease is known not to improve in the absence of an intervention or with available therapies. “For example, objective response rate is often used as a single-arm trial endpoint in oncology given the established understanding that tumor shrinkage rarely occurs without an intervention.” However, it also cautions that in many cases, “the likelihood of credibly demonstrating the effectiveness of a drug of interest with an external control is low, and sponsors should choose a more suitable design, regardless of the prevalence of disease.” Similarly, the Draft Guidance cautions that if the anticipated effect size is modest, an externally controlled trial may not be appropriate due to the concerns regarding the impact of bias and other limitations. The Draft Guidance also does not recommend using a non-inferiority approach for an externally controlled trial.
As highlighted in the Draft Guidance, a prominent feature of an externally controlled trial is the absence of randomization. A challenge caused by this is that confounding factors that can affect the measured outcome, such as certain baseline characteristics (e.g., demographic and related factors, disease characteristics, prognostic or predictive biomarkers, comorbidities, and treatments received), start of follow-up, and clinical observations collected may not be captured or similarly measured across groups. Additionally, for historical controls, changes over time to factors such as diagnostic criteria or methods of obtaining data should be assessed. The goal is to select similar patients in the treatment and external control groups. The Draft Guidance recommends that Sponsors confirm that recognized, important prognostic characteristics can be assessed in the data sources used for an externally controlled trial to make the populations as comparable as possible. Other challenges in the use of an external control include concerns over potentially important treatment imbalances between arms resulting in biases that were not documented or accounted for. Such imbalances can involve adherence, dose, initiation timing, handling of index date, treatment duration, receipt of additional treatments, and factors in health care delivery.
The Draft Guidance recommends that, where possible, the outcome should be assessed by individuals blinded to treatment status, which may require re-adjudication of externally controlled data. Well-defined, reliable, and meaningful outcomes that are typically used in clinical trials may not be available in RWD due to the lack of collection of relevant data in routine care or differences in strategies used to identify certain events. Outcomes of interest are more likely to be recorded in clinical records when the events are objective and require immediate medical attention (e.g., stroke). The Draft Guidance recommends that sponsors should also evaluate the consistency of timing of outcome assessments, which may be influenced by the patient’s clinical status outside the context of a clinical trial. Other challenges related to the selection of outcomes include the differential capture of intercurrent events, and the potential lack of standardization and training regarding clinical outcome assessments.
Decisions regarding study design and the statistical analysis plan (“SAP”) should be made in a blinded manner to any external control data, according to the Draft Guidance, with the exception of planned feasibility analyses regarding such things as availability of key variables or missing data. The analytic method should identify and manage sources of confounding and bias, including a strategy to account for differences in baseline factors and confounding variables, and missing and misclassified data. Sponsors should also propose additional analyses to evaluate the comparability between the trial arms for important covariates.
The Draft Guidance states that sponsors should consult with the review division early in development about whether an externally controlled trial is reasonable, and they should submit their SAP with the protocol before initiating enrollment. Specific design elements (such as data sources, baseline eligibility criteria, endpoints, and approaches to minimize missing data and sources of bias) should be prespecified in the protocol.
The Draft Guidance also states that sponsors must include in their marketing applications relevant patient-level data for both arms. If sponsors do not own the data used for the external control arm, they should structure agreements with the data owner to ensure that patient-level data can be provided to FDA.
Outstanding Questions
This Draft Guidance expands on the 2001 ICH E10 guidance (Choice of Control Group and Related Issues in Clinical Trials) and does a fairly thorough job describing the limitations and challenges facing sponsors seeking to use an externally controlled trial, particularly using RWD. As the determination is ultimately case-specific, the Draft Guidance does not provide many examples of scenarios where such approaches would be feasible and acceptable. This is potentially discouraging to sponsors of drugs intended for rare and serious diseases. If the Draft Guidance was interpreted to mean that applicability was limited to superiority trials in diseases with a well-understood natural history where the anticipated effect size is large, that would not address the reality facing many sponsors of drugs for rare diseases and potentially chill development of important drugs. Nor is this Draft Guidance entirely consistent with ICH E10, which does not, for example, entirely foreclose the use of an external control in a non-inferiority trial (“An external control study could be a superiority study . . . or a non-inferiority study.”).
Additionally, the Draft Guidance states: “Sponsors must include in their marketing applications relevant patient-level data (i.e., data on each participant and patient in the externally controlled trial), as required under FDA regulations, for both the treatment and external control arms.” The regulation that FDA cites as preventing it from considering summary-level estimates in an NDA is 21 C.F.R. § 314.50(f), which states in relevant part:
The NDA is required to contain tabulations of the data from each adequate and well-controlled study . . . . The tabulations are required to include the data on each patient in each study, except that the applicant may delete those tabulations which the agency agrees, in advance, are not pertinent to a review of the drug’s safety or effectiveness. Upon request, FDA will discuss with the applicant in a “pre-NDA” conference those tabulations that may be appropriate for such deletion.
However, as the Draft Guidance explicitly does not apply to the use of summary-level estimates in an external control arm, it is unclear how sponsors of NDAs seeking to utilize summary-level information available may best make use of such information and still comply with this regulation.
Summary-level information should be able to serve a regulatory purpose, and it has, in some circumstances. It is particularly important in the rare disease space, where patient-level data is often difficult to find. It is not unusual for summary-level estimates, such as data published by an investigator drawing on information from one (or a few) academic centers, to be the only data available for a particular disease or condition. It would be unfortunate to interpret the Draft Guidance to mean that these data would be entirely dismissed. FDA should acknowledge that there is value in such data, while remaining true to its regulatory constraints. For example, summary-level data can be very helpful in putting an observed placebo control or active control arm response into “real world” perspective. Rather than dismiss summary-level information entirely, the Draft Guidance should at least acknowledge that summary-level information may be used for other regulatory purposes.
Comments on the Draft Guidance are due May 2nd.
