A Guide to FDA’s Evolving Interpretation of How Sponsors Can Demonstrate Substantial Evidence of Effectiveness for Human Drug and Biological Products

Footnotes for this article are available for download in the formatted PDF at the end of this page.

A new drug’s effectiveness must be established by “substantial evidence,” which is defined in 505(d) of the Federal Food, Drug, and Cosmetic Act (FDCA).  FDA has interpreted the law as generally requiring at least two adequate and well-controlled clinical investigations, each convincing on its own, to establish effectiveness.   Nevertheless, under specific circumstances, the agency has been flexible, relying on a single large multicenter trial or previous findings of effectiveness (i.e., other adequate and well-controlled studies of a drug) to satisfy the legal requirement for substantial evidence of effectiveness.

In 1997, as part of the Food and Drug Administration Modernization Act (FDAMA), Congress confirmed FDA’s interpretation of the statute by amending 505(d) of the FDCA to clarify that the agency may consider “data from one adequate and well-controlled clinical investigation and confirmatory evidence” to constitute substantial evidence if FDA determines that such data and evidence are sufficient.  In 1998, in response to other provisions of FDAMA, the agency issued guidance on “Providing Clinical Evidence of Effectiveness for Human Drug and Biological Products” (the 1998 guidance).  The 1998 guidance described the quantity and quality (i.e., the extent of necessary documentation) of evidence necessary to support an effectiveness claim.

In December of last year, FDA unexpectedly issued a draft guidance on “Demonstrating Substantial Evidence of Effectiveness for Human Drug and Biological Products” (the 2019 draft guidance).  According to FDA, the 2019 draft guidance applies to New Drug Applications (NDAs), Biologics License Applications (BLAs), or applications for supplemental indications, and complements and expands on the 1998 guidance.  FDA explained that its evidentiary standard for effectiveness has not changed since the 1998 guidance; however, as a result of scientific advancements and changes in drug development over the last two decades, there is a need for additional guidance concerning the flexibility of the amount and type of evidence needed to establish effectiveness.

We will highlight the key points from the 2019 draft guidance:

The Quantity of Clinical Evidence to Establish Effectiveness

Meeting the substantial evidence standard based on two adequate and well-controlled clinical investigations

• Two adequate and well-controlled clinical investigations (“the standard”)
• One adequate and well-controlled large multicenter trial that can provide substantial evidence of effectiveness
  • A large multicenter trial is considered scientifically and legally to be multiple trials.  This is generally limited to situations in which the trial has demonstrated a clinically meaningful and statistically very persuasive effect on mortality, severe or irreversible morbidity, or prevention of a disease with potentially serious outcome, and confirmation of the result in a second trial would be impracticable or unethical.  In addition, there should be no single trial site that is the main contributor to the observed effect, either by virtue of having a much bigger effect or many more patients than other sites.

Meeting the substantial evidence standard based on one adequate and well-controlled clinical investigation plus confirmatory evidence

FDA considers the following factors when determining whether reliance on a single adequate and well-controlled trial plus confirmatory evidence is appropriate:  the persuasiveness of the single trial; the robustness of the confirmatory evidence; the  seriousness of the disease, particularly where there is an unmet medical need; the size of the patient population; and whether it is ethical and practicable to conduct more than one adequate and well-controlled clinical investigation.  Confirmatory evidence could include, for example, adequate and well-controlled clinical trials in a related disease area, certain types of real world evidence, compelling mechanistic evidence in the setting of well-understood disease pathophysiology, or well-documented natural history of the disease.  Sponsors considering this approach should consult FDA in advance.  Examples of when a single adequate and well-controlled clinical investigation, plus confirmatory evidence, can establish effectiveness include:

• One adequate and well-controlled clinical investigation on a new indication for an approved drug (generally a randomized concurrently controlled trial), supported by existing adequate and well-controlled clinical investigation(s) that demonstrated the effectiveness of the drug for its other, closely related approved indication(s)
• One adequate and well-controlled clinical investigation (generally a randomized concurrently controlled trial) supported by data that provide strong mechanistic support (i.e., earlier phase clinical results and/or testing that provide compelling mechanistic evidence in the setting of a well-understood disease pathophysiology; but could also be animal studies or a combination of human and animal studies).
• One adequate and well-controlled clinical investigation with compelling results supported by additional data from the natural history of the disease (e.g., a natural history study, case report forms, or registries).
• One adequate and well-controlled clinical investigation of the new drug supported by scientific knowledge about the effectiveness of other drugs in the same pharmacological class (from adequate and well-controlled trials of other drugs in the same class). Whether this scenario applies depends on a number of factors, including:  (1) the strength of the evidence for effectiveness from the single trial, and (2) the relevance of the additional data derived from other drugs in the same class, including the similarity between the new drug and other drugs in the same class, particularly the pharmacologic activity or specificity of mechanism of action.  Keep in mind that reliance on data concerning different drugs potentially raises legal issues.

Meeting the substantial evidence standard for a new population or a different dose, regimen, or dosage form, based on reliance on FDA’s previous finding of effectiveness of an approved drug when scientifically justified and legally permissible.

Additional efficacy trials are not required.  Other types of evidence (e.g., pharmacokinetic data) provide a way to apply the known effectiveness to a new population or a different dose, regimen, or dosage form.  For example, the effectiveness of a drug for pediatric use can sometimes be based on FDA’s previous finding of effectiveness of the drug in adults, together with scientific evidence that justifies such reliance. 

 

The Quality of Clinical Evidence to Establish Effectiveness

The 2019 draft guidance describes how the quality of clinical evidence is affected by the selection of trial design, trial endpoints, and statistical considerations.

Trial Designs

Although randomized double-blinded concurrently controlled superiority trials are usually regarded as the most rigorous design, the guidance reminds us that five types of controls are considered adequate and well-controlled:  placebo concurrent control, dose-comparison concurrent control, no treatment concurrent control, active treatment concurrent control, and historical control (a type of external control).  Specific guidance is provided on three trial designs:

• Superiority Trials – Establishing superiority to a concurrent control group (an active agent, including a lower dose of the test drug, or placebo) generally provides strong evidence of effectiveness because the design does not depend on assumptions regarding the effectiveness of the control.
• Non-Inferiority (NI) Trials – Showing non-inferiority (i.e., the new drug is not less effective than the active control by a specified amount) allows a conclusion that the new drug is effective. NI designs are appropriate only in situations in which the active control has shown a consistent effect (generally compared with placebo) in prior superiority trials conducted in a patient population similar to the population in the clinical investigation being planned.
• Externally Controlled Trials – These trials compare a group of subjects receiving the test treatment with a group of patients external to the trial. Because there is not random assignment or blinding, external control designs are usually reserved for specific circumstances, such as trials of diseases with high and predictable mortality or progressive morbidity (e.g., certain malignancies or certain rare diseases) and trials in which the effect of the drug is self-evident (e.g., general anesthetics).

Trial Endpoints

Although the statutory standard for effectiveness does not refer to particular endpoints or state a preference for clinical endpoints over surrogate endpoints, it is well established that the effect shown in the adequate and well-controlled clinical trials, must be, in FDA’s judgment, clinically meaningful.  Sponsors should consult disease specific guidances issued by FDA, and discuss with the appropriate review division early in clinical development, to identify appropriate trial endpoints.

Statistical Considerations

Trial results should be statistically significant or a high posterior probability of  effectiveness (Bayesian framework).  Statistical approaches should be specified in advance.

 

Examples of Clinical Circumstances Where Additional Flexibility May Be Warranted

FDA may rely on study designs that produce less certainty in some circumstances when a better design is not feasible or ethical.  This may be the case for life-threatening and severely debilitating diseases with an unmet medical need, for certain rare diseases, or potentially for a more common disease where the availability of existing treatments make certain design choices infeasible or unethical.  Therefore, when selecting a trial design, sponsors should consider the specific clinical circumstances, including the severity of the disease, unmet medical need, the rarity of the disease, and whether it is feasible and ethical to conduct a randomized concurrently controlled superiority trial.

When the disease is life-threatening or severely debilitating with an unmet medical need

• Trial Design – Trial designs such as non-inferiority trials or externally controlled trials can be acceptable if they provide substantial evidence of effectiveness.
• Trial Endpoints – Surrogate endpoints that are reasonably likely to predict clinical benefit, and effects on intermediate clinical endpoints, can be a basis to establish effectiveness for accelerated approval.
• Number of Trials – There are scenarios where the conduct of a second trial is not ethical or feasible (e.g., when a single large multicenter trial is considered sufficient to establish effectiveness, as discussed above).
• Statistical Considerations – The typical criteria for concluding that a trial is positive is a p value of <0.05. For a serious disease with no available therapy, or a rare disease where sample size might be limited, a somewhat higher p value, if pre-specified and appropriately justified, might be acceptable.

When the disease is rare

Many of the considerations discussed above also apply to development programs for rare diseases.  FDA recognizes that certain aspects of drug development that are feasible for common diseases may not be feasible for rare diseases.

• Trial Design – Randomized, placebo-controlled trials with equal allocation are generally the most efficient designs to assess effectiveness. However, sponsors should consider alternatives, such as unequal allocation in a randomized controlled trial (i.e., more patients receive the new drug than the control) or a dose-comparison design (i.e., randomization to more than one dose, with or without placebo).  Under certain circumstances, designs such as cross-over trials, randomized withdrawal, randomized delayed start, and a single arm trial with an external control should also be considered.
• Trial Endpoints – For many rare diseases, well-characterized clinical efficacy endpoints appropriate for the disease may need to be developed. In cases where utilizing clinical endpoints is not feasible because changes in symptoms and disease status occur too slowly to be measured in a clinical trial of reasonable duration, surrogate endpoints may be considered.
• Number of Trials – A second trial may be infeasible in certain rare disease settings where the limited patient populations preclude the conduct of a second trial. In these cases, substantial evidence of effectiveness is typically provided by a single trial plus confirmatory evidence.
• Statistical Considerations – As discussed above, FDA may interpret the substantial evidence standard flexibly. Statistical approaches to evaluating treatments for rare diseases should consider the feasibility of trial design, sample size, and endpoints, using methods and thresholds that are appropriate to these settings.

When conducting a human efficacy trial is not ethical or feasible

When it is not ethical or feasible to conduct clinical trials, FDA can allow the use of appropriate animal models to generate evidence to establish effectiveness for products  intended to treat or prevent serious or life-threatening conditions caused by exposure to toxic biological, chemical, radiological, or nuclear substances.

 

AGG Observations

• When sponsors are considering a drug development plan, they should review the information in the 2019 draft guidance and consult FDA before initiating clinical studies.
• The draft guidance suggests that it was issued in response to more drug development programs studying serious diseases lacking effective treatment, more programs in rare diseases, and more programs for therapies targeted at disease subsets. Although the draft guidance provides information on these particular issues, it clearly applies to all drug development programs.
• For sponsors planning to conduct two identical adequate and well-controlled trials, they should consider the following discussion in the guidance:

Although two positive identically designed and conducted trials can provide substantial evidence of effectiveness, precise replication of a trial … at times, can provide less persuasive evidence of benefit, as it could leave the conclusions of both trials vulnerable to any systematic biases inherent to the particular study design.  Two positive trials with differences in design and conduct may be more persuasive, as unrecognized design flaws or biases in study conduct will be less likely to impact the outcomes of both trials.

• Both the 1998 guidance and the 2019 draft guidance discuss the quantity and quality of clinical evidence to establish effectiveness. However, the 2019 draft guidance goes significantly further by addressing, for the first time, other characteristics of the quality of evidence supporting effectiveness that can vary, including trial designs, trial endpoints, statistical methodology, and examples of clinical circumstances where additional flexibility may be warranted.  By making FDA’s thinking more transparent, sponsors should have a better sense of what the agency will consider as substantial evidence of effectiveness.

Download