Real-World Data or Not So Accurate Data: OPDP Issues Letter to Company for Misplaced Reliance on Real-World Data

On July 7, 2021, the Food and Drug Administration’s Office of Prescription Drug Promotion (OPDP) sent an Untitled Letter regarding a professional animated banner.¹  According to OPDP, the banner ad made false or misleading claims and representations regarding the benefits of the prescription drug product. OPDP’s primary concern was the company’s use of a real-world study, which it found had many limitations. It did not raise concerns about the product’s safety. Therefore, the banner ad misbranded the product. This Bulletin will review the Untitled Letter, focusing primarily on the study, and offering our observations.

Highlights of Untitled Letter

OPDP found that claims and presentations included in the banner ad created a misleading impression regarding the statistical significance of the risk of febrile neutropenia (FN) depending on the delivery method for the drug. The real-world study cited to make the claims included “multiple limitations” that would preclude such a conclusion, including the following:

• The study was not designed to ensure that patients with FN were appropriately identified for inclusion in the analysis because the study used “an unvalidated algorithm” to identify study participants. The algorithm identified patients by selecting inpatient or outpatient encounters with diagnosis codes for neutropenia and fever or infection. The data on file provided to OPDP (the agency requested the study) did not include information regarding the performance characteristics (e.g., sensitivity) of the algorithm or the diagnostic codes that were used.
  • According to OPDP, the use of this unvalidated algorithm constituted a “significant limitation” because of the algorithm’s unknown performance characteristics and potential for misclassification of patients at the beginning of the study.
  • In addition, the data on file failed to describe “any measures taken to ensure the quality and accuracy of the results generated by the algorithm.” Therefore, the extent that patients in the study actually had FN could be over or underestimated.

• OPDP found that the study was not designed to ensure that the patients utilizing the different delivery methods “were adequately balanced or controlled for potential bias,” because the study failed to control for factors other than the delivery device that could influence the incidence of FN in the comparative groups.
  • The agency noted that the patients for the study had highly diverse clinical characteristics, but the study report did not include important information, such as risk factors for FN of the two groups or a design or analytic strategy to minimize the risk of selection bias. However, even slight differences in populations could substantially affect the incidence of FN and the conclusions of the study.
  • OPDP concluded that “it cannot be ruled out that selection bias is entirely responsible for the observed risk difference.”

• The agency also took exception to the use of the proper name of the drug in one instance, versus the use of the proprietary name of another product with a different delivery method, which could result in healthcare providers failing to understand that the company’s drug was used in both arms of the study. Healthcare providers could conclude that a biosimilar product is not as effective due to confusion from the use of the proprietary name only.
  • The misleading claims were also found to be particularly concerning from a public health perspective, as they could undermine the public’s confidence in FDA-licensed biosimilar products delivered through the method purported to be inferior, even though the study cited was “inadequately designed and precludes the drawing of conclusions.”

According to the agency, one frame of the banner, which included certain qualifying language “less prominently and in smaller font” than the claims and presentations set forth in earlier frames, was not sufficient to mitigate the more prominent presentation of other frames. OPDP also noted that two limitations to the study were presented in certain banner ad frames under the header “Real-World Study Limitations.” However, the agency found that the inclusion of the two major deficiencies of the study design did not mitigate the misleading claims and presentations in the banner. OPDP added that the misleading claims could undermine confidence in FDA-licensed biosimilar products, which are only delivered through a certain delivery method.

AGG Observations

• We receive several questions from clients regarding the use of real-world data in promotional materials. This Untitled Letter should serve as an example of the type of scrutiny and review OPDP exercises to ensure the appropriate use of such data.

• When evaluating real-world data and its appropriateness for use in promotional materials, companies should ensure all study limitations are considered and be comfortable that the conclusions drawn from the study are scientifically and statistically sound. It might be appropriate to add disclaimers and qualifiers, although this may minimize, but not eliminate, risk.

• The company may have looked to FDA’s 2018 guidance document, “Medical Product Communications that are Consistent with the FDA-Required Labeling,” when drafting its claims.²  There are some in the industry who may have taken the guidance as a suggestion that the agency might take a flexible approach to certain promotional claims that are not in the four corners of the product label, so long as the claims do not present a safety issue. The Untitled Letter reminds us that FDA is still out there and watching. The agency will take action if the promotion relates to efficacy and is misleading or otherwise violative, even if the claim does not present an immediate safety risk.

[1] The Untitled Letter is available here.

[2] The guidance document is available here. We also drafted a Bulletin on the 2018 guidance document, which is available here.