What journalists should know before diving into drug investigations

Investigative journalists Jeanne Lenzer and Shannon Brownlee uncovered a trove of information about the recently approved Alzheimer's drug donanemab, (brand name Kisunla). Their deeply reported article in the September 25 issue of The BMJ highlighted questionable clinical trial results to potentially unethical behavior - including payments and industry ties - among some FDA advisory committee members.

In a lengthy Zoom discussion, Lenzer and Brownlee shared some tips on how other journalists could conduct similar investigations of drugs, medical devices, and controversial clinical studies.

Find independent sources

Journalists must speak with industry experts who have no ties to pharma. "If you don't have industry independent experts, you really can't go after these things," Brownlee said.

She suggested that if reporters don't have a list of these experts, they make their own or use one by The Lown Institute. Reporters can also look at the rapid responses to articles in the medical press such as the BMJ, JAMA, and other major publications; these are often independent critical voices that can be cultivated as sources.

Brownlee urged reporters to ask all sources about conflicts of interest. The Open Payments database, hosted by CMS, is a good starting point to learn about relationships between health care providers, pharma, and medical device manufacturers. "I think it's really important that journalists get into this habit, and they're probably going to get pushback," Brownlee said. That's a clue right there to be skeptical about whatever is being said.

Funky numbers and scales

The article pointed out that the primary endpoint used by Lilly in its trials, the integrated Alzheimer's disease rating scale, known as iADRS, was created by Eli Lilly in 2015. The FDA wanted the company to use the clinical dementia rating scale, sum of boxes, or CDR-SB test, in the donanemab analysis. Both tests measure scales of cognition and activities of daily living, but the CDR-SB is more commonly used in research settings.

However, according to the FDA, Lilly changed the primary endpoint from the CDR-SB to the iADRS during the study. As Lenzer and Brownlee explained, the iADRS scale showed a 2.92 difference between the drug and placebo groups, which is below the five-point threshold for clinical significance. The CDR-SB showed a statically significant, but clinically insignificant difference of 0.7 points.

Patients in the donanemab group worsened by an absolute difference of 2.9 points, or a relative difference of 22%, between the two iADRS scores. However, Lilly promoted the drug as slowing progression by 22%, which is misleading, according to the story.

"This is another reason to read those briefing documents," Lenzer said. "You can dig in and find the most damning stuff."

Additional advice for journalists

Lenzer also suggested that reporters should never assume honesty. While there is fraud and illegal behavior among drug companies, she worries less about that than word parsing. "Pay attention to terms like 'significant,' because people think that means something. Both doctors and patients fall for it."

Make sure you read briefing documents and other supporting materials. While these documents can be hundreds of pages long, it's critical to review them thoroughly to find buried key information that a company or agency may not want the general public to know about, such as underreported adverse events.

Also, pay attention to:

• The word "and," which can also cause a lot of confusion. "They used combination endpoints so they can say this drug saves lives when in fact, there were actually more deaths, not fewer," Lenzer said.
• "May" is another word to be aware of. If you read that a drug "may" save lives, you must also realize it may not.
• Absolute versus relative difference. Ask what the metric is and use analogies so your audience understands the comparison.
• "Surrogate" is also an important word. It is supposed to mean that the proxy endpoint has been shown to tie directly to the clinical endpoint you care about - such as reducing amyloid actually led to a slowing of dementia. That's what Lilly is trying to claim, but they only met the primary endpoint because the company changed it, according to Brownlee.

Journalists must be more demanding in this space given the size of the disease, the money involved, the fear that suffuses the public, and the history of sheer malarkey, Arthur Caplan, Ph.D., professor of bioethics and founding head of the division of medical ethics at NYU Grossman school of medicine's department of population health in New York City said. "AD is horrible. It leads some to fear it more than death. But we still don't understand it; real prevention does not exist, testing is still crude and treatment is still a hope, not close to a reality."