Evidence-based medicine. Who decides Admissibility? Part 1: General Background.

“Evidence-based-medicine” (EBM) is the self-serving term that arose around 1990. Like many such value-judgements turned fact (“healthy,” “high quality”), it represents what is needed rather than what we actually have. I have written about the subject — mostly in passing — but have a number of drafts for various outlets and will present these in the next few posts. Today in the Unsettled Science substack, Gary Taubes addressed the problem in the context of the Less Meat, More Plants: A Rules of Evidence Controversy and I will use the topic in my posts. The stimulus was the paper by Gu, et al. [1], described in my two previous posts. My idea here is that the confusion in nutritional medicine might be helped by an appeal to how evidence is seen in the legal system.

There is a tendency to assume that “evidence” means something obvious following from the etymology (videre, Latin, to see). In medicine, EBM is taken, as in Hamlet, as if it came from  the sensible and true avouch of our own eyes. In a court of law, however, you cannot say the data you present is evidence. A judge decides whether the evidence is admissible. Standards and precedents have evolved to guide them. Beyond admissibility, evidence must be subject to critical evaluation, notably by vigorous cross-examination: evidence must be accurate. Ultimately, a jury must decide if the evidence is compelling. While such procedures are nominally embodied in peer review and editorial oversight, that process has not succeeded well in practice. Editors, analogous to judges, are considered the “gate-keepers” but are often themselves, at the core of controversy.  

Overview. The Frye Standard.

The evolution of rules of evidence in law has largely been driven by application of expert testimony and the need to go beyond the concept of “general acceptance.”  Historically, a key case was the 1923 decision in Frye v. United States.  Frye had been convicted of second-degree murder but had appealed the judgement on the grounds that he had successfully passed a lie-detector test, at the time, a simple blood pressure machine.  The Appeals court ruled that that evidence had to be supported by a scientific community. The court upheld the opinion of the lower court, ruling that the lie-detector had not yet gained such “standing and scientific recognition.”

The limitations of Frye were obvious — what is “general acceptance?” Acceptance by whom? Taubes cites  Adam Cifu, a cardiologist, and John Mandrola, an internist and professor of medicine, who “approach medical science with an appropriately skeptical mindset … but described the ‘consumption of plant-based foods’ as unambiguously ‘good.’ No caveats necessary.” And, of course, Gu, et al., had the full panel of Harvard epidemiologists as authors. 

The Federal Rules of Evidence

The principle of general acceptance (by experts), however, persisted for many years and can still  be considered relevant. The need for standards led to congressional discussion which ultimately led to adoption of The Federal Rules of Evidence in 1975. The Rules stressed the need for relevance and reliability of any matter that was to be entered into evidence.  The Rules are frequently quite general and the major significance is to broaden the range of evidence that could be considered by juries.  Rule 401, for example, defined relevance as  “evidence having any tendency to make the existence of any fact that is of consequence to the determination of the action more probable or less probable than it would be without the evidence.” In other words, whatever helps.

Daubert

The major step forward, following from the need to expand criteria as embodied in the Federal Rules of Evidence, was the decision in Daubert v. Merrell Dow (1993).  Jason Daubert (pr. Dow-burt as in English, not like the Colbert Report) and Eric Schuller suffered from serious birth defects. They claimed that Bendectin, an anti-nausea drug that their mothers had taken during pregnancy, caused their birth defects. They and their parents sued Merrell Dow Pharmaceuticals, the manufacturer.  Merrell’s experts testified that there was no evidence in the medical literature to support the idea that maternal use of Bendectin was a risk factor. Plaintiffs’ experts cited animal studies, epidemiological studies and the chemical structure of similar compounds known to cause birth defects. They claimed that this was evidence for a causative role for Bendectin. Daubert’s evidence, however, was judged inadmissible by the District Court and ultimately the case was heard by the Supreme Court. The court ruled for Merrell but recognized that it had been “called upon to determine the standard for admitting expert scientific testimony in a federal trial.”

Justice Blackmun delivered the opinion of the court: Frye was superseded by the Federal Rules of Evidence. Interestingly, the court invoked the works of philosophers of science, particularly Karl Popper who is usually cited for codifying the idea that "the criterion of the scientific status of a theory is its falsifiability, or refutability, or testability."  Looking ahead, critics of the scientific literature suggest that there is a sense in which medical doctrines, like the diet-heart hypothesis, are not really falsifiable since contradictory results are often not incorporated into the body of the literature. Daubert cited two additional determinants of whether scientific evidence is admissible: whether it had been subjected to peer review and publication and whether it had addressed the known or potential rate of error.  General acceptance was still considered to be relevant but was no longer dispositive (sufficient by itself to decide the case).

Bendectin was ultimately removed from the market but the issues raised by Daubert — what it takes to demonstrate causation, what constitutes informed choice — has a voluminous literature. In future posts I will discuss how evidence in a court of law can be useful to set standards for scientific evidence. 

1. Gu, X, Drouin-Chartier, J-P, Sack, FM, Frank, FB, Rosner, B. Willett, WC. Red meat intake and risk of type 2 diabetes in a prospective cohort study of United States females and males. Am. J. Clin. Nutr.. (2023) 118, 1153-1163. DOI: 10.1016/j.ajcnut.2023.08.021.