Evidence-based medicine. Who decides admissibility? Part 2: Who is an expert?

Introduction

My last post raised the question of whether the rules and procedures for presenting evidence in a court of law would by analogy, help the field of evidence based medicine (EBM) which, in my view has few procedures for determining validity. Beyond analogy, courts are often charged with adjudicating questions of science or medicine. The evolution of rules of evidence in the American justice system has emphasized the role of general acceptance by expert witnesses, represented by the Frye standard, and the need for criteria beyond the opinion of authority. Daubert, the current standard, maintains multiple requirements emphasizing reliability and relevance. Expert opinion is still relevant but it not the major factor in Federal court although Frye remains a reigning standard in a small number of state courts. The influence of expert opinion is likely still a problem. In my analogy of the legal system to decision on evidence in nutrition and medicine, it may be the single major barrier to sorting out the crisis.

From the first, Frye had its critics.  Marcia Angell, former editor of NEJM and critic of wrote in Science on Trial that this was the birth of “general acceptance.”

Angell explains that the critics:

“claimed somewhat improbably, that it would tend to exclude novel, far-sighted testimony by modern-day Galileos. There is no record of this happening once, let alone often.  Furthermore, even if a modern-day Galileo did not make into court at first, that fact should not stop him from prevailing in the scientific community.  Courts do not determine scientific acceptance, as implied by the argument that we need to keep our courts open to the hidden Galileos in our midst.”

In fact, this happens often. The literature is compromised by a proliferation of  groups of experts in the medical community.  We argue that this pervasive majority opinion controlling editorial boards, granting agencies and academic departments is as powerful as the Catholic Church in repressing dissent.  In cases of controversial issues, the ADA, AHA and NIH may hold to strong opinions but dissenters have little chance against this scientific community and whatever their chances in court, the expense would be undoubtedly prohibitive. And it is not really general acceptance. It is acceptance by a dominant political or social body, regardless of how general their opinion is.

Rosen v. Ciba-Geigy

In 1992, Rosen, a 60 year old smoker, claimed that he had had two heart attacks that were a consequence of his having used a nicotine patch that had been prescribed to reduce his smoking. It turned out that he had already had a previous heart attack and had ignored advice to avoid smoking while wearing the patch. After wearing the  patch for three days and then removing it, Rosen suffered the heart attacks. He sued Ciba-Geigy, the manufacturer, for negligence in development and marketing. The trial court, in a summary judgement, ruled for Ciba-Geigy because it held that the opinion of Rosen’s expert, Dr. Henry Fozzard, was not admissible even though he was  "a distinguished cardiologist and department head at the University of Chicago.”

The (appeal) Seventh Circuit affirmed the decision because they said that Dr. Fozzard’s recognized standing was not enough to support his conclusion. An expert was expected to explain why a nicotine patch could precipitate a heart attack.

Scientific Gatekeepers

The need to go beyond expertise was the main point of Daubert v Merrell-Dow which held further that scientific evidence must be reliable and relevant. That meant peer-review, publication and analysis of error, in addition to general acceptance. Justice Blackmun who delivered the verdict in Daubert and others raised the question as to whether this would cast judges in the role of scientist which they were presumably not meant to be. This is still the subject much debate. In practice, in my opinion, we often see legal decisions on scientific issues that are distinguished by accurate technical understanding and are often articulate in explaining the import of the ruling.

Judge Richard Allen Posner, in affirming the decision of the trial court, said that Dr. Fozzard had failed to explain why wearing a nicotine patch or removing it could cause a heart attack. In fact, Dr. Fozzard did not offer “any experimental, statistical, or other scientific data from which such a causal relation might be inferred or which might be cause to test a hypothesis founded on theory." And the trial court also maintained that Rosen would not be able to. prove causality because his expert said he might have a heart attack anyway.

Judge Posner explained that the trial court’s decision was upheld following Daubert: “To reach this conclusion we do not have to become philosophers of science and set forth the necessary and sufficient conditions of ‘real’ science. When the Supreme Court in Daubert told judges to distinguish between real and courtroom science, it was not with the object of discovering the essence of ‘science,’ if there is such an essence. The object, we think… was to make sure that when scientists testify in court they adhere to the same standards of intellectual rigor that are demanded in their professional work.”

In the analogy to gatekeepers of the scientific literature, the requirement for scientific rigor should emphasize mechanism. In deciding to affirm the trial court, Judge Posner said that even if we assume that nicotine can contribute to plaque or even if we grant an extrapolation of animal studies to humans, the missing elements are:

“the distinction between the short-run and long-run effects of smoking, fatty diet, high blood pressure, diabetes, and the other well-known causal factors in coronary artery disease and hence in heart attacks. Wearing a nicotine patch for three days, like smoking for three days, is not going to have a significant long run effect on coronary artery disease; that much is clear..… That is an entirely different question from whether nicotine, or cigarettes, are bad for one's arteries.”

“Nowhere in Fozzard's deposition is there an explanation of how a nicotine overdose (for remember that Rosen was smoking at the same time that he was wearing the patch) can precipitate a heart attack.… Since Fozzard is a distinguished cardiologist, his conjecture that nicotine can have this effect and may well have had it on Rosen is worthy of careful attention, even though he has not himself done research on the effects of nicotine. But the courtroom is not the place for scientific guesswork, even of the inspired sort. Law lags science; it does not lead it.” (My emphasis).

The last sentence is often cited and has general import beyond the legal aspects. Your position or stature doesn’t carry any weight. The science has to carry itself. What you say has to be reliable and reproducible, otherwise it is not admissible. Editors follow the science. And, most the popular media must present the science, not lead ite.  Poor adherence to this principle is the source of the crisis in nutritional science.  Science must stand on its own

In practice in a court of law, evidence is often subject to a “Daubert challenge, “that is, that evidence conforms to the criteria that follow from Daubert.  One aspect of a Daubert challenge is conflicting evidence, particularly exculpatory evidence.  The first post on this topic presented the evidence (reproduced below) that red meat consumption and the incidence of diabetes went in opposite directions in the population.  The requirement for exculpatory evidence is universal in a court of law and is sensibly even more important in evidence-based-medicine.  That will be the next post on this subject.