The United States Supreme Court decided two landmark cases that were supposed to change how a court admits scientific evidence: Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993) and Kumho Tire Co. v. Carmichael, 526 U.S. 137 (1999. These decisions were to transform the role of judges from passive rubber-stampers of accepted scientific processes to vigilant gatekeepers, passing on the validity and relevance of the science itself. Gloriously, forensic evidence was proclaimed to free the wrongly accused and catch the guilty. Somehow judges, most of which are not scientists, are provided evidence, frequently in the form of academic journals, by the lawyers about a legal dispute at trial.
Daubert and Kumho Tire have thus acquired a reputation as the antidote to a perceived indulgence for junk science in the courtroom: “Most of the state trial court judges surveyed believed that a purpose of Daubert is to guard against junk science.” C. Welch, Flexible Standards, Deferential Review: Daubert’s Legacy Of Confusion, 29 Harvard J. Of Law & Public Policy 1085, 1101 (2005-06). “Daubert was part of a three case trilogy that “dramatically tightened the rules for the admissibility of expert evidence in federal courts” to “crackdown on ‘junk’ expert testimony in federal courts.” D. Bernstein, Disinterested in Daubert: State Courts Lag Behind In Opposing ‘Junk’ Science, Legal Opinion Letter of the Washington Legal Foundation (June 21, 2002) Judges are believed to be charged with the duty to understand and act as a gatekeeper (most attorneys do not know that term appears but three times in Daubert) about scientific principles. Daubert even thought on occasion a jury will be prevented from learning of authentic insights and innovations because they were not. Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S at 597.
In contrast to Daubert, Kumho Tire referred to something akin to the expression “junk science,” but not in the majority opinion. It appeared in Justice Scalia’s concurring opinion and only once. Referring to the trial judge’s discretion to choose the “manner of testing expert reliability,” Justice Scalia said that a judge has “discretion to choose among reasonable means of excluding expertise that is fausse and science that is junky.” Kumho Tire, 526 U.S. at 159 (emphasis in opinion). In all of Daubert and Kumho Tire, there is no other reference to any concern about “junk science.”
Kumho Tire, like Daubert, lacks any language barring expert testimony. In neither case did the Court worry that trial judges were allowing juries to hear too many unreliable opinions or that the standard for admitting opinion testimony had been tightened or even needed tightening. The Court offered this simple explanation of the gatekeeping function: “to make certain that an expert, whether basing testimony upon professional studies or personal experience, employs in the courtroom the same level of intellectual rigor that characterizes the practice of an expert in the relevant field.” Id. at 152.
I am amazed at the naivety expressed by the Court in these decisions. It assumes intellectual rigor on the part of academics and scientists. That is a dangerous assumption.
As much as it is uncomfortable to admit, science is not always the dispassionate search for the truth. As detailed by a book by William Broad and Nicholas Wade in Betrayers of the Truth: Fraud and Deceit in Science (Oxford University 1985) science is not a strictly logical process, with objectivity the essence of scientist's attitudes and errors being speedily corrected by rigorous peer scrutiny and experiment replication:
Our conclusion, in brief, is that science bears little resemblance to its conventional portrait. We believe that the logical structure discernible in scientific knowledge says nothing about the process by which the structure was built or the mentality of the builders. In the acquisition of knowledge, scientists are not guided by logic and objectivity alone, but also by such nonrational factors as rhetoric, propaganda, and personal prejudice. Scientists do not depend solely on rational thought, and have no monopoly on it. Id. at 8-9. Emphasis added
The Broad and Wade present a series of case studies associated with the conduct of scientific research, from the manipulation of results to the total fabrication of whole experiments An example of how nonrational factors can come into play has been called the “Sokol affair.” Social Text is an academic journal published by Duke University Press. Since its inception by an independent editorial collective in 1979, Social Text has addressed a wide range of social and cultural phenomena, covering questions of gender, sexuality, race, and the environment. Since 1992, it is published by Duke University Press.
Alan Sokal, is a physics professor at New York University and University College London. In the May 1996 Spring/Summer "Science Wars" issue, Sokal had an article published in Social Text. The submission was an experiment to test the journal's intellectual rigor and, specifically, to investigate whether "a leading North American journal of cultural studies – whose editorial collective includes such luminaries as Fredric Jameson and Andrew Ross – [would] publish an article liberally salted with nonsense if (a) it sounded good and (b) it flattered the editors' ideological preconceptions". Sokal, Alan D. "A Physicist Experiments with Cultural Studies", Lingua Franca, (June 5, 1996).
The article, entitled "Transgressing the Boundaries: Toward a Transformative Hermeneutics of Quantum Gravity", is absurd. The article proposed that quantum gravity has progressive political implications, and that the "morphogenetic field" could be a cutting-edge theory of quantum gravity (a morphogenetic field is a concept adapted by Rupert Sheldrake in a way that Sokal characterized in the affair's aftermath as "a bizarre New Age idea"). It mocks the old-fashioned "dogma" that "there exists an external world, whose properties are independent of any individual human being and indeed of humanity as a whole", it proclaims categorically that "physical `reality', no less than social `reality', is at bottom a social and linguistic construct". By a series of stunning leaps of logic, it arrives at the conclusion that "the [Pi] of Euclid and the G of Newton, formerly thought to be constant and universal, are now perceived in their ineluctable historicity; and the putative observer becomes fatally de-centered, disconnected from any epistemic link to a space-time point that can no longer be defined by geometry alone". The rest is in the same vein.
Sokal himself exposed his pseudoscientific hoax article in the journal Lingua Franca. There are other examples.
A disquieting paper entitled, “The Criminal Justice System Creates Incentives for False Convictions” indicates that police, prosecutors, and forensic scientists often have an incentive to garner convictions with little incentive to convict the right person.. The authors Roger Koppl and Meghan Sacks, cite as an example one laboratory for which collection of court costs following guilty verdicts is the only stable source of funding. According to the paper, in Washington those found guilty following forensic evidence against them must pay a $100 fee, in Kansas the fee is $400, in North Carolina there is a fee of $600 for those found guilty following DNA evidence, similar rules apply in Alabama, New Mexico, Kentucky, New Jersey, Virginia, Illinois and Michigan.
It's not difficult to see how this situation creates a perverse incentive, but what make this case so incredibly worrying is how intrinsically vulnerable evaluation of forensic evidence is to bias. There is plenty of evidence to suggest that when a forensic scientist is given evidence about a case, their decisions regarding ambiguous fingerprint and DNA evidence can be swayed.
The paper cites a 2009 report by the National Academy of Sciences (NAS) which suggests that "the opinions of bloodstain pattern analysts are more subjective than scientific", other areas of forensic science that were assessed to rely heavily on subjective judgment included fingerprint analysis, handwriting comparisons, traditional hair microscopy, ballistics and impression evidence (e.g. comparisons of shoe and tire tracks). The report concludes that “[w]ith the exception of nuclear DNA analysis, . . . no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source.” Id. at 7. Soon after the report was published, the United States Supreme Court acknowledged that many forensic sciences are subject to “[s]erious deficiencies.” Melendez-Diaz v. Massachusetts, 557 U.S. 305, 319 (2009).
Thousands of cases are under investigation due to the lack of intellectual rigor by forensic scientists. 800 rape cases are under review due to one technician's poor work, another technician from a now closed lab may be responsible for thousands of wrongful drug convictions, hundreds have already been released and the investigation is still incomplete. A Minnesota lab was recently shut down after "a subsequent review by two independent consultants identified major flaws in nearly every aspect of the lab’s operation, including dirty equipment, a lack of standard operating procedures, faulty testing techniques, illegible reports, and a woeful ignorance of basic scientific principles.... (the lab) was run by a police sergeant with no scientific background, had no written operating procedures, didn’t clean instruments between testing, allowed technicians unlimited access to the drug vault, and didn’t have anyone checking anyone else’s work. Analysts didn’t know what a validity study was, used Wikipedia as a technical reference, and in their lab reports referred to “white junk” clogging an instrument.".