AUTOPSY OF A CRIME LAB: EXPOSING THE FLAWS IN FORENSICS by Brandon L. Garrett University of California Press (2022)
With the exception of DNA analysis, a great deal of so-called “forensic science” — that is, the analysis of tool marks, bite marks, hair comparisons, fingerprints, blood spatters, arson patterns, and crime scene investigation in general — is not science at all. Its assumptions have rarely been tested in any rigorous fashion, its methodology relies heavily on subjective impressions, and its results are often unreliable. But by clothing itself in the trappings of “science,” it conveys to judges and juries an undeserved impression of certainty that not infrequently results in the criminal convictions of innocent persons.
While the proof of these inadequacies has been accumulating for over a decade, most judges (let alone the average citizen) seem woefully ignorant of these shortcomings and continue to accept most of these kinds of science as scientifically reliable. But a new book by Duke Law professor Brandon L. Garrett, Autopsy of a Crime Lab: Exposing the Flaws in Forensics, may change minds: Its analysis of the shortcomings of most forensic sciences is nothing short of devastating.
Garrett’s book does not simply detail many of the heartbreaking cases — now numbering in the hundreds — in which flawed forensic science directly contributed to wrongful convictions (usually not overturned until after the innocent defendants had served many years in prison). He goes further, explaining why the forensic experts got it wrong in the first place. While sometimes this was the product of the experts’ biases, inadequate training, or blind faith in their own expertise, more often it was the result of the inherent unreliability of the “forensic science” on which they relied.
Take, for instance, the 1982 death-penalty trial of Keith Allen Harward for breaking into a home, murdering the homeowner, and raping his wife. Examining teeth impressions, no fewer than six “forensic odontologists” unanimously opined that medical science proved conclusively that it was Harward’s teeth, and only Harward’s teeth, that could have bitten the wife’s legs in the course of the rape. This, they testified at trial, was not just a probability, but had been established “with all medical certainty.” Indeed, they testified, it was “a practical impossibility” that anyone else’s teeth could have made the marks.
Such testimony was standard practice in the field of “forensic odontology.” Indeed, for decades, the American Board of Forensic Odontology had instructed its members that their “science” permitted them to testify that their opinion that a bite mark came from the teeth of a given defendant was a “virtual certainty” and that there was “no reasonable or practical possibility that someone else did it.”
But in actuality, as a report of the National Academy of Sciences would later conclude in 2009, forensic bite-mark analysis is highly problematic. First, it assumes that every human being’s dentures are different, an assumption that has never been empirically tested. Second, it assumes that teeth that bite into skin leave marks that can be reliably correlated with the originating teeth, another assumption that has never been seriously tested and that ignores a host of confounding factors such as (a) because the victim and the biter are typically moving when the bites are made, rapidly shifting forces can affect the shape, size, and other features of whatever marks are left, and (b) because of the human skin’s elasticity, bite marks left on the skin typically change rapidly over time. Third, forensic odontology suffers from the fact there are no established standards for assessing which features of the marks are properly considered for determining a “match” and which can be ignored as caused by factors other than the bite. In short, the entire process is vague, subjective, and not anything like what a serious scientist undertakes.
But what the jury heard in Harward’s case was simply that forensic science proved conclusively that the bite marks on the rape victim’s body came from Harward and no one else, and so it was hardly surprising that the jury not only convicted him but also sentenced him to death. And while his sentence was eventually changed to life without parole, it was not until decades later that, with the help of the Innocence Project, DNA comparisons were made between numerous swabs taken from the rape victim’s body and the DNA of Harward and various other suspects. Those tests all showed a single DNA profile that clearly did not match Harward’s but that perfectly matched another suspect’s, who had subsequently died in prison after being convicted of kidnapping and other crimes. As a result, Harward was exonerated and released — after serving 33 years in prison.
Harward is just one of numerous defendants who have now been judicially exonerated after being convicted on the basis of bite-mark testimony. Yet, to this day, bite-mark evidence continues to be admitted in the courts of many states. Nor is such permissiveness limited to just this kind of doubtful forensic science. For example, after a number of defendants who had been convicted on the basis of “microscopic hair analysis” were subsequently exonerated on the basis of DNA, the FBI agreed to an audit of the nearly 3,000 cases in which FBI experts had testified that hairs found at the scene of the crimes matched the hairs of the defendants. The audit concluded that in no fewer than 96 percent of those cases, the FBI experts had given testimony that materially overstated the likelihood of the match. More generally, since at least 1984, scientists from around the world have repeatedly opined that it is not possible to scientifically determine whether hairs found at the scene of a crime come from a particular individual. Yet, despite such scientific opinions and in the face of its own audit, the FBI still insists that “microscopic hair comparison is a valid scientific technique.” And microscopic hair analysis continues to be admitted in evidence in a great many states. Indeed, in at least one state (Kentucky), it has been ruled to be “scientific” as a matter of law.
In 2009, as noted, the National Academy of Science issued a lengthy report detailing the many shortcomings of most forms of forensic science. The report — optimistically entitled Strengthening Forensic Science in the United States: A Path Forward — recommended the creation of a National Institute of Forensic Science, staffed by serious scientists, that would evaluate whether a given forensic science was solid or junk or somewhere in between, and would provide standards for its improvement. But this recommendation has gone nowhere. One of the reasons, Garrett suggests, is that most forensic laboratories are closely tied to local police, who see the National Academy’s proposal as an attack on their turf.
But what about the judges? After all, since the Supreme Court decided the case of Daubert v. Merrell Dow Pharmaceuticals Inc. in 1993, federal judges have been obligated to perform a “gatekeeper” function in deciding whether or not to admit scientific evidence, and most states have now adopted their own versions of Daubert, imposing similar obligations. But even though Daubert is frequently utilized in civil cases, it has proved, as Garrett notes, a near dead letter when it comes to assessing the admissibility of forensic science in criminal cases. In other words, most judges in the United States routinely admit all kinds of forensic science testimony in criminal cases without conducting more than a superficial preliminary review, if any, of whether it is good science or not.
Why is this? Some have speculated that it is because most judges lack a scientific background. But these same judges or their colleagues often conduct elaborate Daubert hearings in civil cases involving scientific issues far more complex than those presented by forensic science. Moreover, the Supreme Court’s opinion in Daubert itself provided judges with a list of relevant questions that almost any smart judge could easily propound and analyze without having any scientific background. They mostly involve simple questions of fact, including whether the “science” in question has been rigorously tested, whether it has been peer-reviewed in independent scientific journals, whether it has a known and only modest error rate, whether its methodology is consistently applied, and whether it has been generally accepted in the overall scientific community. In the case of all but a very few of the forensic sciences, the answer to all of these questions would be “no.”
So why haven’t judges, who are very familiar with how to resolve factual disputes, excluded more forensic science testimony in criminal cases? My own view is that, when it comes to criminal cases and all the emotions such cases typically engender, judges (a great many of whom are former prosecutors) are very hesitant to deprive the prosecution of evidence that may make the difference between conviction and exoneration. This is particularly understandable in cases of high visibility or where the judge is up for re-election; but, obviously, it negates any true “gatekeeper” function.
So, how might these and the many other deficiencies in forensic science detailed in Garrett’s book be cured? In his last chapter, Garrett presents a wish list of how forensic science might actually become scientific and reliable:
The sheer number and breadth of these recommendations illustrate how far removed our current forensic science is from reliable evidence and how far we need to go to fix this problem — to the extent it is indeed fixable.
Garrett is not blind to how much of a sea change must occur before his recommendations are likely to be implemented, either through national legislation (along the lines the National Academy of Sciences recommended in 2009), increased judicial scrutiny, or other measures. But he remains cautiously optimistic, in part because the city of Houston, Texas, has created a model forensic crime laboratory that has implemented many of Garrett’s suggestions. The reason Houston did so, however, was in reaction to a scandal — in which the previous Houston crime lab was found to be riddled with fraud.
Will it really require a series of scandals to motivate courts and legislatures to make the kinds of changes Garrett recommends? Or will it be enough to educate the citizenry about the overwhelming evidence that most so-called forensic sciences are terribly — and dangerously — deficient? Certainly, Garrett’s book is a good start toward raising such public awareness. But until this occurs, we must expect that many more innocent persons will wind up going to prison on the basis of flawed forensic science.
JED S. RAKOFF has served as a U.S. district judge for the Southern District of New York since 1996. He served on the National Commission on Forensic Science from 2013 to 2017 and as co-chair of the National Academy of Science’s Committee on Eyewitness Identification.
