Experts with Statistics

Chimps with Machine Guns

“Fibonacci Sequence,” 9 pieces by Alanna S. Graboyes (asgraboyesart.com)

NOTE: I’m periodically posting excerpts from my not-yet-published book, “Fifty-Million-Dollar Baby: Economics, Ethics, and Health.” The goal is to edit the manuscript in plain view, and to ask for your comments, corrections, suggestions, and criticisms.

A highlight of my career has been to teach medical, business, and legal professionals at five different universities, with an emphasis on statistics and other quantitative concepts. While the courses and lectures have ostensibly focused on techniques and logic, the most important theme of my pedagogy was to drill into my listeners’ heads a singular and overarching message:

Without introspection, without skepticism, without ethics, without respect for opposing viewpoints, an expert armed with statistics is, to borrow a line from Better Call Saul, “like a chimp with a machine gun.”

Here are four examples, plus some baseball stats between to illustrate a concept.

1.       In 1989, after Leonard Mlodinow received a positive HIV test, a doctor told him he had a 99.9% chance of dying of AIDS within 10 years. He didn’t have AIDS, never faced such grim probabilities, and is still around.

2.       In 1999, Roy Meadows, pediatrician and professor, presented statistical testimony that sent solicitor Sally Clark to prison for the murder of her two infant sons—until his stats were revealed to be garbage several years later.

3.       For those who believe a hunt for statistical disparities should govern society, consider the fact that baseball statistics showed that in 1995, 1996, and 1997, David Justice had a higher batting average than Derek Jeter, but also that the exact same data show that Jeter had a higher batting average than Justice in those same years.

4.       In 2011, Nobel economist Paul Krugman used statistics to extoll unionized teachers and excoriate non-unionized teachers. Blogger/car expert/humorist Dave Burge (aka, Iowahawk) showed that Krugman was unwittingly advising states to have more white students and fewer black and Hispanic students.  

5.       At the University of California—Berkeley, statistics suggested systemic discrimination against female applicants. But, organized differently, the exact same data suggested a slight bias in favor of women.

Reports of his death were greatly exaggerated

In 2020, physicist Leonard Mlodinow wrote a memoir about his friend and collaborator, the late Stephen Hawking. Over 30 years earlier, in a curt telephone conversation, a doctor informed Mlodinow that he had tested positive for HIV, giving him a 99.9% chance of dying of AIDS within a decade. Mlodinow described this incident in his book, The Drunkard’s Walk.

First, the doctor failed to consider Mlodinow’s risk profile. Mlodinow notes that he was a “heterosexual non-IV-drug-abusing white male” in a time when HIV was concentrated in patients who were primarily gay, intravenous drug users, and/or non-white. For someone with Mlodinow’s profile, population numbers would have looked something like this:

Table 1 describes a population of 10,000 people with Mlodinow’s characteristics (heterosexual non-IV-drug-abusing white males in 1989), all of whom are tested for HIV. Of these 10,000, only 1 is actually infected with HIV. HIV testing yields virtually no false negatives; if you are sick, the test says you are sick. It does yield a false positive rate of 0.1%; 10 of the 9,999 non-infected people will test positive.

Mlodinow’s positive test put him in that second row of 11 men. Of those, 1 was infected and 10 were not. Hence, Mlodinow’s positive test only indicated a 9% chance of contracting AIDS—1-out-of-11—not 99.9%.

The doctor’s outrageously mistaken prognosis is an example of “the Prosecutor’s Fallacy,” so named because it resembles the manner in which prosecutors misuse statistics to exaggerate the likelihood of a defendant’s guilt. In the film, My Cousin Vinny, the prosecutor and an FBI expert witness note with great fanfare that the defendants’ tires were chemically identical to tread marks left behind by two escaped murderers—while ignoring the fact both were examples of the single most popular tire then sold in America.

(TANGENTIAL THOUGHTS: In offering his dire prognosis, the doctor also failed to mention that medical science might discover medications to control HIV/AIDS; two years later, Magic Johnson announced that he was infected and, thank goodness, he is still around. And, expertise and statistics aside, the doctor was callous enough to give a patient a horrifying death sentence in a brief, hurried telephone call.)   

---

Prosecutor’s Fallacy in Situ

Stephen, Christopher, and Sally Clark, 1998 (photo from forejustice.org)

In late 1996, Stephen and Sally Clark, both British solicitors (attorneys), welcomed their son Christopher into the world, only to find him dead in his crib a few weeks later—an apparent victim of sudden infant death syndrome (SIDS). In January 1998, their second newborn son, Harry, died similarly. Sally Clark was soon arrested for the murder of both children. In 1999, Professor Roy Meadows testified that two SIDS deaths in a single well-to-do family would be extremely unlikely. He said the likelihood of losing one child to SIDS would be 1-in-8,500. Therefore, he continued—wrongly—the likelihood of losing two children to SIDS would be the square of that fraction: (1/8,500)x(1/8,500) = approximately (1/73,000,000). There was no other evidence that Sally Clark was guilty, and those who knew her praised her temperament and child-rearing. That 1-in-73 million figure exploded across the British press, and Mrs. Clark became an object of frenzied national hatred. She was convicted of murder just as she gave birth to their third son. In prison she suffered grave abuse as a “baby-killer.” In October 2000, her conviction was upheld on appeal.

Over the next few years, however, the case began to fall to pieces. Professor Meadows’ formula—(1/8,500)x(1/8,500)—was shown to be nonsense. That formula would only be correct if SIDS deaths among siblings were what statisticians refer to as “independent events.” But SIDS among siblings is not independent. Due to genetics and other factors, one SIDS death in a family means that a second SIDS death is far likelier. (I’ve seen a figure of 1/60 for the second death.)

But even if the 1/73,000,000 figure were correct, it would be largely irrelevant because, as in the case of Leonard Mlodinow’s doctor or the fictional prosecutor in My Cousin Vinny, Professor Meadows committed the Prosecutor’s Fallacy. The relevant fact is not that two SIDS deaths among siblings is unlikely. Rather, the pivotal fact is that two siblings being murdered by their well-to-do parents is even less likely. (By one estimate, the probability of two murdered siblings is around 4% of the probability of two SIDS siblings.) Additionally, the investigation showed that a pathologist had withheld exculpatory evidence from the second child’s autopsy.

In January 2003, Sally Clark’s conviction was overturned, and she returned home to her husband and son. A nationwide review found that Meadows’ similarly incompetent testimony in other trials had sent other mothers to prison, and they, too, were released. Meadows and the pathologist were sanctioned, though Meadows soon regained some privileges. Sadly, there was no happy ending. Sally Clark left prison emotionally destroyed, and in March 2007, she died of alcohol poisoning.

---

A Bit o’ Data on the Batter Makes the Batter Better

Table 2: Justice versus Jeter batting averages, 1995-1997

Statistics prove that in 1995, 1996, and 1997, David Justice was a better batter than Derek Jeter, and that, by the same data, Jeter was a better batter than Justice over those same years. In 1995, Derek Jeter got 12 hits out of 48 at-bats, for a batting average of .250. The same year, David Justice had a higher batting average: .253. In 1996, Justice again had a higher batting average (.321 versus .314 for Jeter). In 1997, it was Justice at .328 over Jeter at .291. Obviously, these consistent disparities prove that Justice was the better batter.

But wait … there’s more! Add together the hits and at-bats for those three years, and it turns out that Jeter had a higher batting average than Justice (.300 versus .298) over the three-year period. So, obviously this disparity proves that Jeter was the better batter. To statisticians, this oddity is known as a Simpson’s Paradox—a situation where a trend in disparate groups disappears or reverses when the data on those groups are combined. (This particular example comes from mathematician Ken Ross.)

---

Avatar of Iowahawk

Car Guy Beats Nobel Laureate

In a caustic 2011 column, “Leaving Children Behind,” Nobel Prize-winning economist Paul Krugman offered a back-of-the-envelope statistical analysis to extoll Wisconsin’s unionized teachers and excoriate nonunionized teachers in states like Texas. Shortly after, Twitter car-identification guy and acerbic commentator David Burge (a.k.a. @Iowahawkblog) took Krugman’s analysis down to an automobile graveyard and ran it through an Al-Jon Car Crusher (made in Iowa). His takedown was a blogpost titled “Longhorns 17, Badgers 1.”

Krugman noted that in the five states that then barred collective bargaining by teachers, students’ ACT/SAT scores ranked 44th, 47th, 48th, 49th, and 50th among the 50 states. Wisconsin, where unionized teachers were then in a pitched battle with Governor Scott Walker, ranked second in ACT/SAT averages. Krugman’s takeaway was that unionized teachers seem to be great for students’ scores. Disparities are proof positive, after all.

Iowahawk is a bit of a mystery man. He has a history in antique car restoration and spends the weekends on Twitter identifying autos in photos that others submit. I’ll note in particular that he successfully identified a hard-to-recognize vehicle in which my grandfather was photographed around 1910. Beyond that, I’ve never figured out what he does for a living—but he has a serious mastery of statistical concepts. Of Krugman, he wrote:

every so often a member of the chattering class issues a nugget of stupidity so egregious that no amount of mockery will suffice. Particularly when the issuer of said stupidity holds a Nobel Prize.

Summing up Krugman’s message, Iowahawk continued:

The point being, I suppose, is that unionized teachers stand as a thin chalk-stained line keeping Wisconsin from descending into the dystopian non-union educational hellscape of Texas. Interesting, if it wasn't complete bullshit.

Iowahawk recognized that Krugman had swum into the gaping maw of a Simpson’s Paradox like the Justice-Jeter example. Iowahawk broke the data down into 18 categories: six subject areas and three ethnic classifications (white, black, and Hispanic). In 17 of 18 categories, Texas students performed better than Wisconsin students, meaning that:

white students in Texas perform better than white students in Wisconsin, black students in Texas perform better than black students in Wisconsin, Hispanic students in Texas perform better than Hispanic students in Wisconsin. … the only one where Wisconsin students performed better than their peers in Texas was 4th grade science for Hispanic students (statistically insignificant), and this was reversed by 8th grade.

Further, Iowahawk noted the disparities between white and nonwhite students were smaller in Texas than Wisconsin. Krugman thought he was saying, “Unions and collective bargaining are good for students.” Iowahawk made a compelling case that Krugman was unwittingly arguing that, “For excellence in education, your state needs more white students and fewer black and brown students.” For clarity, Iowahawk added:

Please note: this has nothing to do with innate ability or aptitude. Quite to the contrary, I believe the test gap between minority students and white students can be attributed to differences in socioeconomic status. And poverty. And yes, racism. And yes, family structure.

---

Disparities? Discrimination?

A well-known study of admissions to the University of California-Berkeley explored what some believed to be evidence of discrimination against women. In 1973, 44% of men who applied to Berkeley were admitted, while only 35% of female applicants were admitted. Some said, in effect, that this disparity proved systemic sexism.

But once again, the devil was in the details, and up popped the Simpson’s Paradox. When the researchers broke down the admissions by individual programs, the story changed radically. Out of 85 programs, 4 showed women to be at a disadvantage in terms of admission, while men seemed to be at a disadvantage in 6 programs. In the remaining 75 programs, men and women seemed to have a fairly similar rate of admission. With data broken down by program, the researchers concluded that overall:

there is a small but statistically significant bias in favor of women.

So why the 44%-35% apparent disparity between men and women? Because women tended to apply to programs with high rejection rates, and men tended to apply to programs with low rejection rates. The researchers suggested that K-12 education might have steered women toward professions with lower college admission rates but, if so, the fault did not lie with UC-Berkeley.

---

Next time someone says, “The statistics speak for themselves,” remember Leonard Mlodinow and his doctor, Sally Clark and her bereaved family, David Justice & Derek Jeter, Paul Krugman & Dave Burge, and the apparently fair folk in admissions at Berkeley in the 1970s. In a chestnut attributed by Mark Twain to Benjamin Disraeli, there are “lies, damned lies, and statistics."

---

ADDENDUM (12/17/22): An alert reader notes that the metaphor “chimp with a machine gun” long predates Better Call Saul. I haven’t located the original source, but Google searches show a long history, along with “monkey with a machine gun.”

Robert F. Graboyes is president of RFG Counterpoint, LLC in Alexandria, Virginia. An economist, journalist, and musician, he holds five degrees, including a PhD in economics from Columbia University. An award-winning professor, in 2014, he received the Reason Foundation’s Bastiat Prize for Journalism. His music compositions are at YouTube.com/@RFGraboyes/videos.