Case Selection and Bias in Clinton Foundation Coverage

Yesterday I tried to put the AP’s claims about the Clinton Foundation and Hillary Clinton’s meetings as Secretary of State in context by loosely estimating the success that foundation donors had at landing appointments.

I should have pointed out the case selection problem at the heart of the AP analysis. If we really wanted to look at whether or not a gift to the Clinton Foundation increased the probability that a meeting2 with Hillary Clinton took place than we would need not just the cases of folks who had meetings (selecting on the dependent variable), but also the population of people who attempted to secure a meeting and were unable to do so. Only then would we begin to discern if giving money to the Clinton Foundation had a beneficiary effect on those seeking access to the State Department compared to those who made no contribution. Looking at the fraction of foundation donors who snagged some face time with HRC is basically beside the point.1 For instance, we don’t even know that those other folks who gave money even wanted or tried to get meetings.

But over on Vox, Matthew Yglesias explains why making sense of the AP’s report is a fool’s errand:

Serving as secretary of state while your husband raises millions of dollars for a charitable foundation that is also a vehicle for your family’s political ambitions really does create a lot of space for potential conflicts of interest. Journalists have, rightly, scrutinized the situation closely. And however many times they take a run at it, they don’t come up with anything more scandalous than the revelation that maybe billionaire philanthropists have an easier time getting the State Department to look into their visa problems than an ordinary person would.

In his article, Yglesias draws a connection with publication bias in academic journals, “[T]he AP’s basic reporting project here seems like it was worth a shot and probably also fairly time-consuming. But it did not come up with anything.”

But instead of falling prey to the file drawer problem, the AP published their null findings using a reporting frame which implied corruption between the State Department and the Clinton Foundation. In discussion on Twitter, Yglesias explained that the framing device used in the AP’s story was “a different manifestation of the same bias against null findings” which strikes me as an exceptionally apt way to put it.

Onward to the next scandal. Pseudo or otherwise.


  1. My analysis didn’t even look at donors, only the share of money. The Clinton Foundation apparently has received contributions from 7000 different donors, at least according to a tweet cited in this Washington Monthly article which is probably over the course of the whole existence of the foundation, not just the four years I looked at yesterday. So even from that flawed perspective if 85 of these folks received meetings with Hillary Clinton that’s only 1.2% of the foundation’s donors. Ho-hum, right?  

  2. A meeting mind you, not even a favor and note it seems that we have plenty of evidence that Hillary Clinton’s staff at State was quite capable of squashing requests which came via the Clinton Foundation. 

View the most recent posts in blog, all of the posts in the archive, or browse the tags.

I put my stuff in asocialfolder and this website popped out.

Sign up today.