It bothers me that so much of modern life is opaque and assumes that no one has the time and energy to read the footnotes.

I went and looked at the first study on the School Improvement Grants,

    > In 2009, Congress spent $3 billion on “School Improvement Grants” for the worst-performing schools in the country. As far as we can tell, those grants did nothing.

Is this true? I looked at the linked report and it's strange, https://ies.ed.gov/sites/default/files/migrated/nces_pubs/nc...

At one end it says, "Although schools implementing SIG-funded models reported using more SIG promoted practices than other schools, we found no evidence that SIG caused those schools to implement more practices."

What are these practises? Well... it seems that as a pre-requisite to getting funding states were made to divide schools into four categories / intervention modes,

    > ED required that each SIG-awarded school under Tier I or Tier II implement one of four school intervention models. These models required specific practices:
    > 
    > • Transformation. This model required schools to replace the principal, adopt a teacher and principal evaluation system that accounted for student achievement growth as a significant factor, adopt a new governance structure, institute comprehensive instructional reforms, increase learning time, create community-oriented schools, and have operational flexibility.
    > • Turnaround. This model required schools to replace the principal, replace at least 50 percent of the school staff, institute comprehensive instructional reforms, increase learning time, create community-oriented schools, and have operational flexibility.
    > • Restart. This model required schools to convert to a charter school or close and reopen under the management of a charter management organization or education management organization.
    > • School closure. This model required districts to close schools and enroll their students in higher-achieving schools within the district.
    > 
    > These required practices can be grouped into the four main topic areas promoted by SIG. Table ES.1 lists these four broad areas and the objectives promoted by SIG within each. For example, replacing the principal falls under the “identifying and rewarding effective teachers and principals and removing ineffective ones” objective within the “developing and increasing teacher and principal effectiveness” topic area. 
    >
    > The objectives listed in the table cover all practices promoted by SIG, which different models designated as either required or permissible.

In plain english, it seems that they're saying at one end that schools were "transformed," turned around, restarted, or closed in order to get funding. They also promited performance review plans and quarterly / yearly reviews. Fairly standard administrative procedure.

But then they start the summary by saying that there's "no evidence that SIG caused those schools to implement more practices" – the ones which were heavily promoted by the program and were a part of it?

I feel like I'm misunderstanding something as these seem to be two mutually and logically exclusive things. You are requiring something for funding, but then you're saying afterwards that you can't be sure that they did related things just because you required it?

What am I missing here?

Also some of these graphs look wrong. The error bars on Figure ES.3. go from ~ -0.2 to 0.25... With error bars that large isn't the graph meaningless?

We need more goat sacrifices