First, a little history. During the 2002-08 time period the study looked at 29 closed high schools and the results for future students who instead went to the Bloomberg small schools which showed a 15% increase in the graduation rate. However, the study admits that students who fled these closing schools did worse then the ones who stayed and graduated, Moreover, these students who fled were academically behind their peers in the new schools they transferred to.
During the time period the study covered there was an explosion of credit recovery courses that artificially increased a struggling student's credit accumulation. Even the State finally saw the abuse of the credit recovery system and required the City to put an end to it in 2012. The primary beneficiaries were the Bloomberg small schools who needed to have a high graduation rate to show how successful they were. Therefore, they gave struggling students loads of "credit recovery" courses, many of them were bogus. You can read some of the stories, Here, Here, and Here.
The Bloomberg small schools were notorious in hiring "newbie teachers" and since these teachers lacked tenure, they were pressured to pass along the school's struggling students. Teachers who failed too many of their students were discontinued due to scholarship issues and were banned from working at the DOE. This message resonated loud and clear with the inexperienced staff and resulted in high graduation rates despite these students academically unable to succeed in college or careers.
To ensure the Bloomberg small schools would succeed they opened with some very significant advantages like excluding many ELL;s and Special Education students the first few years of their operation. Further, they limited "high needs" students (namely level 1's) and were given 110% of the resources allocated to the school of similar size and student demographics. These significant advantages were, in contrast to the higher levels of "high needs" , ELL's, and Special Education students that were in the soon to be closed large comprehensive large schools. Moreover, these soon to be closed schools were starved for resources and averaged 80% of what similar large comprehensive schools received. The Bloomberg small school I am in for this year is a prime example as they were once a high achieving school in the mid 2000's, when it first opened. and now they struggle to get even mediocre, students to go to the school as their funding advantage disappeared and they could no longer exclude students. The resulting influx of "high needs" students has destabilized the school environment.
When you put the three aspects together, the bogus "credit recovery" programs, Principal pressure to graduate academically unprepared students, and the deliberate advantages that the Bloomberg administration bestowed on his newly created schools, you basically are getting a higher graduation rate at these small schools, when compared to the closed large comprehensive schools. However, what's more telling is the low college and career readiness scores that show that many of the Bloomberg small school are graduating students who are unprepared for college or careers in the adult world. That's why the NYU report is fatally flawed.