This paper reexamines data from the New York City school choice program, the largest
and best implemented private school scholarship experiment yet conducted. In the
experiment, low-income public school students in grades K-4 were eligible to participate
in a series of lotteries for a private school scholarship in May 1997. Data were collected
from students and their parents at baseline, and in the Spring of each of the next three
years. Students with missing baseline test scores, which encompasses all those who were
initially in Kindergarten and 11 percent of those initially in grades 1-4, were excluded
from previous analyses of achievement, even though these students were tested in the
follow-up years. In principle, random assignment would be expected to lead treatment
status to be uncorrelated with all baseline characteristics. Including students with
missing baseline test scores increases the sample size by 44 percent. For African
American students, the only group to show a significant, positive effect of vouchers on
achievement in past studies, the difference in average follow-up test scores between the
treatment group (those offered a voucher) and control group (those not offered a voucher)
becomes statistically insignificant at the .05 level and much smaller if the full sample is
used. In addition, the effect of vouchers is found to be sensitive to the particular way
race/ethnicity was defined. Previously, race was assigned according to the racial/ethnic
category of the child's mother, and parents who marked “other” and wrote in
Black/Hispanic were typically coded as non-Black and non-Hispanic. If children with a
Black father are added to the sample of children with a Black mother, the effect of
vouchers is small and statistically insignificant at conventional levels.