Summary
Proponents of the MCAS graduation requirement have shared high school dropout data intended to show that the requirement is not a major obstacle to obtaining a diploma. However, they both omitted important data and erroneously interpreted other data. A clear-eyed review of a wider swath of evidence revealed a very different picture. This evidence indicated that (1) the graduation requirement has caused some historically underserved students to drop out; (2) students who failed the 10th grade MCAS were 17 times more likely to drop out than their peers who passed the exams; and (3) the dropout rate for students failing the MCAS has increased from 15 percent in 2015 to 18 percent in 2019.
Massachusetts goes to considerable expense and effort to collect data on MCAS results and dropout rates. Data can be used selectively to bolster or undermine a policy. So, what do the data actually show about the connection between the high-stakes MCAS and dropout rates? This question takes on special importance now because of efforts to eliminate the MCAS graduation requirement through either proposed state legislation, The Thrive Act, or a statewide ballot question for November 2024.
At the September 13, 2023 meeting of the Board of Elementary and Secondary Education, Vice-Chair Matt Hills presented data in support of the MCAS graduation requirement. Included in the presentation was a table that showed that between “two-thirds and three-quarters” of 12th grade students who dropped out from 2015 to 2019 had already passed the MCAS. Given that most students who drop out have passed the MCAS, he reasoned that MCAS cannot be a major factor in students dropping out. On September 24, 2023, the Editorial Board of the Boston Globe made the same argument (“No, the MCAS exam is not holding kids back”) with the same 2015-2019 dropout data.
Despite the seeming logic, the above argument is based upon incomplete data and a faulty comparison. First, it is puzzling why the Globe and Vice-Chair Hills omitted 11th-grade dropout data. After all, students take the high stakes exams in 10th grade, and failing the exams in 10th grade could impact the decision to drop out in the 11th grade. Furthermore, most students who fail the MCAS and then drop out leave school in 11th grade, not 12th. If the defenders of the MCAS graduation requirement had used the 11th grade data in addition to the 12th grade data, their numbers would have been less convincing. During four of the five years in question, students who failed the MCAS comprised a majority of the dropouts (between 52% to 58% in those four years). The 11th grade data clearly do not fit the narrative being told by Vice-Chair Hills and the Globe.
A second and more fundamental problem with the above dropout numbers is that they are based on a faulty comparison. To conduct any meaningful comparison between two groups, it is necessary to use a metric that controls for the difference in sizes of the groups. As far as I can tell, neither Vice-Chair Hills nor the Globe controlled for this difference. They compared the number of dropouts in a very large group, 12th grade students from 2015-2019 who have passed all exams (average annual enrollment of 67,089), to the dropouts in a much smaller group, 12th grade students who have not passed the exams (average annual enrollment of 2,292).
Using separate dropout rates for the two groups is one simple method of controlling for a difference in group sizes. Data from the MA Department of Education’s website indicate that the dropout rate among the 12th grade students from 2015-2019 who passed the exams was only 1.5 percent, in sharp contrast to the high dropout rate of 19.2 percent among those who failed the exam. Another method of controlling for differences in group sizes is to calculate an odds ratio. An odds ratio is a measure of how strongly an event (e.g., failing vs passing the high school MCAS) is associated with a particular outcome (e.g., dropping out vs staying in school). When both 11th grade and 12th students from 2015 to 2019 are considered together, the odds of dropping out were 17 times greater in the group of the students that had failed the high school MCAS compared to the group who had passed the exams. Simply put, a student who fails the MCAS is at much greater risk of dropping out than a student who passes the exams. That important point was never mentioned in Vice-Chair Hills’ presentation or the Globe’s editorial.
Students drop out for multiple reasons. Therefore, it is unreasonable to expect any one factor, such as the MCAS, to account for all or even most dropouts. Furthermore, the dropout data I have shared do notprovide conclusive evidencethat failing the MCAS causes some students to drop out. A rigorous research design is required for that issue, and Brown University Professor John Papay and his colleagues conducted such a study. They found that for low-income urban students, barely failing the 10th grade MCAS math test increased their dropout rate by four percentage points in the following year and decreased their on-time graduation rate by eight percentage points. Their results suggest that many urban students’ lives have been harmed by the graduation requirement.
Overall dropout rates can mask the dropout rates for historically underserved students, who are most likely to fail the MCAS. For example, Latinx students, who have the highest annual dropout rate (4.6% in 2019) among all MA ethnic/racial groups, also have the lowest pass rate on the MCAS among all MA ethnic/racial groups. In 2019, 27 percent of 10th grade Latinx students failed at least one of the required MCAS exams.
Finally, the dropout rate for students failing the MCAS seems to be heading in the wrong direction. From 2015 to 2019, the combined dropout rate for 11th and 12th grade students failing MCAS increased from 15.1 percent in 2015 to 18.3 percent in 2019. Ignoring these and other relevant data will only serve to further jeopardize the future of our most vulnerable students.
Louis J. Kruger, Psy.D.
Professor Emeritus, Northeastern University
Board of Directors, Citizens for Public Schools
References
Altman DG (1991) Practical statistics for medical research. London: Chapman and Hall.
Doll, J. J., Eslami, Z., & Walters, L. (2013). Understanding Why Students Drop Out of High School, According to Their Own Reports.
https://journals.sagepub.com/doi/10.1177/2158244013503834
Massachusetts Department of Elementary and Secondary Education (2019). Spring 2019 MCAS Tests: Summary of State Results. https://archives.lib.state.ma.us/bitstream/handle/2452/808673/ocn753986717-2019.pdf?sequence=1&isAllowed=y
Massachusetts Department of Elementary and Secondary Education. High School Dropout Summaries.
https://www.doe.mass.edu/infoservices/reports/dropout/2018-19/
https://www.doe.mass.edu/infoservices/reports/dropout/2017-2018/
https://www.doe.mass.edu/infoservices/reports/dropout/2016-2017/
https://www.doe.mass.edu/infoservices/reports/dropout/2015-2016/
https://www.doe.mass.edu/infoservices/reports/dropout/2014-2015/
Papay, J. P., Murnane, R. J., & Willett, J. B. (2010). The consequences of high school exit examinations for low-performing urban students: Evidence from Massachusetts. Educational Evaluation and Policy Analysis, 32(1), 5–23.
