We write to communicate our concerns about the data shared by the Department of Elementary and Secondary Education from the Spring 2021 MCAS testing.
Teachers, School Committees and school superintendents told state leaders last year that spring 2021 MCAS tests would be a waste of time because nothing could be learned from the scores that wasn’t already obvious.
However, state officials insisted that it was critical to measure exactly how much learning loss COVID has inflicted on our children, and how big the famous “gaps” among groups of students have grown.Now that the scores are out, we can see that the tests failed to reach even those limited goals. Several thousand 10th grade students didn’t take the tests, along with a few thousand middle school students at each grade level. The groups most likely to skip the tests were those that scored low in the past. Those were precisely the groups that state officials said they were eager to track.
The score report contained two surprises. One was that English language arts (ELA) scores on the 10th grade MCAS went up a little, despite the pandemic. At the September 21st meeting of the state Board of Elementary and Secondary Education, the student member of the Board, Eleni Carris Livingston, asked whether lower participation rates among historically low-scoring students might account for the elevated 10th grade ELA score. Associate Commissioner Robert Curtin responded, “Generally speaking, the answer to that would be no. It was in various pockets of the state and not necessarily defined by levels of prior performance.” However, that response is not true, as the chart below using data from the state’s MCAS report makes clear.
More students skipped the 10th grade MCAS last spring than in 2019 before the pandemic. This chart shows the percentage decline in participation in the 10th grade English language arts test for all students and for subgroups.
Subgroups that have historically scored lower on the MCAS had larger declines in participation on the 10th grade MCAS than the average decline for all students. For example, from 2019 to 2021, 10th grade ELA participation for economically disadvantaged students dropped at three times the rate as for non-economically advantaged students (15% vs. 5%). This difference in participation rates could have had a significant impact on 2021 scores because economically disadvantaged students are a large subgroup (26,000+ students) and tend to score much lower on MCAS than their non-economically disadvantaged peers.
English learners perennially have the lowest scores on MCAS. This subgroup had the most precipitous drop in participation (20%) from 2019 to 2021 on the 10th grade ELA test. One quarter of English learners did not take the 10th grade MCAS in 2021.
If we compare the 10th grade scores for 2021 with past 10th grade scores, what do we learn? Nothing, because the students taking those tests were not comparable. If low-scoring student groups skip a test, that makes the average score higher.
In addition, given that last year’s 10th grade students must pass the high school MCAS tests to get a diploma, it seems important that DESE release the data on the extent to which subgroups attained competency determination on last spring’s tests. To date, we have not seen these data.
The second surprise was that, while most scores went down, the gaps among the subgroups did not grow, contrary to predictions. Black and brown students and those from low-income families suffered far more than their share of devastation from the pandemic, so it was reasonable to expect their MCAS scores to suffer more than average. But they didn’t.
Except there’s no way to tell how much the scores of these subgroups actually went down, because they were more likely than other students to skip the test. If those who skipped were mostly students who would have scored low, the reported scores were inflated, and we’re back to not knowing how much damage was done.
There are other reasonable explanations for the decline in scores. Less test prep probably occurred during the pandemic, so maybe what we’re seeing is test-prep loss, not learning loss.
Excessive stress also could have lowered MCAS scores last spring. A nation-wide survey (1) found that a major source of high school students’ elevated stress during the pandemic was school, including academic tests. Furthermore, research (2) has shown that a high level of stress is related to poor performance on high stakes tests.
Remote administration of MCAS could have resulted in biased results. Fifteen percent of students in grades 3 through 8 took the test remotely in the spring, and those students scored lower on both Math and ELA tests across all grade levels and on the 5th grade science exam than those who took it in person. (The probability of that happening by chance is the same as flipping a coin and getting heads 13 consecutive times.) Given the concerns raised about remote administration prior to the spring tests, the state Department of Education should at the very least acknowledge the fact that this pattern occurred.
What a waste of time to produce results with dubious validity. Now, Department of Education and Board of Education members are spending even more time reading these testing tea leaves, and for what purpose exactly? What will educators do differently than they would have done without the scores?
Test scores may seem precise, scientific and objective, but that doesn’t make them valid, useful or free from error
We would do better to ask educators, parents and students themselves what resources they need to move forward after a traumatic school year.
Thank you for considering our thoughts on the spring MCAS testing.
Louis Kruger and Alain Jehlen
2. https://justicetechlab.github.io/jdoleac-website/research/HADFM_TestingStress.pdf
Louis Kruger is professor emeritus at Northeastern University. He is a former school psychologist and on the board of directors of Citizens for Public Schools.
Alain Jehlen is on the board of directors of Citizens for Public Schools and an editor of Boston Parents Schoolyard News.
