State ed officials explore periodic exams as alternative to year-end standardized tests

The release of standardized test scores in Montana has become something of a fall tradition over the past few decades. For one headline-grabbing moment, parents and the public catch a fleeting glimpse of how students statewide performed in key areas such as math, reading and science — a snapshot gleaned from tests administered the previous spring.

Montana’s latest round of student performance data, released last month, seemed to confirm what many school officials and teachers had suspected since COVID-19 first shut down in-person instruction in spring 2020. Student proficiency in math and English slipped from pre-pandemic levels, with some grade levels showing a nearly 10% dip in math scores in particular. The picture prompted Superintendent of Public Instruction Elsie Arntzen to announce an enhanced focus on improving math skills and brought renewed public attention to a suite of pandemic impacts that public schools are still trying to fully understand.

In many Montana classrooms, the pandemic served to underscore pre-existing challenges in public education, chief among them the ongoing need to identify each student’s strengths and weaknesses to better deliver on the Montana Constitution’s promise of a free quality education for all. To that end, many educators agree that year-end test scores — released in the fall after students have moved on to new grades, new classrooms, even new buildings — are of little use in informing day-to-day changes in individual classrooms. That has state education officials Montana looking for alternatives.

Dovetailing with her agency’s release of standardized test scores in September, Arntzen trumpeted the Office of Public Instruction’s rollout of a pilot program designed to explore a new model for gauging student performance. With the backing of $3 million in federal grant funding, OPI’s new Montana Alternative Student Testing (MAST) Pilot Program is looking into replacing Montana’s year-end exams with a series of tests delivered throughout the school year. 

“If you want to get different results, you have to do something different,” Arntzen told Montana Free Press in late September. “I believe we have a lot of stars aligned, that we are listening to our classroom teachers and we are reflecting on making sure that that teacher’s effort in that classroom is being honored. Having that success being reflected in a one-year, one-size-fits-all test does not work.”

Montana isn’t alone in experimenting with a new testing system. According to Meghan McCann, assistant director of policy at the nonprofit Education Commission of the States, Nebraska has been exploring such an approach for several years. Florida passed legislation in March implementing a new assessment model that will test students in grades 3-10 in math and English at three different points in the school year. Lawmakers in Missouri and Kentucky considered similar proposals this year, but ultimately rejected them.

McCann also notes that the U.S. Department of Education this year awarded 11 grants to 10 state education agencies to enhance how they measure student achievement, indicating interest in new approaches to assessment at the federal level.

“One of the allowances and one of the stated purposes for those grants is to allow statewide summative assessments to incorporate multiple measures of student learning,” McCann said. “So I think even the Department [of Education] is sort of putting out there that that’s an option.”

The $3 million in federal funding underpinning Montana’s MAST pilot comes from one of those grants. Twenty districts across the state are participating this fall, administering three tests at three separate points in the school year in grades 5 and 7. OPI declined to identify those districts, noting that the memorandums finalizing their participation would not be completed until October.

ASSESSMENT EXPLAINED

Student assessment is a complicated subject, and understanding where Montana’s new pilot program fits in requires some context. The example of assessment that likely rings most familiar to the broad public is the ACT, a standardized test that many educators have used to gauge college readiness among high school juniors and seniors since the 1960s, or its older cousin the SAT. College admission offices’ reliance on the scores made such tests a high-stakes, high-profile fixture for millions of students, birthing an industry of prep materials and leaving generations with the shared memory of spring days spent inking in the bubbles on an answer sheet.

That widespread relatability makes the ACT a prime illustration of what’s known as a “summative assessment.” Such tests are engineered to determine whether students have gained a certain level of subject-area proficiency by the end of a given year, and their use extends well beyond college readiness. 

In the wake of the 2001 passage of the No Child Left Behind Act, summative assessments in grades 3-8 have become a mandatory accountability metric by which federal and state agencies measure a school’s performance and progress. Data gleaned from those tests has highlighted learning gaps between particular groups of students, fueling discussions about historically underserved populations in public education. Heather Peske, president of the National Council on Teacher Quality, said identifying such gaps is one of many reasons summative assessments “serve a very clear purpose.”

“For a number of years, we didn’t know how we were doing in educating students of color, particularly African-American Black students and Latinx or Hispanic students. We just simply didn’t know,” Peske told MTFP in an interview. “We cannot responsibly educate kids without knowing how they’re doing on a standardized assessment that compares students to a set of standards that we know they need to learn by the end of each grade.”

In addition to the ACT, administered in high school, Montana requires an end-of-year math and English language skills test known as the SBAC to all students in grades 3-8. Students with significant cognitive disabilities get a test called the Multi-State Alternate Assessment. Students in grades 5, 8 and 11 are also tested on science proficiency.

School Administrators of Montana Executive Director Rob Watson has often heard those summative assessments described as “an exercise in perseverance.” Students take them over the course of several days, he said, and since the computerized tests are adaptive, one missed question will elicit more questions related to the associated skill until the system “gets an idea of where your abilities are.”

“I heard that just this last week from a group of educators that happen to work with some of our more struggling students in the state, perhaps students with learning disabilities or students that are missing some skills,” said Watson, who formerly served as superintendent in both the Missoula and Bozeman school districts. “Those were their words, just in terms of you really have to have a kid that’s willing to persevere to get through all of it. And a lot of kids just give up on it.”

But again, that sort of end-of-year testing isn’t much help to teachers when it comes to tailoring classroom instruction to meet the needs of individual students. In the words of Billings Schools Superintendent Greg Upham, you wouldn’t teach guitar by telling a student to practice for nine months and then testing them. You’d assess the student’s progress along the way to figure out if they need extra time or help to, say, perfect a C chord.

That’s where “formative assessment” comes into play. By implementing routine tests at different points in the school year, educators can get a more immediate picture of how their students are doing in real time, and adjust their instruction to address any areas where performance is lagging. Unlike summative assessments, Montana doesn’t have a uniform statewide formative assessment, leaving it up to local districts to find one that fits with state education standards and local curricula.

According to Anne Keith, a retired Montana schoolteacher who currently sits on the state’s Board of Public Education, formative testing is a tool that educators have long put to use at the classroom level. Keith herself relied on such assessments from “day one” in her 30-year teaching career. As a teacher, she said, a test that clearly marks an individual student’s progress during the school year is far more valuable than a standardized test that compares them to other students in their grade nationally.

“What my job as a teacher is is to take every one of those kids and get them to grow as much as possible within the time that I have them,” Keith said. “So if I have a test that could show you, ‘Hey, your kids started at second-grade level and now they’re up to fifth-grade level, they made three grade-level gains in the time we’ve had them,’ that’s phenomenal.”

HOW WE GOT HERE

Debate about assessment is hardly new. Peske, Upham and others interviewed for this story noted that student assessment has long played a crucial role in measuring academic growth and informing classroom instruction. But as standardized testing became more entwined with federal regulation and the development of educational standards, the notion of a single test as an accurate barometer for student achievement received pushback. Grounds for criticism have run the gamut, from teachers and parents maligning the number of tests students have to take each year to a 2012 Brookings Institute report determining that mandated summative assessments collectively cost schools across 45 states $669 million annually.

The weight given to such assessments also fueled ongoing concerns about the pressure they apply to students. Keith likened high-stakes end-of-year tests to a “big state championship game,” with classroom instruction grinding to a stop for a week.

“We make a huge deal about it,” Keith said. “That’s good for some kids, they rise to that. And there’s some kids that freak out and don’t perform well with all this pressure on them.”

The tests at the center of that debate took a hiatus in spring 2020. As the COVID-19 pandemic prompted nationwide school closures, the federal government waived mandatory assessments for the year. Testing resumed in spring 2021 despite opposition from Arntzen and education officials in half a dozen states. Arntzen’s second plea to the Department of Education for cancellation of mandatory assessments drew the support of Gov. Greg Gianforte, the Montana Board of Public Education, and the Montana Federation of Public Employees, whose president, Amanda Curtis, wrote that testing was the last thing students and teachers needed after a period of “unprecedented upheaval.”

According to McCann, the pandemic has brought new focus to the broader assessment debate. As a result of the temporary pause in the nation’s accountability system for public education, many educators have now begun to ask a more critical and fundamental question.

“We’ve been doing this for a long time, we’ve been giving these tests for a long time,” McCann said. “But why, and what are they used for?”

Prior to this fall’s rollout of the MAST pilot, OPI convened a task force of teachers, principals, superintendents and curriculum directors to sort through such questions and put a finer focus on what, exactly, Montana needed from a new assessment system. The group met throughout June and July to discuss the state’s challenges and goals, and to hammer out details of how a through-year assessment would be administered and how resulting scores would be reported to educators, students and parents. In its final report, the task force also reached a conclusion regarding what the pandemic revealed about the failure of Montana’s existing summative testing model.

“Stakeholders had placed so much emphasis on end-of-year test results that, when the test disappeared, they felt they were left with no data,” the task force wrote. “Having a set of assessments aligned with state content standards and administered throughout the year would have avoided this predicament.”

The task force further recommended that it, or a similar group, continue to help oversee the pilot program during its implementation, and that students, test coordinators and school counselors be added as “formal members” of the oversight entity to provide routine feedback to OPI.

While America’s public education system has spawned a wide array of assessment types to serve various purposes, the formative approach is where the task force and OPI opted to focus the state’s attention. Arntzen’s path forward would, in theory, replace the summative assessments that Montana schools have administered over the past two decades with a model more in line with what districts have to date done on their own. Such a replacement would require the federal government to grant Montana certain flexibility in reporting test scores, which Arntzen acknowledges isn’t a guarantee. But she said the MAST pilot, if successful, promises to cut down on duplicative assessment, soften the high-stakes feel of statewide testing, and provide teachers with data that more accurately reflects their students’ current skill levels.

Arntzen added that the timing of the pilot program is particularly opportune given her agency’s ongoing work to revise statewide standards for math, creating an opportunity to better align through-year testing with the skills Montana expects from public school students. A similar review of the state’s reading standards is scheduled to start by the end of next year. Arntzen also indicated that the coming legislative session will likely involve discussions about the state’s definition of proficiency. 

At the same time, higher education officials in the state have followed a national trend toward deemphasizing the role of standardized testing in college admissions. The Montana Board of Regents temporarily waived test score requirements for incoming students during the pandemic, and voted in May 2021 to make the move permanent. While still a required part of certain scholarship packages, ACT and SAT scores are no longer a mandatory component for admission on Montana campuses. According to McCann, the same is now true in 27 other states.

Referencing last month’s latest round of assessment data, Arntzen said she’s hopeful that this whirlwind of activity, including the pilot program, can start to shed light on questions that Montana has long had difficulty answering.

“In the data that we have on the summative test right now, it is so challenging to see what happens in eighth grade,” Arntzen said. “Eighth grade math is 42% novice, meaning they don’t — the light’s not on yet. And then in third grade, when we first test them, it’s 27% [novice]. So in other words, there’s something going on between third grade and eighth grade where students aren’t recognizing their own learning and they’re not growing.”

INTEREST AND HESITATION

While the pandemic created breathing room for states to begin reexamining their approach to student assessment, it’s also the reason that Russ Lodge, interim superintendent at Missoula County Public Schools, passed on participating in the MAST pilot this fall. He’s intrigued by the idea and eager to see what the program yields, but said his district is maintaining a focus this fall on filling staff shortages and understanding and addressing the social-emotional impacts COVID-19 had on its students. 

“We’re not chasing test scores,” Lodge said. “I want everybody to just relax. You know, if your class comes in and scores aren’t what you’d like them to be or what you were used to seeing 10 years ago, that’s fine. Build a relationship, work in your [personal learning communities] and we’ll move on. We’ll be fine.”

In Billings, Upham expressed a similar reluctance to sign his district on to the effort. With recent updates to local curricula and ongoing efforts to address deficiencies particularly at the elementary school level, he said, his administration “didn’t want to put one more thing on teachers’ plates” this year.

“It’s the right thing, wrong time for us,” he said. “I’m in support of it. I think it’s the right thing to do. I just don’t want to put one more thing on our teachers.”

For participating districts, the MAST pilot won’t replace summative assessments, which are still required next spring, nor will it replace their internal formative testing systems. Rather, those districts are volunteering to take on an extra layer of testing in the hopes that the results provide compelling and actionable information for the state.

Montana Federation of Public Employees President Amanda Curtis, a former schoolteacher whose organization represents public school employees across the state, expressed a similarly mixed sentiment about the effort this year. A test that clearly marks how students are progressing during the school year is certainly more useful for teachers than a test showing how last year’s students performed, Curtis said. But Curtis said the best administrators she has worked for have advised teachers to prioritize developing their students into confident, critical thinkers and to not focus too heavily on standardized testing. At the end of the day, she said, a pilot like this really isn’t anything new for a seasoned educator.

“Teachers are used to being told they need to try new tests,” Curtis said. “Well, they’ll do these and see if Elsie thinks that they work.”

Kalispell Schools Superintendent Micah Hill said his district will be helping the endeavor in a limited capacity this school year. Aware of the ongoing challenges raised by fluctuating enrollment during the pandemic, Hill kept Kalispell’s participation in the MAST pilot confined to fifth-graders at one elementary school and a seventh-grade cohort of roughly 125 students. He said the district has yet to see what the MAST assessments, produced by a national nonprofit called New Meridian, look like. But he views the program as a chance to determine whether a series of 15- to 20-minute tests throughout the year can satisfy the needs of the public education system.

“This is something that we’ve been asking for, I think, as an education community for 10 years,” Hill said. “We recognize that these final summative assessments just are not what our students need, and they’re not what our teachers need to be able to gauge their effectiveness to whether or not students are learning.”

Under OPI’s current schedule, the MAST pilot program will expand during the 2023-24 school year to include students in grades 4 and 6, and expand again the following year to collect data in grades 3 and 8 before the state formally submits a plan to the federal government revising its accountability system. By then, OPI will be under the leadership of a new state superintendent, as Arntzen will close out her final term at the end of 2024.

Whether the pilot ultimately leads to a significant change in Montana’s student assessment system, its development and rollout have nudged the conversation around standardized testing in a particular direction. Several years ago, Watson said, the likelihood of getting the U.S. Department of Education’s blessing at the end of such an experiment would have been slim. Now, the agency’s willingness to invest $3 million in the MAST pilot suggests at least a passing federal interest in exploring new approaches to collecting educational data. Even so, Watson cautions against placing too much emphasis on a single assessment, especially when it comes to defining a school’s performance.

“It’s hard to say if a school’s a passing school or a failing school because there’s so much more to a school than just that one test,” Watson said. “But in a lot of other states, that’s what it’s kind of boiled down to. However kids do on that assessment determines if it’s a failing school or not. I would hope that most Montanans know a lot more about their school and would have other ways to assess whether it’s a good school or a school that needs help.”