Ready for the test
Ohio high school teacher Mona Kotran teams up with a countywide educational partnership to help students boost their ACT scores and college prospects.
Eman Hassan said she starts to shake all over and has trouble holding onto a pencil. Brynne Burgy gets “a general feeling of hotness” and can’t feel her fingers.
The two high school seniors in Akron, Ohio, are describing how they generally respond to taking a test.
But when Burgy and Hassan sat for the high-stakes ACT Math test in April 2016, the normal signs of test anxiety weren’t there. Even more importantly, they both received excellent scores. Ask them to explain their success on the test, and Burgy and Hassan give a lot of credit to the ACT Math prep course they took with their high school’s instructional specialist, Mona Kotran.
“No question, that course was a major help,” Burgy said.
Kotran taught the course in cooperation with Summit Education Initiative (SEI), a nonprofit organization working with schools and other partners in Akron and surrounding Summit County to increase educational attainment for students at all levels. A former math teacher, Kotran said she and her colleagues at Akron Early College High School had been looking for ways to increase students’ scores on the ACT, the standardized test used by many Midwestern colleges and universities to make decisions about admissions and course placement. When SEI approached its high school partners in the county to see if anyone wanted to pilot the new course, Kotran was the first to volunteer.
“We had been doing ACT review in class, but we weren’t getting the results we wanted. I knew there had to be another piece of the puzzle, and this course looked like a good answer,” she said.
Data reveal a problem
Like other members of the StriveTogether Cradle to Career Network, SEI views postsecondary enrollment as a critical transition point in the cradle-to-career pipeline. ACT or SAT preparation is part of the work of ensuring that more students make this transition successfully. SEI’s ACT Math prep course was the brainchild of Matthew Deevers, a former high school English teacher and principal who now works as senior research associate with SEI.
While sifting through data from the Ohio Department of Education a few years ago, Deevers and his SEI colleagues noticed that a significant percentage of students from Summit County were taking remedial math classes in college. In fact, the “remediation rate” for college math among the county’s students was as much as 15 percent higher than it was for other subjects. Knowing that students at public colleges and universities in Ohio do not have to take remedial classes if they score 22 or higher on the ACT, Deevers started exploring how to increase student scores on the test.
“SEI’s goal is that all students graduate ‘prepared, passionate and persistent,’ and to me that means entering college remediation free,” Deevers said. The case for supporting the county’s students to boost their ACT scores was bolstered by additional SEI research. In a study of 1,500 students who graduated from Summit County public schools and went on to college, Deevers and his colleagues found that students who graduated with an ACT score higher than 22 (and therefore did not require remediation classes) were four times more likely to complete their college studies than those who didn’t.
The case for supporting the county’s students to boost their ACT scores was bolstered by additional SEI research. In a study of 1,500 students who graduated from Summit County public schools and went on to college, Deevers and his colleagues found that students who graduated with an ACT score higher than 22 (and therefore did not require remediation classes) were four times more likely to complete their college studies than those who didn’t.
“The bottom line is we had all the evidence we needed to say that helping kids graduate remediation free would make a big difference and that math was the place to start,” Deevers said.
Making the case
In 2014 and 2015, SEI started discussing its ACT research in regular meetings with members of the initiative’s Ready High School Network, a group of administrators, teachers and staff from most county high schools who work together on strategies for increasing college attainment. Deevers said the goal was to start building a case for collective action to improve students’ ACT scores and ensure that more students could enter college without having to take remedial classes.
In addition to countywide data, Deevers and his colleagues presented officials from each school with data on the remediation rates of their school’s graduates. “That’s what really surprised people and helped build support,” Deevers said. “Many of these are schools where going to college is part of the culture, but now they were seeing that a significant number of their students weren’t necessarily arriving at college prepared.”
After the high schools expressed strong support for an SEI-led effort focused on the ACT, Deevers and his colleagues began to explore solutions. At first, they researched commercial ACT preparation programs, but they couldn’t find anything that met their needs and requirements. “The less expensive programs focused almost exclusively on test-taking tips, and the programs that included the most math review were too expensive and not scalable,” Deevers said.
A home-grown solution
Around this time, SEI was accepted into the Lumina Foundation’s Community Partnership for Attainment initiative. Using the Lumina funding, SEI worked with local high school teachers on a prototype ACT Math prep course of its own. The course combines PowerPoint presentations on key math concepts with video of teachers explaining how to solve specific problems. Students then are asked to solve similar problems on their own in timed exercises.
Deevers called it a “catch-and-release” strategy. “We catch them and show them a solution and then release them to do it themselves,” he explained. Deevers said the course was designed so it could be offered in a variety of settings, with or without the presence of an in-person math instructor.
SEI conducted what Deevers calls a “beta test” of the course in spring 2015 with a small group of eight students in an after-school program. Then, in January 2016, Kotran launched the larger pilot course with the 86 members of the junior class at Akron Early College High School. Students met in one-hour sessions once every week until shortly before they sat for the ACT in April.
Kotran said she was impressed by the design of the course from the start. “The difficult thing in preparing students for the ACT is combining problem-solving strategies with concepts from algebra and geometry classes that most juniors took one or two or three years ago,” she said. “This course isn’t necessarily math instruction, but it puts those concepts in front of students again. And it gives the chance for repeated practice with models of high-quality work and opportunities for reflecting on progress.”
Crunching the numbers
Of course, student scores are the true test of an initiative like this, and the pilot run of the ACT Prep course showed promising results. The 86 juniors in Kotran’s spring 2016 course saw an average gain in their ACT scores of 3.4 points (the ACT scale goes from 1 to 36), compared to a pre-test they took before starting the course. Before the course, just 53 percent of the students were scoring at the 22-or-higher level that allows them to enter college remediation free in math. The comparable figure postcourse was a remarkable 83 percent.
“Getting over that college-ready benchmark is the most important thing in this work,” Deevers pointed out. Akron Early College High School’s 83-percent figure for college readiness in math was “significantly higher” than any other public high school in the city, he added.
To make sure the course wasn’t just helping high-performing students get even higher scores, Deevers crunched the numbers even further. He found that students in the bottom quartile on the pre-test were making gains of 5 to 7 points on the official ACT, while those in upper and middle quartiles were gaining 4 points on average. “That was a key piece of data for me, because it showed that across all levels of performance, these students were making gains,” Deevers said.
Based on the success of the pilot course, Kotran launched a year-long ACT prep class in fall 2016 using the same format and approach. Because of feedback from students in the pilot, she added more content on core math concepts. She also has added content to prepare students for the ACT Science and English tests.
Deevers said Kotran has been “the perfect partner” in launching the ACT work. “When you pilot something, you need to have someone who can stand the bumps in the road and give you honest feedback,” he said. “[Kotran] could tolerate the insecurities of being part of a pilot, and she was honest with us about what was working and what wasn’t.”
Going to scale
Meanwhile, Deevers spent the summer of 2016 working with SEI research associate Cristina González Alcalá to turn the course content for the pilot into a book. “We wanted to try and scale this work as quickly as possible so it can have impact in other schools and for other students,” he said.
The resulting book, “Level UP Your ACT Math Score,” includes quizzes, full-length practice tests and other key content from the course, while providing QR codes for YouTube access to the short instructional videos that are such an integral part of the model. At last count, SEI had sold and distributed more than 2,000 of the books, with most going to other high schools in Summit County. Looking ahead, Deevers said SEI will work with area schools that implement the course to track outcomes for their students.
As far as Eman Hassan and Brynne Burgy are concerned, the outcomes of SEI’s approach to preparing students for the ACT are crystal clear. Thanks to their strong academic records and solid ACT scores, both will attend the University of Akron in fall 2017, with Hassan planning to study mechanical engineering and Burgy considering a geology major. Neither student will have to take remedial math.
Asked if she has finally kicked her test anxiety, Hassan said, “I’m sure I will still have to deal with it, but that course helped in a really big way.”
What changed in Akron, Ohio?
The Summit Education Initiative is working with its high school partners to implement a home-grown solution for increasing student ACT scores so they enter college “remediation free.”
- Juniors enrolled in the pilot ACT Math prep course raised their scores by an average of 3.4 points (on the ACT’s 36-point scale) between pre-test and the ACT.
- The percent of students who scored high enough to avoid remedial math classes in college rose from 53 percent in the pre-test to 83 percent on the ACT.
- Student performance rose for students scoring at all levels, with the greatest gains among those with the lowest pre-test scores.
- Now, SEI is working to take these results to scale at other schools through the publication of a new book.
Build your case for action with good data. SEI gathered and compiled eye-opening data at the county and school levels that convinced partners of the need for action.
Pilot and improve. SEI was deliberate about beta testing and piloting the new ACT Math prep course so it could evaluate the approach and make improvements.
Find enthusiastic partners for the journey. In Mona Kotran, SEI found a willing partner to help it pilot the new approach and offer feedback along the way.
Always be thinking about scale. Once SEI had good results from the pilot, it quickly set out to refine and scale the approach so the course could reach more students.
Partnering for success
Summit Education Initiative is a nonprofit organization in Summit County, Ohio, that is working with schools and a range of other partners to improve personal and regional prosperity through increased educational attainment. SEI and its partners envision a future in which every student graduates prepared, passionate and persistent with a clear career pathway.