Evaluating EdTech With the Right Stakeholders atthe Right Time

Casa Grande Elementary School District, which sits about an hour outside of Phoenix, is “a community of learners, leaders and innovators.” Like many others around the United States, this rural K-12 district had to suddenly shift gears to remote/hybrid learning at the onset of the COVID19 pandemic in the spring of 2020.

Casa Grande quickly realized that in order to support students and successfully manage its growing ecosystem of edtech tools, it would have to effectively bridge technology (IT) and learning (curriculum and instruction) team goals, and streamline processes.

Casa Grande looked to solve three main challenges:

1. The district lacked a single place for viewing and tracking approved edtech. The use of digital tools increased significantly, and the district did not have a central hub for teachers to see which tools were district or school-supported.
2. Processes for requesting, vetting and purchasing edtech products were scattered. Teachers in the district requested products through a Google Form that was then tracked in numerous spreadsheets, which may or may not clearly identify if a tool was compliant with state and/or federal student data privacy guidelines.
3. The district was unable to measure academic return on investment, i.e., connect purchased tools and usage to student academic growth. District-purchased products capture a large portion of the budget, and leaders wanted to ensure they were using products that positively impacted students.

Casa Grande started by building its product library in LearnPlatform. It began with tracking usage of priority products it would eventually run rapid-cycle evaluations on. This was a critical step in methodically building a library so they didn’t have to do it all at one time, realizing their limitations during an already uncertain, hectic time. The team also shifted its scattered request, vetting and procurement processes into a single request workflow in the platform, a process that has already generated positive results.

In their first round of rapid-cycle evaluations, the team at Casa Grande discovered that, for its higher achieving students, the math product had a slightly positive correlation to outcome measurements. This result gave the team a solid baseline and benchmark for which they would continue to evaluate and grow moving forward.

After this first evaluation, Bradshaw and the other district leaders identified the most beneficial times and ways to implement the math product, and rolled out recommendations to schools. The district worked with principals, sending timing and target usage recommendations, as well as updated implementation plans that could be passed on to teachers.

The Casa Grande team agreed not to make any product decisions during remote learning – it instead decided to build on the first year of work, reflecting on what went well and what could be improved for subsequent analyses. The team realized that the timing of running RCEs is absolutely critical to making informed, timely decisions. The first set of analyses serves as a solid baseline for the district as they plan to run more rapid-cycle evaluations on products leading up to their budgeting season.

In addition to running more analyses focused on ROI and student outcomes, the district will structure its incorporation of teacher feedback related to edtech tool efficacy and implementations. The team at Casa Grande recognizes teacher voice as an essential piece of edtech evaluation, and plans to use LearnPlatform to systematically incorporate this qualitative data alongside the quantitative data reports from rapid-cycle evaluations.

When speaking about the evaluation of education technology, Bradshaw consistently mentioned others in the district as essential stakeholders: curriculum and instruction, teaching and learning, information technology, principals, teachers and superintendents are all critical players. She emphasized that district leaders have to step back and think about what information they have, what may be missing, how to get that information and who should be involved. While it’s easy to get caught up in the day-to-day of trying to get things done, leaders have to ask questions like: Whose voice is missing? What data do we have and how can we move forward with it? How can we best represent students’ interests?