EdTech Evidence: What Every School Needs to Know

In today’s tech-powered world, education technology (edtech) isn’t going anywhere. However, savvy K-12 educators also realize that tech, for tech’s sake doesn’t improve teaching and learning outcomes. That’s why it’s critical to consider evidence when evaluating edtech tools. However, weighing evidence in edtech decision-making is a relatively new muscle for schools across the country.

In a recent survey conducted by ASCD on what factors impact educators edtech decisions about which edtech to use:

“Only four percent of educators indicated they used evidence-informed decision making—relying on research evidence and professional expertise to inform new technology implementations. Fifty-four percent said they use their professional expertise and knowledge of context, but only six percent used scientific research.”

An educators’ professional expertise is invaluable. Pairing this professional expertise with high-quality research provides a new opportunity to build robust evidence-informed decision making.

The good news? You don’t need research expertise to analyze the evidence base behind edtech solutions–you just need to know what information is critical. 

In this article, we’re going to cover the basics of edtech & the evidence you need to evaluate its efficacy:

• Edtech: What It Is & Why It's Important for Today’s Schools
• How to Evaluate Edtech: 7 Factors Schools Should Consider 
• What is Edtech Evidence? 
• Why Does Edtech Evidence Matter?
• 4 Essential Research Questions Every Leader Should Weigh

Edtech: What It Is & Why It's Important for Today’s Schools

First things first, edtech is the integration of technology into education to enhance teaching and learning. Edtech broadly refers to many different tools, from interactive whiteboards and tablets to online learning tools–like a learning management system–and educational apps like Quizlet and Nearpod. 

The importance of edtech in K-12 schools cannot be overstated. First and foremost, it empowers educators to create more engaging, interactive learning environments. With edtech tools, teachers can expand learning beyond the walls of the classroom by incorporating video learning, scale differentiated instruction, and gamify learning to capture students' attention and make complex concepts more accessible and memorable.Additionally, edtech facilitates collaboration and communication among students, teachers, and parents. 

Edtech also plays a pivotal role in preparing students for the future by equipping them with the essential digital literacy skills they need to succeed in college, careers, and beyond.

How to Evaluate Edtech: 7 Factors Schools Should Consider 

In our research, we’ve found that the average school uses over 2,700 edtech tools. This means that schools are no strangers to technology in the classroom, but given the large quantity of tools being used, understanding which tools impact student learning can be challenging.

Schools need to critically evaluate every tool in their environment. Doing so can be resource-intensive, but here are eight factors to consider as you cut through the chaos:

1. Alignment with curriculum and learning objectives: Sometimes, a flashy new tool can end up being a bigger distraction than impactful. Ensure that the edtech tool actually aligns with your school's curriculum standards and learning objectives.
    
2. Ease of use and accessibility: The tool should be user-friendly for teachers and students, with intuitive interfaces and minimal technical barriers. Consider all the levels of digital literacy across your faculty. Do they tend to be open to technology? Or do they require more time to adopt new tools?
    
3. Interoperability and integration: Evaluate how well the edtech solution integrates with your existing tech stack, such as your learning management system, SIS, and other educational tools.
    
4. Personalization and differentiation: Assess the tool's ability to provide personalized and differentiated learning experiences. Look for features that allow adaptive learning paths and customizable content to cater to individual student needs.
    
5. Data analytics and reporting: Robust data analytics and reporting capabilities are essential for monitoring student progress, identifying areas for improvement, and informing instructional decisions. Evaluate the edtech solution's ability to provide meaningful data insights and reporting features. (You’ll also want to be sure that any vendor you work with has a robust student data privacy policy–after all, you’re entrusting them with precious PII and technology progress so does the importance of cybersecurity)
    
6. Professional development and support: Consider the professional development and support resources provided by the vendor. Comprehensive training and ongoing support can ensure effective implementation and maximize the tool's impact on teaching and learning. If your teachers, students, and staff have access to self-serve forums, community, and support, they’re more likely to feel empowered to experiment with tools.
    
7. Cost and sustainability: Evaluate the edtech solution's cost-effectiveness and long-term sustainability. Consider the initial purchase price, ongoing maintenance, updates, and potential future costs. Sometimes, the “free” tools aren’t actually as free as they seem when you calculate all of the resources needed to implement them successfully.

In addition to each of these factors that may vary school to school, there’s a growing need for empirical evidence that shows the tangible benefits of edtech in the classroom. How can schools make evidence-based decisions as they weigh each of these factors?

What is Edtech Evidence?

Edtech evidence refers to the body of research and empirical data that evaluates the impact and effectiveness of educational technology tools on teaching and learning outcomes. 

It goes beyond anecdotal experiences or subjective opinions, relying instead on rigorous research methodologies to generate reliable findings. It encompasses a wide range of methods, including experimental studies, quasi-experimental designs, case studies, and observational research.

Critical components of edtech evidence include:

• Quantitative Data: This involves collecting and analyzing numerical data to measure the impact of edtech tools on student performance metrics such as test scores and grades. 
• Qualitative Insights: In addition to quantitative data, qualitative research provides valuable insights into the experiences, perceptions, and attitudes of students, teachers, and administrators regarding using edtech tools. Qualitative methods such as interviews, focus groups, and observations offer a deeper understanding of the contextual factors influencing technology integration in educational settings.
• Longitudinal Studies: Edtech evidence is not static; it evolves as technologies and educational practices change. Longitudinal studies track the long-term effects of edtech interventions, allowing researchers to assess their sustainability, scalability, and impact on student outcomes over extended periods.
• Meta-Analyses and Systematic Reviews: Meta-analyses and systematic reviews synthesize findings from multiple studies to provide a comprehensive overview of the existing evidence base for a particular edtech tool or intervention. These analyses help to identify consistent patterns, strengths, and limitations across studies, offering valuable insights for educators and policymakers.

By synthesizing findings from diverse research studies and methodologies, edtech evidence provides a nuanced understanding of the complex relationship between technology and education. 

Why Does Edtech Evidence Matter Today?

With the edtech landscape largely unregulated, school leaders must recognize the significance of research-backed evidence in evaluating these tools. Unveiling the efficacy of edtech tools through research helps make informed decisions and contributes to the overall evolution of education.

Here are some of the many reasons why edtech evidence matters:

• ESSA-Compliance:The Every Student Succeeds Act (ESSA), passed in 2015, established a framework for evaluating the effectiveness of educational interventions based on the strength of evidence supporting them. ESSA categorizes evidence into four tiers based on the rigor and validity of the research methodologies used. These tiers help educators and policymakers assess the quality of evidence supporting various educational programs and interventions. 
• Ensures Efficacy: The presence of evidence validates whether an edtech tool delivers on its promises. It serves as a litmus test, separating tools that genuinely enhance learning from those that fall short. Administrators must look beyond marketing claims and examine the empirical evidence supporting a product's effectiveness.
• Informs Decision-Making: Education budgets are limited, and every dollar spent must yield tangible results. Especially as we inch closer to the ESSER funding cliff. Evidence-based research provides valuable insights into the potential impact of an edtech tool on student learning outcomes. Administrators armed with this knowledge can make more informed decisions, choosing tools that align with their educational goals and priorities.
• Promotes Accountability: Research conducted by reputable third-party organizations lends credibility to the efficacy claims made by edtech providers. It holds them accountable for their promises and encourages product development and implementation transparency. School leaders can demand evidence-based explanations from vendors, ensuring that they invest in solutions that have been rigorously tested and validated.
• Adapts to Diverse Contexts: Anyone in education knows: education is not one-size-fits-all. The effectiveness of an edtech tool may vary depending on the context in which it is implemented. Solid research that considers diverse student populations, educational settings, and instructional practices provides valuable insights into the adaptability and scalability of a tool. Administrators can use this information to tailor their implementation strategies and maximize the tool's impact in its context.

School leaders prioritizing edtech evidence demonstrate their commitment to evidence-based practices and student-centered learning. They pave the way for a more discerning and strategic approach to technology integration, ultimately fostering a culture of continuous improvement in education.

There are several questions that administrators can and should weigh as they start evaluating research on an edtech tool.

4 Essential Research Questions Any Administrator Should Ask

Is there a Logic Model and/or Theory of Change?

A Logic Model, or Theory of Change, is required to meet the ESSA Level IV standard for evidence. These artifacts clearly define the required inputs (like student usage) and expected outputs to show how the use of a product, as designed, will lead to positive outcomes. The Logic Model also serves as a practical guide for product users, outlining the requirements for solid implementation. 

Who conducted the research?

Generally, third-party organizations like research organizations, universities, and independent academics conduct the most reputable research. When reviewing research completed by the company or researchers without strong credentials, take a closer look. When was the study completed? How many students participated? 

What context was the research conducted in?

Understanding the makeup of a test group and the context in which the study was completed will help you to understand what the findings might mean in your organization. Consider if the research was completed with students in the age group you work with. Was the study completed with a sub-group of students or students from general education? Where was the study conducted - a suburban district in the United States or perhaps another country? These questions will help you put findings into context in your product vetting process. 

How recent is the research?

Edtech changes quickly! Keep an eye on the research date you are reviewing and consider if the product has changed considerably since then. Has the student population or the school system? One pertinent example of this could be research on a digital assessment tool. The results from that study - with the same group of students in the same schools - could be wildly different in 2024 from the results in 2020. 

Start Small, Then Scale 

Building an evidence-based ecosystem takes time, intention, and effort, but the positive impacts on student outcomes make it worthwhile. Choose one action item from this article that you can get started on today, test it out, and then plan your next steps. For more steps your school can take, explore Addressing Edtech Challenges with Evidence to keep building out your evidence-informed practices. 

Every school’s edtech journey looks different. As you get more comfortable evaluating your existing tech tools, you’ll build confidence to make informed decisions moving forward.

LearnPlatform by Instructure can also help you on this journey. If you’d like to see it in action, be sure to reach out.