How Districts are Leading with Evidence: A Panel Discussion

Okay. I think we're gonna go ahead and get started. Even though it's a small group, we're using a mic so others can enjoy this video afterward. And we're gonna make this a little bit of an interactive session since we've got a more intimate group So you might get mike miked and put down for posterity as well. Hey, everybody. I'm Christie Lindell.

I am the VP of Product for Instructure. I came in, seven months ago through the Learn Platform acquisition. And, yeah, couldn't be more excited to be here. So we're here today and it's not just me here talking to you about districts and providers and how we're approaching ed tech management and effective ed tech, ed tech effectiveness, but you don't wanna hear from me. I've got the experts here.

And so, I've got with me Martha, from Vancouver Public Public schools, and she'll do a little more of an intro about Vancouver and herself and her role. And also Steve Moose, who's the cofounder of, Voice Thread. And just heard there might be a couple of you in this room that are users of voicethread. So we're gonna spend a little bit of time making this a, more interactive asking questions. And so as we go through, you know, think about the questions that you might have, and we'll definitely have time to get to those as well.

So before we get started, it would be super helpful for us to know who's in the room. So I've met two people. But tell me who in the room here is from higher ed. Excellent. Okay.

So we've got a couple of higher ed. What are your roles, in your, institutions? Okay. Excellent instructional design. Okay. Excellent.

Alright. So are the rest of you from K twelve? Let's see the K twelve vans. Excellent. Okay. So those K twelve, how many teach do I have? Okay.

And how many people in the tech department? Okay. A good number, sort of ish, instructional design technology coach. What other roles do you have in, Q twelve? Anyone? Canvas admins. Okay. Lots of Canvas admins.

Okay. Excellent. Oh, excellent assessments. Yep. Yeah.

So is anyone in the room a current learn platform, a customer or user? Oh, excellent. Where are you from? Okay. Excellent. Welcome. Yes.

And then yes. Oh, yeah. Excellent. Nice to see you. Keep talking.

Oh, I will repeat it. Okay. I got it. Thank you. Anyone anyone else in the room, not from K twelve or higher ed, a different role? Yeah.

What do you do? Oh, excellent. Welcome. Maybe you win the prize for coming, furthest and fastest. Okay. So we're gonna go ahead and get started.

Just to set the stage, I'm gonna give a brief brief overview of learn platform, and then I'm gonna have Martha talk about her school district and her role and then we're gonna have Steve talk about voicethread. And so we'll just set the stage with that. And then we'll just lead into a couple of questions and we'll keep this pretty casual. So I think you all saw this earlier. You've probably seen it at least four times.

Learn platform is was founded in twenty fourteen to basically expand, the equity and access to the right for every student to the right teaching and technology that works for them. And so we've been tracking usage of tools for the last six years And you see that general rise. But you see that it hasn't flattened out, you know, so we had this huge spike that we experienced a few years ago, but that hasn't gone down. I think actually what's behind the scenes that's even more interesting to me is we've also tracked the number of tools per student, unique student, unique tools that they use. And what we found in US districts is there is about forty two for both teachers and students, the number of different tools they're using through the school year.

Which is not that many. So how do we get to twenty five ninety one if each student and teacher is only using, you know, a a lesser number? Forty two. So if that's the case, I think it means that there's a lot of autonomy going on. There's a lot of individualization and a lot of choice. And a lot of people using different tools.

So for those of you that are trying to manage things at an organizational or a central level, that's a big challenge. So just knowing all of the things that are going on, you know, I think, I didn't start with learn from the very beginning But the stories I heard were, you know, we were were a research based organization. We were really founded with the mission. We actually have our founding researcher right there. Not wearing the panda suit.

Danny. Stanley, how Doctor. Danny, and we actually have a couple of other, of our learned colleagues, Greg Terry Madra, cherubouni, and, Sarah Cornelius. So thanks for joining you guys. So we we thought everyone would be ready to measure the outcome the tools in use, but no, they're like, hey, could you just tell us what we're using? And could you tell us how often we use it? And then we can start our journey to really manage this ecosystem effectively.

So, we learned a lot, and we know that it actually doesn't matter if your k twelve or higher ed There's two things that are really important. I need to make evidence based decisions. I need the data. I need the data in one place to be able to choose what I'm gonna adopt and what I'm gonna continue using. And I need to do it efficiently because we're all so busy.

There's no way to manage that number of tools efficiently in all of the moving parts. There's always different stakeholders involved. There's different processes. There's communication of your decisions that you're making and all that needs to be handled as efficiently as you can. So that's why we're here.

And so these are some of the things that we do. I'm not gonna read the slide. I'm just gonna point out a couple of things. We do have eleven thousand tools in the library, available with lots of information. So the one thing I will call out down here is we do bring in third party, certified locations, such as one ed tech.

We've got the trusted app, certification. We are looking actually to expand for those of you that track, LTI cert citation common cartridge, some of the other, one end tech. But we also partner with, Project Unicorn. Is the student data privacy, pledges in the platform, and there's a number of other things on data privacy, and the quality of the tools, that we make available. We're also tracking additional things like accessibility.

You know, I sat in a session yesterday. Oh my gosh. I think he the person suggested that every tech director actually vet a tool for accessibility. And I'm like, wow. They said they're supposed to go page by page and no.

All of the requirements and be the accessibility expert, you know, like, wouldn't it be great if we just had a third party resource to give us some sense of the accessibility of the tools, and a and a promise? So we have all of that to inform those decisions. Oh, I have some checks. And then on the other side, we will also work with ed tech providers. And so those ed tech providers, are engaging to build and grow their evidence space. And so we actually have a really talented research team that partners with people like voicethread, to build their evidence.

And so they can, convey to their customers why their products were built. The outcomes that they were built to a con to accomplish and achieve for teachers and per students. And then they can prove that those results are actually happening, and it's that measurable piece. And I think what you'll find here is the reason that we're all here is this is a partnership It doesn't work if we don't have so we're the platform and the facilitator. But it doesn't work if you don't have the ed tech companies partnering with the institutions and with the administrators and the teachers to make all of this work together.

And so we'll talk about how that happens. Okay. So we're coming together, and I'm just gonna turn it over to Martha to talk a little bit about Vancouver. Hello, everyone. My name is Martha Dela Past Casto.

The digital content administrator or the software administrator for Vancouver Public Schools, and that's Vancouver, Washington, not Vancouver, Canada. We went into a learning platform really wanting to understand and monitor what teachers were using and how they were using it in their classroom. As well as build unity across departments, for our organization, a lot of the departments use different programs. And so we wanted to really collaborate with them and really, build a community within the organization so we all can review and approve the applications that we use together. We also want to standardize that process so that it's automated and it's a really nice work flow throughout the review process and hopefully approval or denial.

Some of our challenges are really managing and ensuring that they're engaged. And what that means is when a app is approved, we wanna make sure that the teacher receives appropriate training. So we have dedicated teams for that. We also want to provide visibility. So that means we have a public page that Learn platform provides.

Where the, staff, teachers, students, anybody can actually go into there and see what we have. And we want to make sure that the teachers understand why these applications are safe or not safe for the school district. And I'll hand it over to Steve. Here we go. Can you hear me okay? So, Steve Booth, a voice read.

And, we've been at this for about seventeen years, and we really didn't know anything about education whatsoever when we got started. And so we have learned so much. And one of the things that we learned early on was, There is no such thing as, like, the education market. There's a thousand of them. And so one of the most important things we learned right away was, you have to listen to all the different people that you're gonna be working with because they're what is on their plate is gonna be different for all of them.

And so, we've really just developed this ability to practice listening, listening, listening, and trying to get to the harder problems so you can develop generalized solutions that will then work for everyone really flexibly, and we'll probably get into this a little bit later. But as far as the company goes, we're still at it. We're still very happy. We like what we're doing. We still think the core problem that we're trying to solve is as big today as ever, and that core problem is that there's not enough high quality interpersonal interactions in learning experiences.

We're stuck in this world between Zoom and a text based discussion board or an email. Like, that's the world we navigate Zoom is too hot, anxiety producing. It's got all kinds of issues, even though I love it. We use it. It's necessary.

But It's nice to have something in between. It's a little more low stakes, and less anxiety producing for students. So we're still at that, and we're gonna keep going at it. But we're here today, I think, to talk about this, issue of how do we all work together to figure out how we improve what we're doing because we have a real problem that we sit on this data lake that is really rich and incredible. It doesn't belong to us.

Belongs to our customers and the students individually. So we're sitting on it. We're not using it. And it a little frustrating because even though we're contractually allowed to use it to improve the tool, we don't have access to all contextual data. That our customers have.

And they're the ones who can tell us, like, this is, this is working, and that's not working. We can only sort of guess at it. It's like shadows on the wall. And we need, to get more information in order to do that. So immediately when we saw the Learn platform, like, right off the bat, I was like, oh my god, this is, this is an answer.

Somebody needed to solve this problem because it's too big a problem for customers, too big a problem for us. Somebody needs to create this like Rosetta Stone. So that, like, data can be shared. Safely, securely. People who are, competent to evaluate data, are doing those evaluations rather than you know, either us or, you know, some instructional designer who's a, you know, army of one at a community call and, you know, tasking them with, you know, analyzing, we have over three thousand research papers that talk about using Voiceerette.

That's not fair to hand that on the plate to this instructional designer and say, yeah, why don't you go figure out if any of these have any validity and, you know, actually point towards real changes and outcomes. So anyway, this is what we're up to. We're still we're still at it. And, excited to have this conversation. Thank you, Steve.

Okay. So we have a couple of questions and form your own questions. As we go. And so I think we're just gonna start with the basics. So I'm gonna start with Martha.

So we're always gonna kind of start our our thoughts with How are how is the school district, thinking about this question? And then we're gonna think about the flip side. Well, what does this mean for the ed tech company and the provider? And how are they thinking about it in screen. So the first question is really a Brown. How I guess the question is less about how you define a tech management But how do you do ed tech management? So talk to us a little bit about in Vancouver what ed tech management is for you. Okay.

Thank you. ETech Management is really integrating the multiple departments and the multiple staff and stakeholders that are part of this review process. And what we want is we want an application that really integrates with the student's current instructional needs, as well as has, data security and safety for the students. And, for us, it really starts with that. Of, we actually are not the definers.

It's the people who are doing the work, the, the administrators, the teachers, they're the ones who are gonna know and they all have a different, agenda. And I'll just give you an example, like, one sort of nitpicking example of, If you give it a student assignment to make a presentation and they make twenty two comments in that presentation, that sound that's excellent. But then there'll be another student who chooses to spend five times as much time, does five times as many attempts and then produces, actually, like, a higher quality product, but does it in five comments, which we see all the time. Evaluating, like, what worked and what didn't, that's not for us. Like, I'm not the instructional designers who set up that assignment or knows what the learning goals are.

We can't from our side. So it's, you know, if we were to extrapolate and say, well, a student that did twenty five comments, they're clearly doing better, aren't they? No, we, we have to be very humble and careful about making assumptions about the data. So, it really, for us, defining effectiveness, it's more of a job of listening. We actually don't do it. It's not our goal is to define effectiveness.

That's for the people who are doing the work to define it. What we really need to do is just enable them to do their job better. And I think that's kind of fundamental. So we're getting used to these microphones. Okay.

So the next question for me is, are the what are the biggest, you know, you're managing a lot of tools? Maybe you could tell us how many tools you have, reviewed and approved for use, what are some of the challenges? And, you know, and I guess what data do you use to make decisions? About what tools you're gonna start using or, or continue using. One of our biggest challenges is really finding an application or software that can do more than just one thing also is really when an application expands, it's great. They can do more, but also, for example, Nearpod recently expanded to allow YouTube, but YouTube is not allowed for our students in our district. So we have to quickly figure out a way to turn that off. And really figuring out how to disable that and really be in contact with those providers so that we have a clear gateway of communication in case those events do occur? For us, the biggest challenge really is managing, I had mentioned this before, we've got this huge amount of really hot PII, really wonderful rich data that it's actually why people want to use us because teachers love hearing, seeing their students.

They, they want that rich data. They can use it locally. We're sitting on it. It's really valuable, but we kind of view it as toxic waste. Like, it can only be dangerous and hurt us.

And it's actually not appropriate. We should be making it available in a useful format to our customers. In a way that they can use it. But a lot of them don't have the sort of infrastructure to do that. And again, I go back to why the Learn platform comes along and sort of ends it sets up that infrastructure where they see, oh, we can we can help make those connections so that that data can end up getting used.

And that's kind of our long term goal, and I can see it in the development of the Learn platform over time. That's where we want to get to so that there is a more generalized and seamless use of student data to improve teaching and learning and our own tool, like all of those things that gotta happen together. So I had just a follow-up question for you, Martha. I think you mentioned when we talked yesterday that you are doing a review of a number of tools this summer. I would love for you to talk about kind of what that review process is and how do you make decision What are you looking at, or what is your rubric that you use or the process you use when you're evaluating tools? Sure.

So I am using a great rubric that was compiled by my instructional technology director. And what we really look at is if it's compliant with Kappa and ferpa, really data safety and security, and want to really see it will benefit the student within the classroom? Will it be part of those forty two applications that they use or if it will just sit and lie there, and the learn platform has a great tool and analytics tool where it tells you specifically how many students are using that application So I'm going down that list and really checking to see which classrooms, what schools are using that application. And if the data isn't being collected, then I would go ahead and troubleshoot or really evaluate and go over with my instructional director and see if we we need to get rid of it. Excellent. Thank you.

Okay. So we talked a little bit about data points. Maybe I was getting ahead of myself here. I'm curious, Steve, what do you see? You know, because Martha is you know, obviously managing many tools in one organization and your one tool that's getting used in many organizations. So what are you seeing? When you're engaging, with prospects or customers that are are repeat, you know, what are they looking at with voice thread? What kind of questions do they have for you about your product? And when they're when they're making decisions about whether to use, voice thread? I I think everybody is actually looking for the same stuff, which is in the end, efficacy, like, that you're spending people's money.

It should be spent on things having an impact. That's like basically all of our jobs. Right? So if if everybody wants that same thing, but then they all have different capacities. And that is something we really do have this bird's eye view. We work with some of the biggest, most, you know, prestigious schools in the world for, like, over a decade.

And then we also work with, like, you know, I tell the story of, like, community college with ninety thousand students and four tech people in tech support. Both of them are customers, but they have different capacities. Right? So they, they have different needs. So, again, I go back to we have to listen first, like, what can you handle? What do you really wanna know? Like and so there are different levels. Sometimes, you know, they don't want it to be they're not, if he was just four of the those people, they're not looking for a deep dive.

They're like firing like they're just like, what, put fires out, keep the ship afloat and have, you know, you know, be reliable, all that kind of thing, we have to respond to that and say, okay. Well, we can give you this information. Does that help? And, so we'll help them in a different way than the institution is of the same size, but has essentially five hundred people in a role of supporting people. They're usually it's role reversal then, and oftentimes it'll be that large institution. They're the ones who are, like, coming after us and are like, okay, we want this, this, this, this, and we want this upside down, and we want, you know, A and B comparisons, and all of that's great, but we can't treat them both the same way.

So, I mean, I think can that's why tools that are sort of that are and platforms that are built to allow the customer to use it in the way that fits them, that is really that it's a real, very helpful for us. Because it's very hard. Like, we are You know, we're not that a scientist either. Right? And so, I mean, I look at the research on Voice Red, but I can't evaluate it. And neither can most of our customers in a really vigorous way, because it's not all of the same quality.

And as we went through our process with the Learn platform researchers, we went through. There was like three thousand research papers. And we had to win all of that down because it's all of different quality. But they're qualified to do that. I'm not the most customers are not qualified to do that.

And so, we're big fans of all of those certification programs, because I feel like if they run well, that's exactly what they're doing. They're sort of helping everyone out by, you know, having real professionals on staff doing that kind of work. Okay. I wanna open it out to all of you for a minute. So if we're talking about making decisions, Martha talked a little bit about data privacy being really important, in terms of a key, you know, I'm gonna adopt use the tool.

I absolutely have to make sure that it's compliant with data privacy requirements perhaps by law and, for sure, by my district. And you're talking about, Steve, you're talking about research. What kind of outcomes has this product produced and how can I, interpret the outcomes that I would receive or am, you know, or I'm gonna wanna measure my own outcomes and how do those compare? What other data points are any of you or your institutions using when you're deciding to use, tools? Anything else? Yeah. Hi there. One of the things, I'm in from Austin ISD.

It's a K twelve institution, with roughly seventy five thousand students. One thing that we consider a lot is just overall how many of our users are actually using the tool that we're paying for. So just straight up, just usage, And when we evaluate that, oftentimes, we see we're we're way over licensed. Right? We've we purchased, licenses for all students, but maybe only a few thousand students across a handful of campuses are actually using the tool. So that leads to a lot of conversations about what was the implementation plan.

Should we scale back or should we scale up our our support for this tool? So that that's a big one for us. That's excellent. Yeah, usage is key. It's usually the first place you start Right? You know, so just what is getting used? Who's using it? What did we think was gonna happen? And how does that compare? I think there was a great book, like, way back in the eighties or something that was, like, oversold and underused or something like that. How about anybody else? Anything else that's key in your criteria Yeah.

Yeah. Always compatibility, right, with whatever other platforms you're using. Right? So, like, you don't automatically assume that it works with Canvas. Right? So, especially if you're looking for something in particular. So whenever you're doing com I mean, I'm from higher ed, so I'm from Florida International University.

I'm sort of like their ed tech specialist at their teaching and learning center. So compatibility is definitely one. And then also when you're looking at effectiveness, right? Like, I think you kinda mentioned that, like, what, how many things can the tool use? Or in some cases, I'm looking for something very specific that it needs to be able to use. And can I get that that particular tool? And then I think the last one would be ease of use as well. That's I mean, we we've gotten so far and still, you know, a lot of folks challenges with just like, okay, what are the steps just to do, you know, just one thing or just to get this done, right? So, yeah.

Yeah. Cause guess who gets to support those tools and all those questions. Yeah. People in this room. Anyone else? Yeah.

I would just add to what he said, you know, ease of use, obviously, is a big issue, you know, for students and whatnot and what software programs you're using along with it. The other issue we run into is we have multiple devices that students are using. And a lot of students are using mobile and iPads, and we have desktop computers, and we have laptops. And so it looks different on every single one. So to be able to support that, that's something we have to go through and check and make sure that it's compatible.

Okay. Sort of a loaded question. Anyone concerned with accessibility? Anyone not Yeah. That's a better question. That's good.

I come from Maryland, and we recently passed state legislation that requires all districts to review and vet our digital tools for accessibility. So we're going through that process now and have to submit a report to our State Department of Education, October first, and then October first going forward every year. So that is a pressing matter that we're facing right now. Yeah. I mean, definitely driven by good.

I mean, if a product is accessible. It makes it better for all users. Are there are there requirements that are helping you to to, review those tools? I'm just curious. Requirements from this state or those are a little vague. So we have, you know, teams from our special education department who are doing those, those reviews of our digital tools.

But okay. Yeah. Standards could be a good thing as they emerge for accessible Right? Just we do have VPats. We have WICK standards. We have tools, but they're not really easy to interpret and easy easy to know where a tool stands and you know, there's a lot of nuance to it.

And so And the the VPAT will be, I believe, required next year's submissions, but a lot of vendors don't have the pets or haven't heard of them when you ask for them. So Yeah. Definitely emerging. Thank you for sharing. Thanks everyone for sharing.

I think we talked about it just a little bit here, but the last question I wanted to segue to talk about was really about implementation. And so if I am a district administrator or an institution administrator you know, I might look at my usage and say, well, how were we planning to use this tool? Who we were who did we think was going to use it? And, you know, and you might just start there. If you're an ed tech vendor and I happen to work for one now, and, I worked for a few in the past. There's there's definitely a tendency to point fingers and say, well, they didn't implement with fidelity. And so they're not getting the results.

And so we're in this because there were all these implementation challenges that were beyond my control. I would love to just, hear a little bit from both of you about some, you know, how do you look at implementation fidelity and how do we think about managing that in a really positive way? Our district is really great in that it has a specialized team of instructional educators. They are called digital learning coaches, DLCs for short in our district who once an app is approved, we hand the to them. They learn everything about the app, and then they build all the instructional content. They build, group trainings, one on one trainings, all of the instructional content that's needed they put in a Canvas course, or they do documents, they do newsletters to introduce the product, They are a really great team to implement, implement the any product that really comes into our district.

Yeah. When when we hear that, we get so excited because we're like, oh, this is gonna work. Right? Because that is the missing piece. We can't do it, like, You know, we have our everybody has a role to play. You guys can't build this stuffs that we do or manage it.

It's crazy complicated. We can't do the implementation part, we, for a bazillion reasons. So we have to have this, it's a missing piece sometimes, and it's tragic. It's like you just get so upset watching it because it's so hard to try to get this to happen and that it happens And then you're like, wait, none of no one even knows that our tool exists. Right? There wasn't a blog post about it.

Right? And so the expectation that it's just gonna get picked up in use is really, that's not fair to us. But I think all professionals you always have to be expanding your realm of responsibility and say, okay, maybe it's not my primary responsibility, but maybe there is something I can do. And keep expanding. And, and I think teachers do this all the time. They probably overextend, right, and take responsibility for too much.

And, and yet So we still look for things. And for us, we still I can't march into a classroom. I can't, you know, arrange teacher trainings. I can't do any of those things, but when can do is work on what was mentioned earlier, which was simplicity of implementation, right? We, so we listen for, like, what are those problems? Like, what do you need that would make that easier? And that, so that becomes our focus. That's how we help the implementation.

But we would never, we have a template that we give, but it we we show it to a new customer. We're like, Here's what some other people have done, you know. So it's an editable document. You go take it, and if you don't have one, like, they would need it. But somebody else, it gives them a sort of heads up.

Like, hey, you may want to think about these five things during an implementation. In case you hadn't done that. But it's really for them to customize and make it local and make it work for them. So that's an example of us like we don't want to step on other people's toes, but we just offer it. Like, here here's a template, you know, take a look at it.

So that's kind of how we but implementation, it's There's actually in the health sciences field that's called implementation science. You can get a PhD in it. That's how big a deal, how big deal implementation is. It's not like a sideshow. Like, you can have the best everything.

It's like the proverbial missing leg on a stool, like, it ain't gonna stand up, you know, without that being done properly. So it does have to get done. So I'm curious, anyone wanna share implementation, challenges or best practices that you have yeah, I'd love to hear from you. Thank you, Matthew. So I'll share a challenge we actually have a very similar system to what you mentioned for Vancouver Public Schools.

I work in Orange County Public Schools. We're a very large district, two hundred and fifteen thousand students. Whenever we purchase a new product, we advertise it to teachers. We have newsletters. I work on the digital learning side of things.

Sort of like a digital learning coach, but we also work with the curriculum coaches as well. We're one big happy team. We all fall in the same department. But at the end of the day, in Florida, especially in Orange County, teachers are allowed to choose what they do in the classroom. So that seems to be not a problem necessarily, because I think that's great as a former teacher, but that usually hinders fidelity.

So it's like, yeah, we buy all this cool stuff. Let's use it. Oh, no one's using it because they're used to doing their thing. So that's an area I'm curious. How do we get more teacher buy in? That's an awesome question.

Alright. Do you have thoughts on that? I do. To follow-up on that, as a district, I've come from about three different districts. This final district the same as the others have very different cultures not only within the district, but within the schools, and sometimes within the classroom. So you'll have a lot of teachers, I wanna say, throwing out random number fifty percent that are really excited to work with the DLCs and want to learn everything about a program.

And then we have the other fifty percent who, you know, have their way, and it's a way that they want to keep. So it's a little bit of a challenge for us as well. But we just really, we hype up the program. We really advertise it. We really get them excited about it.

We demo it, and we even do some code teaching with the teachers to try and get them excited about it. Makes sense? Anyone else have any answer to that one? Yeah. I think definitely we can, you know, I love the idea that you it about, you know, you have this plan and you have these DLC coaches. I wish we had those. But we have STS's in our school at this which we call school technology specialists, and, you know, we help support them to support inside their school.

We have one in each school. What I have found, I think that's I'm hoping to be really beneficial, not to put a plug out for another Canvas product, but we recently adopted impact what I have so far used that for is when I have new products, that we wanna implement, I do more of instead of like, this is a required training because time we know is teachers don't have it and schools don't have it. And what little time they do have in trainings at the beginning of school is taken up by school administrators, things like that. So one of our thoughts was let's do an on demand. So we have resources, we have videos, we have things that we purposefully make pop up on certain screens.

Like, if you're going to the grade book, it's it's gonna pop up and you're gonna have some support there in videos and materials you know, that teachers can actually access in the moment when they need it. And we're finding that that's been pretty helpful. As I've looked through statistics to see how many people are clicking on that and actually watching it. It's been helpful. And we're hoping that it's in the moment of need, whether it be at two AM.

When that teacher's going through that grade book, they have those resources right there to click on and go, oh, gosh. You know, I didn't know I could do that. And we did that with, like, quizzes inside the studio, you know, Canvas studio. And I had a lot of people watch that and go, oh, oh my gosh. I didn't know I could do who is in there.

Although, we've had studio for two years or three years, you know, and it's like, oh, so to see the success in that where I was like, oh, okay. They didn't initially do the training. But here's another place. They can access it. So just a thought.

That is awesome. Thank you. I'm glad you brought that up. I actually just recently, took on responsibility for the impact products, so I'm just learning. So that was really good insight too.

But I think we have ideas and hopes, actually, for the impact product to allow, you know, the view to use that for your messaging and for your resource, you know, for any tool, not only the the canvas or the instructor products. So yeah, more to come. Let's see. What kind of questions do we have from all of you? What what else would you like to talk about? Give me questions for each other for anyone on the panel? Yeah. Go ahead.

Question I'm sorry. I did have a question regarding, analytics, I guess, tracking data. So when you're tracking student usage by application, is that a partnership between the vendor and the platform? Is that how so let's say that I have a tool that is not part of your learning platform right now. And I wanted to get capture specific data from that tool. With the steps be that I would have to talk to that vendor to get in contact with the platform.

So that way, they can start to build a partnership and then discuss how to get some of that information over. Would that be the process? So that's a great question. So what Learn platform developed a few years ago and we actually just expanded is we have, browser extensions in iOS app extensions where we can track data from both students and teachers and the apps that they're launching. So it's pretty generic, launch data, and, it can be minutes on system if it's the browser data. But we track the the general usage of any tool that is found in the learn library.

So it doesn't have to be an approved tool it can be any tool in our eleven thousand, tool library. And then we track using those domains in the way that they're logging in so you can get a general sense of what's being launched and used. Yeah. Of course. Yeah.

There is a product that's not being tracked and you wanted to be tracked, you can reach out to one platform and we'll add that to the technical tool. So it's eleven thousand, and it's growing based on our customer. It's amazing. So the extensions, or is that something that's this so are the extensions something that is the extension something that's automatic or is it something that has to be in stalled any point by anybody. Definitely not automatic.

I mean, that's, at an institution's choice. And so, yes, we would give you, instructions to be able to do that. And then, yes, the, institution themselves. Are the ones to deploy those. And I know, you know, we have several examples probably in this room of people who have gone through that process.

It's, yeah, It's not a difficult process, but it's definitely a partnership that we go through. Alright. We've got about four minutes left. Anyone have another question. Anything that we should have asked? Yeah.

Okay, Greg. You're you're our sales expert in this room. What question, should we talk about? What do you hear? Often? Thanks, Chrissy. So, you know, it's it's pretty it's pretty interesting as we're talking to to districts, that often have the need to understand what's going on inside a canvas that click throughs, how it's being used, the fidelity of, of use within canvas But I don't think that's any different outside of the LMS as well, right? So understanding that there's this huge ecosystem that's out there The average district, as you pointed out, is using thousands and thousands of tools. And it's just not a sustainable practice, right? We keep thinking about this number cannot continue to to, to grow, we're dividing our attention.

And so I think just for me, you know, being at at the conference altogether, that everyone is saying a lot of the same things, just saying it different ways. And, and so I think there's a huge value in just knowing, well, what's out there what are we using? What do we do about that? What do we think about that? Every district has a different viewpoint. What we're doing in, in Maryland, around accessibility, might be a little bit different from what's happening in another state that really does value accessibility as well, but they're having the conversations a little bit differently And so just being able to wrap your arms around that ecosystem and know what do we have? What's out there? And ultimately, you know, the goal for any of this, and this goes back to our conversation with our, our provider partners, is does it work? Is it, is it doing the job that we hired it to do given our use case and our students, and and, resent only the answer is always yes. It it works. When we do it well.

And so what does that mean? You know, how do we, how do we measure that? How do we monitor that? And how do we just be intentional and strategic about what we're choosing to use and, and not use. Yeah. I don't think Martha brought this up. But she shared in some of our conversations that, you know, in some cases you have multiple tools that do the same thing. Right? So having, you know, so there's a lot of, I'm hearing a lot of discussion to that point of consolidation.

So how do I identify those tools that are similar? And then how do I evaluate? Like maybe they all have a great use and and so I just need to understand that. But I wanna look at I wanna be able to identify them. I wanna look at all the data, and then I wanna be able to have a conversation. So I know that's that's definitely a piece of that as well. Okay.

Well, I really appreciate you guys, you know, helping us to make this really meaningful and interactive. I really appreciate Martha and Steve joining us. I hope that we've given you a little bit of a flavor of I think working together, how we can really manage. You know, this this big ecosystem, but we can do it to get better results for our teachers and for our students, and I think that's why we're all here. And so thank you very much.

And we do have another session that goes that does have a learn platform demo. So if, this wasn't quite enough and you this you're looking for more specifically, we do have a session five fifteen where we're, doing a follow on or come see us. You know, we'd love to talk to you. We're here all day. Alright. Thank you everybody.