Intelligent Insights by Instructure: A Panel Discussion

I'm a, senior director of product management for Instructure's data and insights. So, I'll help describe a little bit of what that is. But today, we are here, of course, to talk about intelligent insights. Did everyone see the the keynote or attend the Florida? Let me ask it differently. Has anyone not seen or does not know of what Intelligent Insights is? Anyone? Couple. Okay.

All right. Well, I hit a couple slides, but we'll go through it just here very, very quickly. And I thought worthwhile mentioning, because I've actually heard it come up a couple times is like why intelligent insights? And so, we shifted a little bit and have built intelligent insights now on top of the exact same data that we expose through our Canvas data services. So I say that because we're not doing anything with intelligent insights that you couldn't do yourselves. And that's kind of the data journey that we've continued to hear about and we've seen customers go through, in varying degrees of success.

So it all starts with data. And how do you get access to that data? Who gets access to that data? And then how do you start to make sense of it? And one of the first things that we did, of course, is last year we launched admin analytics, as well as our course analytics on top of that. So, for those, I kind of see them as like descriptive. Right? They kind of tell you what happened. They're not really good indicators of things that might happen or of helping really make data driven decisions.

They're more just kind of a dashboard. Right? But they're very useful in what they do provide. But we've seen customers consistently want more on top of that. And so, you know, we've we've experienced customers that have, you know, built their own data lakes, their own data warehouses, their own BI teams to, you know, do custom reporting in house, maybe pay for custom reporting through services, try to put together the right LTI applications to really kind of pull that insights and correlate and additional information together. And, again, that's something you absolutely can do.

But as we continue to talk with customers, we really wanted to provide an easy way to be able to do that without leaving all of the burden up to our customers to do. And that's really what Intelligent Insights is built to offer. So we think about it, kind of as like a a a continuous kind of loop. And really, it starts with how do you define your institutional standards? Around your course experience or or your course readiness? You know, how do you define them on driving better learning and teaching outcomes? Like, what is a standard that you've built and how do you define that? And so intelligent insights is is really a way to help you do that. Now, once you do that, it's really important.

How do you measure and monitor against that? How do you know if you're successful? How do you know if your standards and the experience that you want to provide are being up held? And that's, again, a core feature of Intelligent Insights is kind of that real time taking action. So how do you reach out to faculty? How do you reach out to students? How do you engage further with data and continue to drive deeper into it. And then once you start to do that, it really helps you redefine what those standards are, and it starts all over. So it's just a continuous improvement cycle. And so that was really kind of the the core of why we built Intelligent Insights is instead of, again, you as a school or institution trying to do all of that yourself, we wanted to make it very easy to get to, very easy to do, and something that we can really build a strong foundation on going forward.

So while we are launching intelligent insights today, we plan on continuing to improve and iterate on that same foundation going forward. So this is our data platform and our data strategy going forward for Instructure. And with that, instead of you listening to me talk about it as a product guy for Instructure, I would absolutely love to introduce a few of our customers that have been on board and intelligent insights in one form or another for anywhere between the last six to eight months, I would say. So I'm gonna introduce them, and I've got a set of questions. But please write them down.

We wanna do a little q and a at the end, and would love to make it very engaging and very interactive. But I will start, and I'll introduce Cindy from Miami University of Ohio. And I've got some interesting facts about Cindy here as well. So, Cindy is the instructional design and technology specialist. Did I get that correct? And you've been helping faculty with Canvas and dealing with with that for? Twenty years or more.

I'm not gonna tell you how old I am now. No. We I was did was not going there. Absolutely not going there. So, well, thank you for being part of this.

I'm gonna skip to a colleague, so we're gonna mix it up here. We've got, Kit as well from Miami u University of Ohio. And, Kit, if I have this right, you're the interim coordinator for faculty engagement for regionals. Oh, mic. That is correct.

Very good. And now, I've got, also some facts here, but I wanted to share that, you are an avid hiker and amateur fossil collector. Is that right? That is correct. So we are in Cincinnati, Ohio or I mean, the regionals are located kind of just outside of Cincinnati, Ohio, and Miami University in Ohio is is located in Oxford, which is about twenty minutes outside of Cincinnati. But Cincinnati is a very famous fossil place.

In fact, there are multiple levels of fossil strata that are named for places in and around Cincinnati and Cincinnati itself. So, if you were interested in, you know, collecting trilobites or brachiopods, which are ancient clams, or you wanna get cephalopods, which are kind of proto squids, like an ancestor of nautilus and things like that, It's the place to go. That session is after this one. So, just stick around. Beverages will be served.

I love it. Credits will be awarded. I would love to introduce Sean next. So Sean is from San Diego State University and is the Senior Director, Instructional Technology Services. Correct? Yep.

Okay. And you're gonna see a theme a couple themes here in our middle group. But, Sean's a Colorado native, with a passion for traveling. Yep. And if I wrote it down correctly, you've been to a hundred plus countries and counting.

Indeed. Yep. With another trip upcoming here shortly. Yeah. So on Saturday, I take off for a trip to Mongolia.

So Wow. Had had to pack for two bags. Some of Las Vegas with one and Mongolia with the other. It was an interesting weekend last week. Next to Sean, we've got James.

So James with Cornell University, and he is the Senior Instructional Design Technologist in Center for Teaching Innovation. Yes. Okay. And now you see why I had to write these down because there was no way I was gonna remember some of that. I have to say that on stage this week and it's hard for me.

Okay. So, fun fact about James, an avid Charger fan. Matter of fact, you're gonna go visit the, training camp after the, conference here? I'm gonna go to San Diego, work for a couple weeks, and then spend a week of going to the Los Angeles Chargers training camp with my family, getting signatures, watching the players, and meeting them. And if I recall from conversation, you're also an avid traveler. You had to one up Sean and Jill because you've taken the kid, it seems like quite often.

So my kids were actually born in Korea, and then we traveled out from there before coming back to the States. Very nice. And last but not least, we have Jill. So Jill is from all of it. Yep.

You got it. Okay. Also, an avid traveler, I learned. Forty nine states, about fifty countries, and -More than I don't know. I never counted the countries.

-Five continents? -Yep. Goal is seven. I'll hit all seven before I I die. So -Okay. And loves to cook.

Yep. Loves to cook and bake, you know, so all of that. Games I love gaming as well, so all kinds of gaming. Perfect. Well, I would love if you guys would just give them a round of applause for joining us today.

Very, very great to have. So now onto the topic. Now that you know who we are, I had a question here and would love to just throw it out there. But so prior to Intelligent Insights, how were you in the school or institution really thinking about data and and using it to help drive, you know, data driven decisions, help drive teaching and learning outcomes. Kind of how are you really leveraging data within there? And Sean, maybe I know working with you guys, you've you've really embraced kind of data and data decision making with learning to learn.

Absolutely. So, so at San Diego State and really across the California State University system, I'd say there's a strong appreciation for data informed decisions. And at times that's been really challenging when that appreciation and value does not come along with the resources to aggregate the data in a way that we'd want. I know Sharon and others were talking about data lakes, and houses, and lake houses, and all kinds of things. Sounds nice, because we don't have that, especially when it comes to Canvas data.

So we actually transitioned to Canvas. We finalized I see some of my colleagues in the front row who were part of this, rather stressful time where we finalized our pilot of Canvas in spring twenty twenty, which also, as you may recall, was when COVID came upon us. So we were, just kind of pivoting and and selecting Canvas and moving to that in time for a lot of questions from our colleagues in academic affairs and student affairs and others saying, okay, how do we know if students aren't logging in? How do they know if they we even got their, the message to the right people that says, okay, classes are actually not gonna be in person this fall, etcetera. So, we were doing a lot of kind of hacking of our own data to, to get at those, and we worked with some consultants and kind of, you know, we trained ourselves and some of our colleagues on SQL and some other, tools to get to those those data, but it wasn't pretty. So we made it work.

We were really hacking it, which is you know, and we made it through the pandemic just like, you know, other institutions that had to kind of bootstrap their way through it. So had this tool and Gen AI been to a state of maturity back then, it would have looked a lot different. And, there's just a huge contrast between what we did then and and the last six months of what we've been able to do. That's great. I can add to that.

Yeah. Please. Absolutely. So we've been on Canvas, a very long time. Being a Utah institution, we're one of the original schools, since, Instructure is headquartered in Utah.

We've been we piloted Canvas, in when when they initially started Canvas in twenty ten, twenty eleven, and have been on Canvas ever since. And we have a data warehouse set up in house, on our Amazon warehouse servers and different things like that. And we have an institutional data team that does all sorts of stuff for institutional data. But that does and it does pull in Canvas data stuff and our institutional data team does do SQL data for their own institutional data needs. But as soon as something comes up, they query something, they ask us something but we have to scramble to pull something together, you know, instantly.

We have to just put together something in a quick Python script, basically. Go, we have to go grab the API table, throw together a quick, Python script and have something together in a few minutes, basically. And it's like, oh, okay. Here's the data tomorrow, basically. And that's what we do currently.

I I'm curious, you know, as as you guys have built teams and have looked at doing that, you know, how what have been some of the challenges, like, with the Canvas data itself? Like, is it is it easy to learn? Has it been simple to deal with? I would love just to I mean, I know the answer to it, of course, having dealt with it. But I would just love your guys' perspective around kind of ease of use and getting access and understanding some of that. I think I can take this one pretty easily. When we first started with Canvas ten years ago, I came from an institution that did a lot of work with data. It was a a private school and data was what they did.

When I ended up at Miami, that was not the case. Sometimes they don't wanna ask the question because they don't wanna know the answer. But they need to know the answer and especially in this day and time. So during the COVID, we had IT folks going in and pulling the data tables, and then sending out all of these reports across campus. And the data that they presented was wrong.

We could tell them it was wrong, but it didn't matter. Those of us working with Canvas knew what the data was, we just couldn't get to it. So when this intelligent insights came in, I was like jumping up and down going, now I can get to data. Now I can ask the questions like, are the courses ready? Are this faculty grading what they need to grade? What's the comparison between, you know, what they did in one term with a face to face course versus what they're doing in an online course. So that is just a valuable piece of information.

Whether the university uses it or not, it helps inform us to prepare faculty for fully online courses and to use the technology in a very productive way. And I definitely wanna I wanna come back to that, Cindy, because I think that's a really important part. But James, I wanted just to, I know with, Cornell you guys are really looking at, a data strategy and and using it for data decision making. Would love to learn a little bit how you guys are thinking about about that, and maybe what are some of the challenges that you're seeing as you embark down that journey? Yeah. So so far, we've we've been working more on the data we gather outside of Canvas, from our user interactions.

We have a tagging system that we've worked up and refined within our ticketing system to try to gain a better idea of trends in what types of issues are coming up for people. And we have a large resource library and we've got Google Google Analytics there, so we can kind of determine what people are looking at a lot, and that can help guide future document creation, as well as what kind of workshops and events we offer. And a real one of the strategic goals for us for our learning technologies team this year is actually to develop a better strategy and process for the evaluation of our data and how we make decisions off of it. And I think that the new data will be able to get through intelligent insights is really going to make a major impact on what that decision ends up being just from the the testing of, Ask Your Data so far, the way that we've gotten data into more hands on our team, There are two very technical people who can read Canvas data stuff. Talking to them sometimes is like me trying to read Canvas data myself.

So being able to tap in to ask your data and find things and sometimes at this point, like, they ask they'll ask me for something and I'm like, oh, I got that from ask your data. And they're like, you didn't have to run any code or anything? I'm like, no. I just I talk to it and I iterate and I I worked through and I got I got what I was looking for. And I think that's a great use case. And Sean, you guys were one of the first on kind of the Ask Your Data.

How has that changed as you kind of describe the the work that you and team have went through since twenty twenty and beyond? Like, as as the group now has been using intelligent insights and ask your data, like, how has it changed any of that process that that you guys have previously done? Yeah. I mean, a hundred percent it changed. So in a variety of ways, including our own team. And and we have some database analysts, you know, within the wider IT division, but but not really on our team. And again, we've kind of been able to hack our way through with consultants, things like that.

Now, you know, you mentioned iterative, and I think that's a really important piece of Gen AI in general, but especially with this tool, in that you can ask, you know, plain natural language questions like, what students within this subset of courses or student population are at risk? Period. Question mark. And, you know, in that case, I believe I recall the tool then in natural language responded and said, well, I took, you know, this subset and looked at seventy percent or less, and it, you know, there's nine hundred and seventy three students that match that criteria, but but then you can go in and say, actually let's go to sixty percent because seventy percent is still a passing grade, that kind of thing. So within our team it's really, you know, quickened the pace and and we've been able to again do iterations of analyses, but I think the bigger game changer is democratizing the data selectively. You know, we don't want every faculty member yet to have this tool for this, some of these tools for a variety of reasons.

But in terms of the administrators in those other verticals within the institution like academic affairs, student affairs, rather than kind of filtering through our, you know, chain of email or phone calls and saying, look at this. Okay. That's not quite what I want. Look at this. And as some I think you pointed out, you know, this is often the eleventh hour, and we need it right now.

And half our team's at InstructureCon in Vegas, and etcetera. So being able to democratize it for them, and they're really good at asking questions. They they know much more nuanced relationships between this subset of students and these subset sets of courses and this time in the semester, which is, you know, right around the the add drop date, that kind of thing. So they can ask the questions as many times as they want and get immediate access to the data and get those, interventions out to the students. So both of those ways it's changed a lot.

Jill, go ahead. So one of the the features is, product students in need of attention. And I, gave, our her title is lead scholarship coordinator. I had to write it down to make sure I didn't forget the the title. So what her job is, she's in the financial aid and scholarship, department.

I gave her access to this. What her job is is before students drop the final class for the semester, they have to talk to her. And I gave her access to students in need of attention. And, I gave her access to it and and she loved it because she was able to actually take a look at what they were doing in Canvas and reach out to them preemptively. And she was able to actually contact students and make sure that they were going into Canvas and where they were during the term and try to get contact with them because we all know that students who are struggling in Canvas are struggling academically, struggling financially, struggling socially, having other struggles that maybe she could help them with.

Maybe they're having trouble getting a job. Maybe they're having trouble, socially. Maybe they're having trouble getting food. She could help them get a scholarship, help them do something and help them get some other need that help them with, find tutoring, help them with some other resource to continue with their education and bring them back in and get them back on track. So it was a great resource and she loved it.

And so it's been wonderful with this tool for her. Great. So, Kit, I wanted to ask, you know, I think you had mentioned during one of our conversations that as you had started using Intelligent insights more, there were a couple, moments where it actually you pivoted on kind of a direction or, you know, an outcome based on the data that you were able to get to. Could you maybe just help help the group understand what that actual use case was? Absolutely. So I had the experience with this technology where as I was getting access to Intelligent Insights, I was using it as it was supposed to be used, but then I was finding ways of bending it to get at other information that I needed.

And I was in a meeting with the head of academic advising, and I was showing him the students in need of attention tool because that's not a tool I personally need, that's a tool his office needs. So, I'm walking him through it, and I'm showing it to him, and we decide we're gonna look at students that have zero grade just just to see if there's any. And then there were dozens of them. And so, it turns out students in need of attention actually doubles as faculty in need of attention as well. So we found ourselves in the middle of this meeting needing to pivot and go talk to deans and chairs, and we were reevaluating how we find out faculty aren't grading on time through student feedback.

And being able to see it in real time and to see that there was a course where something had not been graded and we were getting close to midterms. Students don't know where they are. We can't help them with academic advising. If the faculty isn't engaged, why should the students be engaged? So it became a moment where we could really kind of have a discussion about how we are going to change that behavior. And the fact that we have this tool that is agnostic, that can tell us what is happening and not happening, and it's not students pointing at faculty.

It is actual indisputable data. And it was just a really wonderful moment that something we weren't intending for it to do, it it did. And we were able to reach out and get the faculty back on track, and get the students back on track because of that. So it's a very important tool. I think following up on that, we'll coin the term like faculty in need of attention.

But I I know, Cindy, you had shared a use case of using course readiness for something very similar. And I I'm curious of, like, how you were applying course readiness for kind of a teach the teacher opportunity there. That is a valuable tool that I use all the time because if I don't know what the faculty are doing, for example, if they don't add an announcement or if they aren't participating in a discussion or if they're not grading in a timely fashion. What is it that they don't know that they should know? And that leads me to evaluate how we do our training, but also act as a resource for the departments that say, hey, here your faculty, Here how they are interacting with their students but it's not necessarily the best way. Perhaps we can help you help your faculty.

So we end up being a lot of not only train the trainer on how Canvas is used, but also how faculty are interacting with their students across the board. And so the intelligent insights really kind of helps us see if the faculty are ready. I love that. I wanna go back to a a a conversation as well, Sean. You had mentioned like democratizing access to data, which I I actually think is extremely powerful.

And I think, James, you were mentioning a little bit about that as well as kind of taking data out of the realm of, you know, a a business or a data scientist or a business analyst. Sean, I'm curious just expand a little bit on on kind of what you meant there and kind of the value that that's added for you in the school. Yeah. Yeah. There's so much data.

Right? We all know and we know how valuable the data is. Luckily, with these educational tools, the data, you know, we're not getting targeted advertising and things like that with some other tools. But nonetheless, we have this valuable data. And again, at SDSU, we haven't yet been able to to use utilize those as much as, as we should. Certainly for our, a lot of underrepresented students were Hispanic serving institution.

One of our campuses in Imperial Valley has ninety five percent, first generation students, ninety seven percent Hispanic students. So, these students many students are in need of attention and don't have those study skills, especially with, you know, going back to the pandemic, but certainly today in terms of having someone say, Hey, have you done your homework? It's Sunday evening, you know, are you ready for for the week? So, I think democratizing, definitely, like we talked about before, gives, you know, just a faster just in time access to the data rather than having to kind of go through the bureaucratic process of getting data. But I think in another way, it it democratizes access to Gen AI, which I think has some kind of secondary good effects. I was thrilled to see the Conmigo integration. We did a big survey at San Diego State of all of our students, and about twenty percent responded, which is great.

That's about seventy eight hundred students in October around the use of generative AI. And a little over eighty percent of the students, said they were currently using AI in their coursework, and they thought it would be a a an important part of most professions. This is across disciplines, whereas only one in five of their instructors were mentioning anything about AI pro or con in the syllabus. So I think, you know, there's been a bit of a bearing in the head in the sand saying, well, you know, I hope the students aren't using it, but I'm not gonna go there because I don't even know how to go there. We've since rolled out a big, campus wide and now system wide micro credential, but I think what these tools will do, certainly for the academic leaders and the student affairs leaders, is demonstrate some more value that they may not have seen yet for generative AI, and that coupled with things like, Convigo and other tools, I think will have at least some moments to say, Yeah, maybe I should say you can use AI here, not here because of X, Y, and Z, and and start to close that, equity gap a little bit as well.

Speaking of tools, and I know the the LTI usage was a little bit late in the game here. It was one of the pieces that, you know, we're bringing, as part of GA. But I know, James, you, you know, when you were able to see the walk through, really thought there was a lot of value added in the, LTI usage, reporting. I'd love to get your feedback on kind of where you see that, value applying, how how you and the team plan on using that, and, just thoughts in general around it. Yeah.

I I can't guarantee how it will be used because I get to make no decisions. But I think we're really interested in being able to see exactly how LTIs are being used out in, on our campus, in our community. We know of certain use cases for vocal people who come to us with questions or come to us with success stories, but we really don't know how deep it goes on how many folks are actually using different LTI's. And for most of these LTI's, we are paying per head for our FTE, total number of students at the school. But if only five percent of the students are actually utilizing the tool, you're either paying twenty times the amount you think per actual user or you need to start thinking of ways to get other users using it.

If if we know we have a valuable tool, we can look out there with this with the LTI usage tool, we can see by department, by sub account who is using a tool. If we see that there's a lot of focus in one area and we feel like there are other related sub accounts, but there's not usage there, we can start talking to the people in that one area about how they're using it and see if there are other use cases that might be able to use that and maybe they just socially aren't interacting with each other, so they don't know about this tool. Maybe they don't read our every semester we send out what's new in learning technologies. Maybe they don't read that as much. Maybe they're really focused on their research, so they tend not to see some of the learning technologies that are more embraced by those who are really focused on the pedagogical side.

And so whether it is determining that we can shrink costs on LTIs by showing that they're not adopted enough for us to be paying what we're paying for them or to make better decisions in how we choose what to focus on. I think it's really gonna help us get out there in the community and make new connections, and help people find new ways to utilize what we already have to benefit their learning outcomes. You know, I'll I'll steal a little bit from I don't know if they're in the team from, Florida State, College at Jacksonville. On the LTI usage, one of the one of the aspects of it is identifying champions, right, of that. And just with the sheer number of LTI applications, you're never gonna be the expert in each and every one of those.

So being able to identify who the champions are and bring those in, you know, when there's questions, when there's adoption, I you know, it was a use case again that I think we hadn't thought of, or thought thoroughly through. That was just a great case. Yeah. And just briefly, we we've talked about that as well, in terms of identifying champions, power users, and of course people that aren't using it yet. So targeting messaging, saying, hey, did you know we have a site license for this, come to this training.

But, one of the, just briefly, the things that has been really successful from our team is, having fellows from every college. So a faculty member who is, we're actually providing a course release, and they're doing advising and training, and that kind of thing. So this is a new way. Right now it's kind of a convenience sample, you know, who we, run into at Starbucks, that kind of thing the most. But this is a way to say, hey, this person's really kicking butt in this one area from the LTI analytics.

Let's talk to them about being a fellow next year. Yep. Can I can I add also? We're also we have over ten years of LTIs that we're really excited to clean up. And then also to clean up, like, the one point one integrations to bring up to one point three integrations. And then also, I'm really excited to find what colleges are using integrations where we can bring in site wide licenses or try to reduce costs down where individual colleges are each paying for where maybe a site wide license would bring down an institutional cost.

You know, one of the one of the other questions that I wanted to, kind of bring up and and it's been one of the goals of intelligent insight, but I'm curious how we've done. And that's around customization. Right? So, you know, we've tried not to create a just a dashboard or just a page that's the same for everybody, but something that really you can define your own institution kind of standards and criteria. So I'm curious if you guys how how easy have you found intelligent insights or how customizable have you found that to meet like your individual school needs? Really for me, it's it's been an eye opening experience. When I first started using the the product, it was, okay.

I'm gonna ask this question. What kind of response am I gonna get? Do I really believe what I got? So then I would spend hours looking looking and just verifying that what I got, and I would give feedback, which was why it was so great to be on this team that helps, you know, test it and kick it, and try to break it. And Kit and I are really good at breaking things. Yes, we know. It it we pride.

We're pride, proud, proudful about that. But one of the, the biggest things that I think I get out of it is that once you ask a question data from. I mean, very, very specifically, you know that, oh, it did this, it did this. And at first when I used it, I felt like I needed to know about tables, and titles, and fields. And over the course of testing it, working with Canvas, it was just a very progressive, very improved product that now I feel like I can go in, ask the question in a normal kind of voice, and and get what I want, or I can modify it right away and say, oh, I forgot about this.

I need to add this. And so it's an ongoing conversation that is a logical stream of consciousness for me. Yeah. With the tools, the criteria that you can actually put together in course readiness and students in need of attention is incredibly customizable. And particularly with course readiness, we have teaching essentials that we want to adhere to and some of those basics are really easy to put in as a criteria that we can look at.

But to be entirely honest, the Ask Your Data absolutely shines on this one. It keeps a record of your conversations. I have a separate pin board. I have a coordinator of instructional design who asks me all kinds of questions about Canvas, and I can actually have a personalized pin board for her. And then I have our data guy who asks all kinds of different other questions.

I have a pin board for him. And then we have again academic advising. I have a different pin board for them. So it allows me to actually create customizable, readily checkable data that I can go back to time and time again, and I can give input on student retention. I can give input on how course design is in fact, like, affecting student success.

So it it's highly customizable. I'm a huge fan of that. I wanna speak about, Kit mentioned the being able to customize the students in need of attention. And I'm actually really eager for, the future part of that where, as far as I understand, students in need of attention, they're looking to get it past the admin side to the faculty side of things, where faculty who know how they use Canvas, because I don't know about other institutions, but at Cornell, no two faculty use Canvas the same. But they can set what they see as succeeding or struggling in their class.

And then on the back end, the admins can see based on what they found to be struggling, trends in students that might be struggling across multiple courses. And I'm I'm really excited and cannot wait till we get our hands on that part of it. Coming soon. Is Gary Guy in here? Hello. If he is.

Okay. There he is. Okay. So I wanted I want to definitely open it up for audience questions. But before we do, like, what recommendations would you have for your peers or, other schools on, like, how do you get stakeholder buy in? So, you know, this is a new product.

It's a new offering. It's a for sale. So there is, you know, a little bit different than some of the other things that Sheeran talked about during the keynote. There is some convincing, there is some buy in that's needed. And how what what approach have you guys taken to go get that buy in and approval for intelligent insights? Yeah.

Joan, go ahead. So, so so we have, of course, like any university, there's politics, especially at a state school. We have a program called, we call Speedway, which is technically our Bachelor's of General Studies. And so it crosses multiple disciplines. And because of that, the program director can't have admin access across all of these programs.

And so she wants to know when the courses if the all the courses are published at the beginning of the semester. And so we couldn't use course readiness because it crosses all of these disciplines. It it just was too complicated to load in. So we ended up using Ask Your Data. It it just made it so much easier because of of being able to talk to the AI.

It ended up working so fantastically just going into the AI and just talking to it because all of the section numbers, start with the same section, number loaded into it, just ended up using this, wording, loading into the wording. I can't seem to talk anymore. It's it's almost five o'clock. Right? We're getting there. We're getting there.

Almost six o'clock. Happy hour. We're just getting in there. But anyway, talking to the AI ended up telling it this section number, across all these different disciplines, telling it just that the section numbers were the same. And in moments, we were able to know whether what was published and what wasn't published.

We were able to spit out this, CSV to end up giving this program director knowing whether it was published or not, and to hand to her, which was incredibly wonderful. I'm able to pin it and then we're able to refresh that each semester. And actually, it runs three times a semester we end up having to know this. So it's really easy for us to be able to pull that data, and that's really valuable for us. And and those are little things like that.

And it like like Kit was saying, it just gets easier and easier. When we started with Ask Your Data, we ended up needing to run reports like that. We ended up having a class. We run, a class, info ten ten and English ten ten twenty ten in concurrent, it's an English class and an information literacy class from the library. They run together.

And the only correlation, they they have the same section number. So o two and o two together. I needed to run a, survey in both of those classes. It was the very first thing we got as soon as, we got Ask Your Data. And two of us, started to do it, and we challenged each other to see who could get the AI to get this this report faster.

And we had it done within five minutes. Wow. So it was actually the one person had it done within five minutes. So but, yeah, this is so useful for us. And he just loves to play.

That person loves to play and ask your data. So this kind of stuff is so useful for our institution. So Sean, I'm curious, same question for you. Like like, what would be your recommendation to the group on on stakeholder buy in? Yeah. So I think I mean, one thing that comes to mind is the, just the sheer cost, delta of, you know, hiring multiple database analysts and having a data lake, which we need regardless, but, you know, specific to Canvas versus, you know, relatively lesser cost for this as a service, I think, and the fact that it's AI, you know, powered and I think more more powerful, but also tying it to specific, institutional priorities.

So one overarching one that we have at San Diego State is a prior core priority of students at our core. Right? So being able to say, hey, if we want to be able to give students the interventions that they need deserve, we need to put our money where our mouth is. Students at our core is a way to do that. And, but right now, in twenty twenty six, we're preparing for a, campus wide re accreditation, reaffirmation process. So as we gear up for that, and I'm on the committee for doing that, to be able to say, hey, if we need data that shows, you know, these department learning outcomes and these, you know, strategies that we've implemented since the last one in twenty, sixteen, right, ten years ago, we need to, you know, we can we can do it the hard way or we can do it the easy way with stuff like this.

So I think return on investment with, big strategic campus wide initiatives like reaccreditation. That's awesome. Thank you. I think we might have time for maybe one or two questions. I know we're butting right up against it, but we'd love to, just if there's any of that go here.

I will also say there is a code if anyone's interested in learning more about intelligent insights. Just please scan it, go. Go ahead. Yes. Hi.

Kit and Cynthia. I really enjoyed your example as an MBA student by night. There are few things I hate so much as not getting my grade on my mid semester papers until after the finals been turned in. And but also as an LMS, sometimes admin by day, I really worry that that sort of capability is going to create an antagonistic relationship with my faculty. So I'm curious to know how you deal with that in general, building on what James said, that no faculty member uses Canvas the same.

How we think about the incompleteness of this data and talk about it in a healthy way? Thank you. Oh, absolutely. When we kind of first discovered that the main conversation that happened was how do we get some kind of change on this without suddenly appearing like we are surveilling the faculty? Because that's not what we intended to do. It just kind of we were looking at the data and we saw it. So, the conversation that what really happened was about who this data belongs to, and my big thing was actually getting buy in from everybody.

I thought this was the coolest tool ever. I showed every chair, every dean, any person I could because so many things were being asked like, are these teaching essentials actually being followed? Are are and we can use course readiness to check that. Do we have students that are in need of attention that we are not catching in our other systems? And so how we've kind of faced it is by letting the chairs know that this exists, and we're letting them know that this isn't something we are looking at regularly, but this is something they should be looking at regularly. And so part of getting that buy in from them was saying, hey, this is a way for you to actually address evaluating your faculty at the end of the year. This is another tool in your pocket that you can use.

This isn't something I'm looking at. It's not something I'm giving to you. It's something that you can use yourself. And then it was the same thing with kind of that return on investment that now you, the chair, have access to this data. You get to justify why you should get funds to your department.

You get to actually pull up data and say, look at all the students I have enrolled in online classes. Don't even think about cutting one of these programs. Don't think about cutting that micro credential because I can actually prove I have students. I can prove they're doing well, and I can prove that they are staying in the online environment and they are staying within the university. So I found turning the tools over to the actual chairs and turning it over to the deans and telling them it was kind of theirs to do with as they will has really managed to get over that.

It was really just that first time that we had a discussion about there being any kind of animosity. And then the moment they realized that this was actually their tool, and I gave them examples about how I I rarely lose arguments at Miami because I have data. So if I come and I ask for something, I typically get it. And the minute I kind of phrased it that way to the chairs where I was like, this is your ticket to prove you need the things you are asking for. It clicked, And so the animosity definitely ebbed, but I can see that that is something we have to be sensitive to for sure.

And I think with that, we will wrap up our session. But, again, I wanted to thank each of you personally for participating, for being early advocates, and great partners with us. So round of applause, please, and thanks for everyone.