[University of Asia & the Pacific] Data, Insights, and the Instructure Learning Platform
To Canvas Connect. It's so lovely to have you all here. And as you could see, the room is quite open. We have about half the room filled. One of the greatest things that I love about face to face events is the opportunity to network, be together and closer with one another. So if I could just kindly request, do you guess at the back if you could just make your way forward, closer to the front, close the gaps.
That would be much appreciated. Alright. So today, we have a lineup of three tall for the leadership track here. So firstly, before we do begin, I'll introduce myself. My name is Ramsey Alamudy.
I'm the lead Canvas custom success manager for Asia Pacific. I work with a team who supports our customers across Australia, New Zealand, Singapore, other Southeast Asian countries, and the place where my heart and rest most in the Philippines. Over the past five years, has been an extraordinary experience here where back in two thousand and sixteen, we had approximately ten or so clients, in the Philippines. Now we move across five, six, seven years, and we're well above the sixty customer, Markier. And as you can see earlier today, the room was completely full.
Today, we have, a very good friend over the past six years working at Instructure and also a very deep customer. Why also call a friend Jason Devilla, who will be talking today, on the topic, data, insights, and the Instructure Learning platform. Firstly, as an introduction to Greg, Greg is the manager of solutions engineering. He's been with Instructure for the past six years. And prior to working at Instructure, He had spent several years working in the higher education industry across Australia.
Then within those six years, he's then worked across various institutions across Asia Pacific supporting our customers with their implementation and solutions with Canvas. He's an expert in live events, Canvas data too, and the various products we have. So he brings a wealth of knowledge to the stage, in this conversation. And then secondly, A person who I owe a lot of thanks and a lot of appreciation, particularly in supporting our Canvas community here in the Philippines, Jason Devilla, who is a managing director of Center of Teaching and Learning at the University of Asia and the Pacific. Previously a full time faculty member then moving into management and into administration over the past fifteen years.
So I'll pass it over to you, Greg. Thank you, Ramsey. And, thank you, Jason, for joining joining me on stage. Welcome. My name's Greg.
At the moment, I'm also accepting Ken or Ryan Gossling while I've got blond hair. The initial part of this presentation, we really just wanted to present a little picture of the the data option that are available within Canvas. So when I'm talking to customers, often we're talking about specific use cases, and maybe they're not seeing the full range of products that we have available, but also we've introduced some new products in the past few years. We've acquired some tools as well. And so today, we just wanted to run through why we have these options and where the overlap exists, what the different use cases are as well.
So roots already stolen some of the thunder around, some of the products here, but we'll go into some more detail. Thank you, Ruth. And Jason will then sort of, combine this with their use case and how they're currently using Canvas data two and impact to give that some more color. So we'll talk about Canvas data two, live events, and then impact as well. Just to set the stage, Canvas data two is really sort of our big data product that's often used in data warehousing.
It's a tool that your administrators can use just to really simply download a dataset such as some information about where all of your courses are residing within your Canvas instance. That might be really just to answer a simple question. How many of our courses are using, a feature such as Turnitin. We could just do a simple count in a spreadsheet But this could also be used within a broader strategy of, risk and retention. It may be that you're ingesting all of your data.
You're doing this regularly. Might do that several times a day, and that might be informing you about potential students at risk within your organization. Often this is, a specific team that will, work on modeling, and they have a particular skill set as well. So sometimes this is a very, keyless skill set that organizations might not have. We also have partners that are able to leverage this data and provide you similar, outputs as well.
If you've ever experienced Canvas data, this is a very broad data set across your entire platform. One of the challenges that we had with our previous version is that customers weren't getting timely data. It was taking up to thirty six to forty eight hours for that data to refresh. We're really pleased with Canvas data too that that's, sub four hours. So customers are getting really frequent up to date data from this system, and we're getting a really broad range of data as well.
So even more data than we're able to deliver previously. This is also available in a range of modern formats, the one that parquet, that's actually part of a a data modeling, projects run by the Apache organization, really, really flexible, really robust as well. Ruth also mentioned that this is currently providing data around Canvas. And so Canvas data too, that name is probably a little limiting because we are going to introduce additional products in there. So catalog is, coming very, very shortly.
Studio will also be included in that, in the coming months as well. The future is though that this platform will be your source of all data within the in learning platform. So a really flexible solution. Also within this model, we've introduced the ability for you to generate partner tokens, meaning that it's more cure more flexible to provide this data through to instructor partners, who might be providing you analytic services, who might be helping you with retention modeling, etcetera. We're really pleased to be able to deliver that through to customers.
We're getting pretty good feedback, across the board as well. This is a very different solution to the column that we have here, live events. And live events is used for a range of different purposes. It's a, a solution that you're able to receive instant jing from Canvas based on triggers that occur within the platform. So what that might mean is when a user submits an assignment, your definitely notified that that assignment's been submitted, and you might then want to process something, based on that action.
Maybe you want to analyze was that submitted late do wanna trigger an email, a range of different reasons why that might be used. Some specific use cases though, where partners are using live events is, atomic search. They're one of our partners that introduce a global search across your Canvas account. Atomic search will receive a live event and that would tell them that something's changed within the platform. They can go in there and reindex and make sure that they've got the freshest data available to them.
And we'll talk about some more use cases soon. That is a limited dataset, and it is quite a technical, tool to use as well. Whereas that very far right column, this is one of our products that's, really akin to the way that Google Analytics works So if you've ever used Google Analytics, analytics is able to track what we're doing within a website. And so impact is able to look at what a user's doing on a specific page rather than just getting data that a user went into SpeedGrader, impacts able to inform us what they clicked on within SpeedGrader. Did they open the rubric did they provide audio and video feedback through to the students exactly what were they doing within the solution? So really, really granular look at what's occurring in your platform.
Also, this is the one that we're able to, view in app reports within. So we're able to deliver you out of the box reports as to why going on in your platform. That might be answering what features are being used in the platform. It might be telling us specifically which users aren't leveraging a specific, feature set maybe that's outcomes. And then that's able to drill down at a granular level subaccount by user, a very, very flexible way of analyzing your data.
Out of the box, there's about six hundred different things that impacts able to monitor for your organization. As I mentioned, it's very granular. Jason will give us some real, world use cases soon as well. And this tool is also able to monitor which external tools are being used in your platform as well. So rather than just getting feedback on what's happening in Canvas, this is able to expose what third party apps are being used as well.
So hopefully, you can see there just by those three columns, there's a very broad range of data available. Where might EU use some of this? So Canvas data two, the use case that most common is this is around business intelligence. This is building up reports that might be sent to your executive team. It might be that you're embedding this data back into Canvas. But often what's occurring with this in a, data warehousing situation is they were aggregating that with other data sources.
So you're able to get that complete picture of your students across the organization, combining that with other data sources like your student information system. This then allows you to have that complete view across the organization over time. You can keep, keeping your Canvas data up to date and having that complete life cycle or life line, the complete view of your Canvas environment is what I'm trying to say. So live events often this is used for external tools. This is something that you might not even know is being used in your Canvas environment.
These live events might be triggering off to a third party that you're using currently without you knowing. This is just one of those tools that really does help keep our system efficient and provide you with timely information. Impact, on the other hand, this is very useful for monitoring utilization within Canvas We can also do things like set, campaigns within impact where we're showing messages, support articles, and targeting specific users and then monitoring the success of our campaign. We can set targets for adoption within this tool as well. And also we can monitor how different support channels are being leveraged and get feedback from users on what support we useful to them, what might need adjusting as well.
So those are the three tools that we wanna talk about. Obviously, there are other sources of data within Canvas such as our API that we'd be using every day, maybe not, without even realizing it. Canvas data too often does require a specific, team to make use of that data, but we've provided you some really useful tools as well that can help you get start to get this data into a database, that data warehouse in your organization, and we can make that an efficient process for you to continue ingesting that data as well. Jason has been using Canvas data to an impact within the University of Asia and the Pacific. And so my first question to you, Jason, is how are you able to leverage both Canvas data and impacts and how are they used in, unison at your organization Okay.
I guess, there's a little backstory to why we got Canvas and then impact. When we were thinking of another s, and we had trimmed the shortlist down to two candidates in late twenty seventeen. We tested the usability across faculty, and the other LMS, not Canvas, actually scored higher on the faculty preference score. But when we took a step back and looked at other features, management decided that there was a larger strategic value to the ability to export raw data from Canvas because the other LMS needed a separate app and a separate subscription, to get that data. So in our case, We wanted to really move from anecdotes to data.
When you're typically asked, how good is your program? People will say, oh, our graduates are hard by in such a company and to go to this and that industry. But, real answers would be how many of them are hired within the first six months How fast do they, how do they compare in their performance against, graduates from other institutions, how fast it to get promoted and other measures like that. So we really wanted to figure out what was going on inside the university, and we knew that the value of data for us what's going to be, figuring out what is really going on so we could tweak what's happening and improve the way we deliver our education. And then also we had a lot of information, but they were scattered across different, domains. And so we thought that we needed insights into how to the quality of teaching and then also the quality of learning in the institution.
So that was our great hope in, getting canvas because we knew we could count the canvas data. But as Greg said, the first version of canvas data was a little bit unwieldy to work with. We wanted to build custom dashboards, but it's been a long journey because you understand what's required, the data warehousing, the strategic alignment, it's quite a bit of work. But I think it's still worth thinking about your LMS and your whole academic operations from the point of view of what you can harvest from the day that goes into it because it's really it can give you a lot of insights. In our case, we harvested back from the last three years, and we kind of identified teaching behavior that students said contributed a lot or significant lead to an improved learning experience.
So, for example, consistent communication, regular announcement announcements given on time. And so inside impact, or in Canvas, we pack it to the regularity by which teachers use announcements, for example, updated performance, meaning students know how they're doing in the course, any point in time, that means upgraded scores, upgraded assessment task, results. And we pegged that to the use of assignment because that tells us that a teacher has thought through the whole syllabus. He's, he or she has listed down all the requirements has given them the proper weights and updates them regularly. Gradebook access also is another indicator for us of how regularly a teacher, updates students regarding their performance.
I think all of us probably deal with some teachers who check everything at the end of the sem. So the whole term students have no idea how they're doing, and this is one way by which we figure out what's going on. And then another, aspect of teaching that's really important is for them to receive feedback. And as Greg said, we can check, which of the teachers use SpeedGrader because for us, that's one of the best ways by which they deliver feedback, not the scores for assessment tasks, but also comments. So we look at it there.
And so by mapping, these important and behaviors to the ones that are that have a significant impact on the learning experience, we go into impact, and then we to figure out what's going on. And then that tells us what we can do at the program level or departmental level in terms of interventions. When we were first approached by impact, which was still no easy soft then. We thought our first impression was that it's just a messaging platform. And why do you need to subscribe to assess service just to send messages to, very specific subsets of your users.
You can do that in Canvas with announcements, broken down to subaccounts But what we've discovered is that, impact gives you a lot more flexibility. For example, custom user groups, rather than filter out all the teachers and figuring out who was hired in the last, three months, we can create a custom user group for all the new teachers for the current school year. That means we can create a campaign with very specific content for what they need, as they're onboarded for the school year. So we don't need to send all the teachers, for example, reminders on how to use a gradebook, but new teachers will do will will need that. And so we can focus the messages to them.
The other thing is that you can dive into Kanba, into impact and, figure out in real time very specific behavior. So like as I said, if you're an institution that requires teachers to report the midterm standing of students. We can look into, impact and check how many features have not accessed the Gradebook, which means they haven't updated scores, over the past two weeks or three weeks. And then can send the messages to them and remind them what they need to do. So that's how, impact has helped us in monitoring operations, and then pushing buttons where we need to push them when we realize that there's a certain gap in terms of operations.
I'll just show you one example of how we use, in back data in for making pedagogical decisions. So this is a graph that shows, grade book access, for two departments within our institution. So, here you'll see that, Gradebook usage is roughly just under fifty percent, and the total number of teachers are using it all throughout the sem. And then it spikes towards the end of the term as everybody rushes to complete their grades. Right? So and then this so when we look at, adoption rate, comparing two, these are figures from two different departments.
So one, has an adoption rate of seventy one point forty three percent with, a usage rate of about fifty percent from SpeedGrader. That tells us that three out of every ten teachers are not updating, their grade books, which tells us there's a problem in terms of providing students, updated statuses, which allows us then to send prompts or messages or to go through, to go to certain managers to to encourage them to take action. Sum. In the other department, you can see that their adoption rate is ninety two point six eight percent. They're one of our, our best use servicing, for Canvas.
And, speed grader rate is, about sixty one percent, which tells us there's a healthy rate of them using speed grader and giving feedback to the students. So in this case, we focus our efforts on our remedial efforts or button pushing to the unit that has, low utilization rates. So that's how, impact data and the look into. And this is just one of it can be a bit overwhelming when you first start looking into the number of tools and, that you can check inside Canvas. But this is just an example of how we we use that data to figure out how what's going on and then to figure out also if there are certain gaps in behavior that we need to address and then to send the proper prompts or messages to department shares or program department or particular teachers, when they need when they need to step up, for the sake of the students.
So that's an overview egreg of of how we've used impact and a little bit of conversation to him. Thank you. So it seems like we could definitely get that data out of Canvas data, but the benefit here is that it's all in the application. You can just generate those reports can then find the users who are and aren't using those features, contact them accordingly. Yep.
Yeah. Exactly. So the combination of being able to look at out of the box reports and then, tailoring the messages to specific sets of users is for us what makes it really, really useful in terms of figuring out what's going on and then taking the first steps to remediate, gaps that we see. So having used impact for quite a while now, if someone was be implementing impact at the moment. What advice could you provide them? Okay.
As I said, we're still in the journey of, getting from raw data available in Canvas data one and now Canvas data two to custom dashboards. It's not an easy journey, but I think the first steps you guys we'll have to think of is, make sure you have a data strategy in place. You know what's required to do it when he Greg mentioned data warehousing earlier, so you have be in close. You have to work closely with your ICT or IT department. You'll also have to work closely with, higher management because you're going to need align, your strategic direction, and then together identify which performance indicators you want to look at, and then you have to decide on what metrics you're going to really, generate.
And then you're going to that to the data points inside canvas data or impact that you want to look at. In our case, I gave you an example of we want to figure out how consistent the teachers communicate with their students, and the data point for us is use of announcements. Of course, that's not the entire picture. We merge quantit we use quantitative and qualitative assessments to get a better feel for what's going on. But it's a very good starting point for you to figure out what's going on in your sitution.
So that's is there something else? Yeah. So these are samples of it this is always in it's evolving for us we're trying to figure out what the real, metrics and data points should be. But right now, this is what we're looking at and this is what we're using to to get a feel for what's going on in our institution. Thank you. And I think, you know, you've mentioned their strategy.
I was wondering, do have like a roadmap for what you're planning on doing at two levels. One, an operational level to, sort of improve the the utilization of the LMS. And are you aligning anything that you're strategically trying to do as an organization through to impact? Okay. In terms of general strategy, I think philippine educational institutions. Although we really try to aim for being good at teaching research and extension, that's what ought to support all of us.
I think a large part of the job is teaching and teaching well. And so that's one of the key directions we we are looking at mean, we really want to raise the quality of teaching everywhere, but of course, we start in our own backyard, and we're looking at that as one of the big, directions we want to go into. As I said, if you ask people in our institution now how things are going, you mostly get anecdotes. We want to be able to provide data and figure out really what's going on so that we know where we're starting and where we want to go. So that's like a big picture answer to your question.
In terms of operations, as you saw, we can dive into, specific teaching behaviors that are Not really mandated, but recognized as something that all teachers should be doing regardless of discipline, regardless of tenure, because it they all contribute to an improved learning experience and the attainment of learning outcomes. And so we can we actually generate reports per term to figure out how things are going on, and then we pass on the results to the department shares. I let me share with you a little bit of, where we want to take our institution. So we realize that typical teacher will have a class where he or she will have some superstars who are super engaged and some laggards who are, developing a bit more slowly, and the normal reaction is to is to not pay attention to those who are struggling, but All the research says that if you if you doubled up on, engagement and attention and interventions, they will also make one or two standard deviations, improvements in terms of performance. And so we want to get there.
We also wanted to shift teaching practice in the direction of mass mastery learning paths because we want to move from simple assessments and tests to, better aligned outcomes and assessment tasks. That's going to take several years because changing teaching culture is a big it's a big job. It's a long it takes a long time. And, also, we want to empower academic managers to the way that we're looking at now and using it, we want to decentralize that function to department shares and program heads so that they can take action as needed. They don't need to wait for us to compile the reports to make the recommendations before something happens.
So these are some of the, institutional goals we have. And, for these, we recognize the value of Canvas data and impact as a platform. Thank you for your insight. Ramsey, we've a couple minutes for Q and A. Yep.
So thank you, Jason, again, for that insight. Are there any questions that people have in the room for Jason? F I questions. You did a great job. And Thank you. Thank you, everyone.
I don't know if you're all asleep or you all understood something, but thank you for listening. Thank you again to Greg and Jason from UNA p. Raise your hand if you found some of the information or the challenges that Jason faced, similar to your circumstances at your college. Yep. See some hands raised. Yeah. And one of the key things that we really wanna do throughout these sessions is ensuring that you can see the practical nature of the products that we share with you, such as impact, canvas, and credentials, and actually see the life cycle of how how these have been implemented, and the value these are giving you'll, pees at various other institutions.
That would be much appreciated. Alright. So today, we have a lineup of three tall for the leadership track here. So firstly, before we do begin, I'll introduce myself. My name is Ramsey Alamudy.
I'm the lead Canvas custom success manager for Asia Pacific. I work with a team who supports our customers across Australia, New Zealand, Singapore, other Southeast Asian countries, and the place where my heart and rest most in the Philippines. Over the past five years, has been an extraordinary experience here where back in two thousand and sixteen, we had approximately ten or so clients, in the Philippines. Now we move across five, six, seven years, and we're well above the sixty customer, Markier. And as you can see earlier today, the room was completely full.
Today, we have, a very good friend over the past six years working at Instructure and also a very deep customer. Why also call a friend Jason Devilla, who will be talking today, on the topic, data, insights, and the Instructure Learning platform. Firstly, as an introduction to Greg, Greg is the manager of solutions engineering. He's been with Instructure for the past six years. And prior to working at Instructure, He had spent several years working in the higher education industry across Australia.
Then within those six years, he's then worked across various institutions across Asia Pacific supporting our customers with their implementation and solutions with Canvas. He's an expert in live events, Canvas data too, and the various products we have. So he brings a wealth of knowledge to the stage, in this conversation. And then secondly, A person who I owe a lot of thanks and a lot of appreciation, particularly in supporting our Canvas community here in the Philippines, Jason Devilla, who is a managing director of Center of Teaching and Learning at the University of Asia and the Pacific. Previously a full time faculty member then moving into management and into administration over the past fifteen years.
So I'll pass it over to you, Greg. Thank you, Ramsey. And, thank you, Jason, for joining joining me on stage. Welcome. My name's Greg.
At the moment, I'm also accepting Ken or Ryan Gossling while I've got blond hair. The initial part of this presentation, we really just wanted to present a little picture of the the data option that are available within Canvas. So when I'm talking to customers, often we're talking about specific use cases, and maybe they're not seeing the full range of products that we have available, but also we've introduced some new products in the past few years. We've acquired some tools as well. And so today, we just wanted to run through why we have these options and where the overlap exists, what the different use cases are as well.
So roots already stolen some of the thunder around, some of the products here, but we'll go into some more detail. Thank you, Ruth. And Jason will then sort of, combine this with their use case and how they're currently using Canvas data two and impact to give that some more color. So we'll talk about Canvas data two, live events, and then impact as well. Just to set the stage, Canvas data two is really sort of our big data product that's often used in data warehousing.
It's a tool that your administrators can use just to really simply download a dataset such as some information about where all of your courses are residing within your Canvas instance. That might be really just to answer a simple question. How many of our courses are using, a feature such as Turnitin. We could just do a simple count in a spreadsheet But this could also be used within a broader strategy of, risk and retention. It may be that you're ingesting all of your data.
You're doing this regularly. Might do that several times a day, and that might be informing you about potential students at risk within your organization. Often this is, a specific team that will, work on modeling, and they have a particular skill set as well. So sometimes this is a very, keyless skill set that organizations might not have. We also have partners that are able to leverage this data and provide you similar, outputs as well.
If you've ever experienced Canvas data, this is a very broad data set across your entire platform. One of the challenges that we had with our previous version is that customers weren't getting timely data. It was taking up to thirty six to forty eight hours for that data to refresh. We're really pleased with Canvas data too that that's, sub four hours. So customers are getting really frequent up to date data from this system, and we're getting a really broad range of data as well.
So even more data than we're able to deliver previously. This is also available in a range of modern formats, the one that parquet, that's actually part of a a data modeling, projects run by the Apache organization, really, really flexible, really robust as well. Ruth also mentioned that this is currently providing data around Canvas. And so Canvas data too, that name is probably a little limiting because we are going to introduce additional products in there. So catalog is, coming very, very shortly.
Studio will also be included in that, in the coming months as well. The future is though that this platform will be your source of all data within the in learning platform. So a really flexible solution. Also within this model, we've introduced the ability for you to generate partner tokens, meaning that it's more cure more flexible to provide this data through to instructor partners, who might be providing you analytic services, who might be helping you with retention modeling, etcetera. We're really pleased to be able to deliver that through to customers.
We're getting pretty good feedback, across the board as well. This is a very different solution to the column that we have here, live events. And live events is used for a range of different purposes. It's a, a solution that you're able to receive instant jing from Canvas based on triggers that occur within the platform. So what that might mean is when a user submits an assignment, your definitely notified that that assignment's been submitted, and you might then want to process something, based on that action.
Maybe you want to analyze was that submitted late do wanna trigger an email, a range of different reasons why that might be used. Some specific use cases though, where partners are using live events is, atomic search. They're one of our partners that introduce a global search across your Canvas account. Atomic search will receive a live event and that would tell them that something's changed within the platform. They can go in there and reindex and make sure that they've got the freshest data available to them.
And we'll talk about some more use cases soon. That is a limited dataset, and it is quite a technical, tool to use as well. Whereas that very far right column, this is one of our products that's, really akin to the way that Google Analytics works So if you've ever used Google Analytics, analytics is able to track what we're doing within a website. And so impact is able to look at what a user's doing on a specific page rather than just getting data that a user went into SpeedGrader, impacts able to inform us what they clicked on within SpeedGrader. Did they open the rubric did they provide audio and video feedback through to the students exactly what were they doing within the solution? So really, really granular look at what's occurring in your platform.
Also, this is the one that we're able to, view in app reports within. So we're able to deliver you out of the box reports as to why going on in your platform. That might be answering what features are being used in the platform. It might be telling us specifically which users aren't leveraging a specific, feature set maybe that's outcomes. And then that's able to drill down at a granular level subaccount by user, a very, very flexible way of analyzing your data.
Out of the box, there's about six hundred different things that impacts able to monitor for your organization. As I mentioned, it's very granular. Jason will give us some real, world use cases soon as well. And this tool is also able to monitor which external tools are being used in your platform as well. So rather than just getting feedback on what's happening in Canvas, this is able to expose what third party apps are being used as well.
So hopefully, you can see there just by those three columns, there's a very broad range of data available. Where might EU use some of this? So Canvas data two, the use case that most common is this is around business intelligence. This is building up reports that might be sent to your executive team. It might be that you're embedding this data back into Canvas. But often what's occurring with this in a, data warehousing situation is they were aggregating that with other data sources.
So you're able to get that complete picture of your students across the organization, combining that with other data sources like your student information system. This then allows you to have that complete view across the organization over time. You can keep, keeping your Canvas data up to date and having that complete life cycle or life line, the complete view of your Canvas environment is what I'm trying to say. So live events often this is used for external tools. This is something that you might not even know is being used in your Canvas environment.
These live events might be triggering off to a third party that you're using currently without you knowing. This is just one of those tools that really does help keep our system efficient and provide you with timely information. Impact, on the other hand, this is very useful for monitoring utilization within Canvas We can also do things like set, campaigns within impact where we're showing messages, support articles, and targeting specific users and then monitoring the success of our campaign. We can set targets for adoption within this tool as well. And also we can monitor how different support channels are being leveraged and get feedback from users on what support we useful to them, what might need adjusting as well.
So those are the three tools that we wanna talk about. Obviously, there are other sources of data within Canvas such as our API that we'd be using every day, maybe not, without even realizing it. Canvas data too often does require a specific, team to make use of that data, but we've provided you some really useful tools as well that can help you get start to get this data into a database, that data warehouse in your organization, and we can make that an efficient process for you to continue ingesting that data as well. Jason has been using Canvas data to an impact within the University of Asia and the Pacific. And so my first question to you, Jason, is how are you able to leverage both Canvas data and impacts and how are they used in, unison at your organization Okay.
I guess, there's a little backstory to why we got Canvas and then impact. When we were thinking of another s, and we had trimmed the shortlist down to two candidates in late twenty seventeen. We tested the usability across faculty, and the other LMS, not Canvas, actually scored higher on the faculty preference score. But when we took a step back and looked at other features, management decided that there was a larger strategic value to the ability to export raw data from Canvas because the other LMS needed a separate app and a separate subscription, to get that data. So in our case, We wanted to really move from anecdotes to data.
When you're typically asked, how good is your program? People will say, oh, our graduates are hard by in such a company and to go to this and that industry. But, real answers would be how many of them are hired within the first six months How fast do they, how do they compare in their performance against, graduates from other institutions, how fast it to get promoted and other measures like that. So we really wanted to figure out what was going on inside the university, and we knew that the value of data for us what's going to be, figuring out what is really going on so we could tweak what's happening and improve the way we deliver our education. And then also we had a lot of information, but they were scattered across different, domains. And so we thought that we needed insights into how to the quality of teaching and then also the quality of learning in the institution.
So that was our great hope in, getting canvas because we knew we could count the canvas data. But as Greg said, the first version of canvas data was a little bit unwieldy to work with. We wanted to build custom dashboards, but it's been a long journey because you understand what's required, the data warehousing, the strategic alignment, it's quite a bit of work. But I think it's still worth thinking about your LMS and your whole academic operations from the point of view of what you can harvest from the day that goes into it because it's really it can give you a lot of insights. In our case, we harvested back from the last three years, and we kind of identified teaching behavior that students said contributed a lot or significant lead to an improved learning experience.
So, for example, consistent communication, regular announcement announcements given on time. And so inside impact, or in Canvas, we pack it to the regularity by which teachers use announcements, for example, updated performance, meaning students know how they're doing in the course, any point in time, that means upgraded scores, upgraded assessment task, results. And we pegged that to the use of assignment because that tells us that a teacher has thought through the whole syllabus. He's, he or she has listed down all the requirements has given them the proper weights and updates them regularly. Gradebook access also is another indicator for us of how regularly a teacher, updates students regarding their performance.
I think all of us probably deal with some teachers who check everything at the end of the sem. So the whole term students have no idea how they're doing, and this is one way by which we figure out what's going on. And then another, aspect of teaching that's really important is for them to receive feedback. And as Greg said, we can check, which of the teachers use SpeedGrader because for us, that's one of the best ways by which they deliver feedback, not the scores for assessment tasks, but also comments. So we look at it there.
And so by mapping, these important and behaviors to the ones that are that have a significant impact on the learning experience, we go into impact, and then we to figure out what's going on. And then that tells us what we can do at the program level or departmental level in terms of interventions. When we were first approached by impact, which was still no easy soft then. We thought our first impression was that it's just a messaging platform. And why do you need to subscribe to assess service just to send messages to, very specific subsets of your users.
You can do that in Canvas with announcements, broken down to subaccounts But what we've discovered is that, impact gives you a lot more flexibility. For example, custom user groups, rather than filter out all the teachers and figuring out who was hired in the last, three months, we can create a custom user group for all the new teachers for the current school year. That means we can create a campaign with very specific content for what they need, as they're onboarded for the school year. So we don't need to send all the teachers, for example, reminders on how to use a gradebook, but new teachers will do will will need that. And so we can focus the messages to them.
The other thing is that you can dive into Kanba, into impact and, figure out in real time very specific behavior. So like as I said, if you're an institution that requires teachers to report the midterm standing of students. We can look into, impact and check how many features have not accessed the Gradebook, which means they haven't updated scores, over the past two weeks or three weeks. And then can send the messages to them and remind them what they need to do. So that's how, impact has helped us in monitoring operations, and then pushing buttons where we need to push them when we realize that there's a certain gap in terms of operations.
I'll just show you one example of how we use, in back data in for making pedagogical decisions. So this is a graph that shows, grade book access, for two departments within our institution. So, here you'll see that, Gradebook usage is roughly just under fifty percent, and the total number of teachers are using it all throughout the sem. And then it spikes towards the end of the term as everybody rushes to complete their grades. Right? So and then this so when we look at, adoption rate, comparing two, these are figures from two different departments.
So one, has an adoption rate of seventy one point forty three percent with, a usage rate of about fifty percent from SpeedGrader. That tells us that three out of every ten teachers are not updating, their grade books, which tells us there's a problem in terms of providing students, updated statuses, which allows us then to send prompts or messages or to go through, to go to certain managers to to encourage them to take action. Sum. In the other department, you can see that their adoption rate is ninety two point six eight percent. They're one of our, our best use servicing, for Canvas.
And, speed grader rate is, about sixty one percent, which tells us there's a healthy rate of them using speed grader and giving feedback to the students. So in this case, we focus our efforts on our remedial efforts or button pushing to the unit that has, low utilization rates. So that's how, impact data and the look into. And this is just one of it can be a bit overwhelming when you first start looking into the number of tools and, that you can check inside Canvas. But this is just an example of how we we use that data to figure out how what's going on and then to figure out also if there are certain gaps in behavior that we need to address and then to send the proper prompts or messages to department shares or program department or particular teachers, when they need when they need to step up, for the sake of the students.
So that's an overview egreg of of how we've used impact and a little bit of conversation to him. Thank you. So it seems like we could definitely get that data out of Canvas data, but the benefit here is that it's all in the application. You can just generate those reports can then find the users who are and aren't using those features, contact them accordingly. Yep.
Yeah. Exactly. So the combination of being able to look at out of the box reports and then, tailoring the messages to specific sets of users is for us what makes it really, really useful in terms of figuring out what's going on and then taking the first steps to remediate, gaps that we see. So having used impact for quite a while now, if someone was be implementing impact at the moment. What advice could you provide them? Okay.
As I said, we're still in the journey of, getting from raw data available in Canvas data one and now Canvas data two to custom dashboards. It's not an easy journey, but I think the first steps you guys we'll have to think of is, make sure you have a data strategy in place. You know what's required to do it when he Greg mentioned data warehousing earlier, so you have be in close. You have to work closely with your ICT or IT department. You'll also have to work closely with, higher management because you're going to need align, your strategic direction, and then together identify which performance indicators you want to look at, and then you have to decide on what metrics you're going to really, generate.
And then you're going to that to the data points inside canvas data or impact that you want to look at. In our case, I gave you an example of we want to figure out how consistent the teachers communicate with their students, and the data point for us is use of announcements. Of course, that's not the entire picture. We merge quantit we use quantitative and qualitative assessments to get a better feel for what's going on. But it's a very good starting point for you to figure out what's going on in your sitution.
So that's is there something else? Yeah. So these are samples of it this is always in it's evolving for us we're trying to figure out what the real, metrics and data points should be. But right now, this is what we're looking at and this is what we're using to to get a feel for what's going on in our institution. Thank you. And I think, you know, you've mentioned their strategy.
I was wondering, do have like a roadmap for what you're planning on doing at two levels. One, an operational level to, sort of improve the the utilization of the LMS. And are you aligning anything that you're strategically trying to do as an organization through to impact? Okay. In terms of general strategy, I think philippine educational institutions. Although we really try to aim for being good at teaching research and extension, that's what ought to support all of us.
I think a large part of the job is teaching and teaching well. And so that's one of the key directions we we are looking at mean, we really want to raise the quality of teaching everywhere, but of course, we start in our own backyard, and we're looking at that as one of the big, directions we want to go into. As I said, if you ask people in our institution now how things are going, you mostly get anecdotes. We want to be able to provide data and figure out really what's going on so that we know where we're starting and where we want to go. So that's like a big picture answer to your question.
In terms of operations, as you saw, we can dive into, specific teaching behaviors that are Not really mandated, but recognized as something that all teachers should be doing regardless of discipline, regardless of tenure, because it they all contribute to an improved learning experience and the attainment of learning outcomes. And so we can we actually generate reports per term to figure out how things are going on, and then we pass on the results to the department shares. I let me share with you a little bit of, where we want to take our institution. So we realize that typical teacher will have a class where he or she will have some superstars who are super engaged and some laggards who are, developing a bit more slowly, and the normal reaction is to is to not pay attention to those who are struggling, but All the research says that if you if you doubled up on, engagement and attention and interventions, they will also make one or two standard deviations, improvements in terms of performance. And so we want to get there.
We also wanted to shift teaching practice in the direction of mass mastery learning paths because we want to move from simple assessments and tests to, better aligned outcomes and assessment tasks. That's going to take several years because changing teaching culture is a big it's a big job. It's a long it takes a long time. And, also, we want to empower academic managers to the way that we're looking at now and using it, we want to decentralize that function to department shares and program heads so that they can take action as needed. They don't need to wait for us to compile the reports to make the recommendations before something happens.
So these are some of the, institutional goals we have. And, for these, we recognize the value of Canvas data and impact as a platform. Thank you for your insight. Ramsey, we've a couple minutes for Q and A. Yep.
So thank you, Jason, again, for that insight. Are there any questions that people have in the room for Jason? F I questions. You did a great job. And Thank you. Thank you, everyone.
I don't know if you're all asleep or you all understood something, but thank you for listening. Thank you again to Greg and Jason from UNA p. Raise your hand if you found some of the information or the challenges that Jason faced, similar to your circumstances at your college. Yep. See some hands raised. Yeah. And one of the key things that we really wanna do throughout these sessions is ensuring that you can see the practical nature of the products that we share with you, such as impact, canvas, and credentials, and actually see the life cycle of how how these have been implemented, and the value these are giving you'll, pees at various other institutions.