Competency-based Healthcare Education with the Help of Canvas LMS and Canvas Data

Welcome everybody to our session today. It's called competent based health care education with the help of Canvas LMS and data. And my name is Jennifer Love. I am a business systems analyst with information and technology services, the teaching and learning group at the University of Michigan, and I'll let Pushimi introduce herself. We did turn you on. Hello.

Welcome everybody. I am really happy to see you all. I my name is Pushimi Gondala. I am an application developer, I at ITS Station and Learning, at university of Michigan working with Jennifer on the same team. Alright.

So we are gonna talk to you about our project today, which is centered around interprofessional education. How many of her of you have heard of IPE? Excellent. We've got a few hands in the rooms. That's great. So for those of folks who are not as familiar with IPE, we're gonna start with kinda grounding you in what is interprofessional education that you can get a chance to understand the context within which we were working on this project.

So interprofessional education at the University of Michigan has been going for a number of years, the center for IPE was created back in twenty fourteen when a number of deans from the different health schools and health professions both on our Ann Arbor campus, as well as on our Flint and Dearborn campuses, came together in order to change the way health care is done. In order to improve how students are learning, what they're learning, so that they can improve health care as they graduate. So at its simplest, What interprofessional cate education is is when students from the different health professions are learning with each other, They're learning about each other's professions. So they're doing this in tandem, which is a new and different way to do health education because for the longest time, it's been very unidisciplinary. So the nurses went to nursing school and laid their nursing.

The social workers went to social work school and they learned social same pharmacist and dentists and medical students. And so what IPE is doing is changing that paradigm. From unidisciplinary, so single siloed disciplines into disciplines where students are learning with, from and about each other's discipline in order to improve health care. So really, that's the goal. So why are we doing interprofessional education? Well, the goal of it is going to be, improving the quadrant full aim of care, as it's called in healthcare professions.

So what we want is students who have gone through their learning experiences, whether these are courses, whether these are their kinds of learning experiences where they can come out and work in the healthcare professions in order to improve patient experience They'll there'll be better experiences for the patients that they work with as well as improving their own staff experience. So they're they're gonna be better practitioners of their particular discipline because they've learned from about, and with other disciplines. So they'll understand what it's like to work on a multi discipline eighteen. Overall, they want to improve health outcomes for a locality, for our country, for the world. In improving health outcomes, and at the same time lowering costs.

So these are the quadruple of aim of care. This is what we're trying to get through by changing the paradigm from the siloed education into interprofessional education. And so the one of the things that the ways this is done or is being assessed is through this idea of helping students earn five competencies. And so while they're taking courses, while they're having these different experiences and learning together, they're all working toward building these five competencies. And the five competencies are intercultural humility, so understanding Who am I? What are my biases? What are my beliefs? What am I bringing to the table? And what are other people bringing to the table in terms of biases and belief and the ways that they see the world.

And having respect for those, interprofessional communication. So not only how does The doctor on the team communicate with the nurse on the team communicate with the social worker on the medical team. But how do they also then communicate together to patient to other people involved in the area of care for someone. Rolls and responsibilities. I think this is an important one, especially in inter, professional education.

Because if I'm training to be a nurse, then I'm also not only learning what is my role and responsibility as a nurse. But what is the social worker's role and responsibility? And then how do we connect across those things? So they're learning their own role and responsibility but also learning how the roles and responsibilities of other people so that they can work better together. And of course teams that place right into teams and teamwork as one of the other efficiencies that they are learning. They're learning how to be on a team. All of the pitfalls, all of the challenges that come with teamwork, but all of the wonderful outcomes that especially in the medical profession when medical professionals are working together as a team.

Not everybody just coming in and do their part and walking away. And finally values and ethics. They we really wanna bring, quality care for the patient. And so this is one things that they're learning as well. Oh, just my microphone keeps moving.

I'm gonna move where the microphone is because the badge won't stay still. So values and ethics is the final thing that they're learning. So these are the five competencies. And the way that the center provider center for Interprofessional education or IPE is working is that they are working with faculty in the different schools in order to develop various courses or experiences that build these competencies for their students in ways that are cross disciplinary. So this is the world at the context in which our project began.

With the center for IP. Great. So thank you, Jennifer, for that introduction. What IP is. So now I'm gonna talk about the challenge what we faced are at our institution with interprofessional education.

As Jennifer mentioned, So the interprofessional education IP is offered by ten schools and colleges, which are listed there to across three campuses to our students. So the student participating in these offer, IP offering grew and As of now, five five five thousand students takes these offering. Also, the number of offering grew from being five to thirty five offerings. So, the center wanted to understand with this, with the growth that they saw. They wanted to understand Well, who are what are the offerings that are being offered by a particular school or college, who are the faculty that are teaching these offerings And who are the students that are participating in these, I IT offerings? I mean, what background they belong to, which major, carrier, etcetera.

And as students, and also the faculty who are teaching it, So as students are taking these offerings, as Jennifer mentioned, they are also gaining these five I ID competence. So it's not just five IP competencies. There are levels of IP competencies, which are defined introduce, reinforce, and practice, introduce means they are getting, like, getting started, with this competency And they are kind of getting familiar, and they are, with reinforced. And the third is that they are ready to apply in the real world, environment. Apart from that, each offering has a dosage level or the contact level.

They wanted to track. So with overall, IP, the center for IP, with the growing demand of participation into these offerings, into the IP curriculum. So they faced with an overall challenge of how do we do the curriculum mapping? Do we do the competency tracking? How do we know who are all the students participating in that? So they had all these questions, and they knew that they needed a way to get the data somewhere so that they could do the analysis that they needed to do to answer their questions about how are students moving through the curriculum? Who's taking what? What faculty are teaching where? And so they did was they reached out to information and technology services. We tend to be the data people on campus. And they said, Hey, you know what? We've been thinking about this for a while, and you know, Canvas is already gonna have all of our students in it.

And so what if what if we got all the other data? What if we got the competency data in there? What if we got all the offering data in there? Is there a way to do that? Could you help us? And so we said, yes, fortunately we have a boss who really likes to work with faculty and reach out and do projects on campus. And if there's any IT folks in the room, if you ever get the chance to work with faculty on projects that they are really passionate about and care about, do it. It's absolutely wonderful. It's been one of the best parts of my career is working with faculty. And so we began this collaboration between these various deans and folks who were, leading the IPE group, and those of us on in the ITS side.

So our teaching and learning team, as well as our data services team, came together to try to solve this problem in order for them to do the analysis that they needed to do. I I think I just wanted to add one more thing. I forgot what the challenge is that. So these offerings, these certify offerings, they are not always, traditional semester based coast. They are clinics, field experiences, simulation, online learning, and service learning, And these, the learning, of these offerings occurs in diverse formats, like the regular courses, online courses, one time setting, one time events, and experimental setting.

And even, like, modules embedded in the courses. So that's one of the challenge that, you know, came, came together to come to this collaboration. So we started with Canvas. I mean, they think they had some great insight there saying, hey, you know what? All of our students are already there. So the course based stuff where they're learning in courses to get these competencies, that stuff's there already.

But like Pushimi said, they're also having all these field experiences and simulations and one time kind of things, but those are were not gonna be in canvas. And so what they wanted to know was, well, if we get those in canvas, can we then find a way to get the competency data tied to it. And so we said, okay, let's explore. You know, what is Canvas got that we can do? And so what they ended up doing was building courses for all of these other offerings, for all of these other experiences, and they could simply upload the enrollment because they knew which students had taken those offerings. So great.

Now we've got the students in Canvas, and we've got all the offerings in Canvas. But then the question was, how do we get the competencies tied to and the level of competencies that students are earning in each one of these offerings tied to those courses, then tied to those students so then we can track who's learning what, what competencies are they building where? So enter the idea of learning outcomes and rubrics. And so what we did was we leveraged canvas learning outcomes and each of the competencies became its own learning outcome, and they could earn that outcome at the three different levels. The introduce the reinforce or the practice. So as we're building the rubric, each one of those competencies becomes, a learning outcome on the rubric, and then we know what's what level they're earning based on the data that IPE has.

And then we also added to the rubric, the option for contact hour. So how much time, that concept of contact hours are they learning each one of these things? Maybe for one session, they're really digging into the intro to intercultural humility. They're not doing anything with ethics, but they're doing with roles and responsibilities, and maybe they've got ten contact hours for that particular offering. And that would be true across all thirty five of the different offerings. Have varying levels of which competencies they're earning, and they're gonna have varying levels of contact hours.

And that is a lot of data to try to get into and associated with each one of those canvas courses, but what we proved out first was if the data is there, you can report on it. And so then it became a question of how do we get the data there. And that's where we started to leverage, the unison data plot form, which Prashimi will tell you about in order to get the data both out of Canvas once it was there. Yes. I think we love Ray UNizon data platform.

It's a robust learning analytical infrastructure that lives in Google Cloud, and of Michigan in collaboration with Unison, built it, and Unison is a higher at consortium, by, it's a nonprofit And, you University of Michigan is one of the founding members, of it. And this is a high level overview of the unison data form. I wanted to spend some time over here, about this branch, about this platform and how we used in building up the IT data platform. I think to start out with, it contact the unison data platform or UDP as we call it. It consists of two foundational stores called contextual store and event store.

So let's talk a little bit of contextual store. Contaxial store is commonly modeled and collased data from student information systems, learning management system, and various learning, teaching and learning tools. So if you want to find information like what so all the students, what are the background they belong to? Like, example, what's their ethnicity whether these students are first generation students versus, what you'll get from the sorry. So what do you, for learning management systems, right? It contains the data of what are all the published courses in Canvas, for for a for us, the learning management system is canvas. So, but it can also model, as I said, it's a commonly modeled, It can also model any other learning management system data.

So you can get the data like what are the published courses, whether modules have been enabled in the courses. So apart from that, it also contains event data. So, and I was talking about this event, that event store So these event data is captured in a Caliper format. So Caliper is a one ed tech standard. So if you want to find out whether a student has watched a video or what is their last activity in the course, all that is present in that.

So the so Unison and the the consortium makes, you know, process the data and put it at the at these two foundational stores. So there are data marks that's kind of targeted to a particular business case. So that's the overview of what this platform is. This is the learning, learning analytical infrastructure that university of Michigan relies on. And now I want to talk about the IP data platform.

So the challenge we started out with is that how do you how do you the center how does the center for IP look at the data of the competency tracking and the curriculum mapping. So all that information is been viewed or accessed by this, by the administration of IP through the Tableau. So Tableau is what used for the presenting. That that. So the data that is feeding the tableau is coming from two data sources over here.

And UDP, I just talked about it. So I wanted to so there are two components of the data we are accessing from UDP. And especially the contextual store, we are pulling the information from. So the information as the students are from ten different schools and colleges. You know, what are the academic program, background, major, and minor? You know, they wanted to center wants to under and all that stuff.

So that information is present, in the the SISIS component of UDP. Apart from that, you know, all the offerings are now part of Canvas courses. Right? What are the enrollment to each Canvas course, what are the course IDs, etcetera. Everything is now, is in the Canvas data, and the, and the UDP's LMS data component. That's where it gets all captured.

And the another thing is, right, the competency assignment, as Jennifer talks about, students are taking these courses or offerings. And they are gaining competencies. And the design is that we wanted to tie the competencies to a learning outcome. So what we did is in order to give competencies based on a criteria. So students are given competencies based on certain criteria, right, just being enrolled in the course itself would give them the competencies that are set out for this particular course.

Or it could be based on the grade based. So we have wrote a Python script. Every end of the term, when students finish taking these offerings, we assign based on that criteria set by the center for set of IP And we'll give them, the competencies along with dosage via the rubrics and learning outcomes. So the reason why we have to write the script, some of our offerings, our particular courses have fifteen hundred students, It is not practical for faculty go on, you know, accessing each, each and every student and give them competence in why we had to write a script. It was it would be a better if Kan was provided a way to bulk bulk upload this in the hands of an instructor.

But that's not in their pipeline right now. So we wrote a script every end of the term, based on the criteria that is set up, students are getting competencies. The second aspect or the second component of this, data pipeline is the IP data store. It's purely the functional, aspect of the data, which offering connects with which course ID and what are all competencies associated with it, and what is the criteria for giving competencies and dosage levels, and which is the faculty that is teaching which are the units participating in an offering. All that data is in the separate data serve as of now, that particular data source is a Google sheet, but we want to get this into a relational data source Well, that's our next phase of the process.

So and both these datas get combined in a de novo de novo is a data virtualization layer. So it does, all the joy joints across these data sources and simply exposing the data that Tableau needs for visualization in forms of tables and views. So Tableau is kind of reduced to just visualization. All the heavy thing of joining, all that stuff is, taken care of by the de novo. Alright.

So that's the technical piece. That's getting the data into canvas so that it can be extracted back of Canvas can align with the SIS data combined with the IPE data to what end? The end is the visualizations. The visualizations that are answering the data questions that the IP, the Center for IPE had about what students are taking, which, which of the different opportunities are they taking which ones are they learning which competencies in? Who's which schools are participating more? Who students maybe aren't learning the competencies or what path what a certain is each program taking. So I don't expect you to really be able to read this really closely. The slides are available.

What we're really looking at is a data visualization here that on the left hand side is showing all of the individual offerings that were available, the term that they were offered in, and then the little bar is showing how many students participated in that offering. And then it's broke down on the right hand side by colored boxes, which are maybe the school of dentistry. So you have dental students. You've got nursing students. You've got med students.

You've got social work students. And the size of the box is showing you how many students in that discipline participated. Now, this is the really high level view, but it also allows them to drill down. And maybe look at individual offerings and then see, okay. Well, how what's the breakdown of students by the different disciplines in that individual offering? Or maybe start with the other side.

Let's look at dentistry students and then drill down that way and see which offerings are they taking. So that's one kind of visualization is seeing how the students are taking offerings in order to earn competencies. The next visualization that we have here is really focused instead on those learning outcomes themselves. So the one at the top is talking about the learning outcomes that students attained. And the colors are showing, were they intro, were they practicing, were they reinforcing? So we can see for each one of the competencies, are students getting all three levels? Are they only getting intro Are they only getting practice levels? And how in the world did we get there if we don't have any intro levels? So that's showing at a high level.

And then again, drilling down by the different disciplines. So what's what are student what are nursing students earning? What are dentistry students earning? And these are just two very simple examples of entire dashboards that they're building, and that they continue to build because now they have access to the data, they can really answer these questions and understand how is interprofessional education working across the University of Michigan? Where can it be improved Where are we doing great? And then of course it's gonna keep leading to more and more questions as data always does. And so what this project has done is given them the ability to really again to ask those questions and answer them in meaningful ways. And I think the outcomes and the benefits we saw as we're working and engaging in this project was, I think the center for, IP could report out to their leadership about student participating in IP offering and schools and, the schools and colleges who are contributing to the IP effort and the degree of competencies the students were gaining. And the kind of the approach or the process we took for our school is for ten schools, three campuses, you know, five thousand students.

But this approach could very well be scaled down to a single program. So and finally, as Jennifer mentioned, collaborate this started as a collaboration. It ended up being with a relationship And in this fact, the fact that who are all came to the project, they are they are from the diverse backgrounds. We really did not know what there's for their subject, where their expertise. And so I think what we valued more is relationship that will be built despite the diversity, we all came together.

So this is a just a quick view of the team, and you can see that we had people from these multiple schools and colleges, with their various expertise and interest in interprofessional education. They were a great team to work with. You've got folks from our teaching learning team, as well as our data services team. And bringing all of these folks together made it possible. And so I think collaboration again is one of those huge lessons that we learned.

That it's great to reach out to folks across campus who might have the skills and expertise that your group may not have in order to answer the questions that you need answered we're also very proud of the fact that this project did learn in did earn a learning impact award. So this is valuable work. We knew it was valuable our campus and the people that we were working with, but it's been wonderful to see that it's valuable to other people. I mean, the fact that you're all here. I hope that means it has some value or interest to you.

And at this point, I don't know how we're doing on time, but we are more than happy to take questions from anyone. You have five minutes. Alright. In the very back first. In the system, as far as learning outcomes, and that good standards being set up already is attached to assign for modules, etcetera.

The set three robust fields power already by our instructional design teams and those individual power or was that part of this process in which you had billing? Maybe some programs were already tracking for accreditation across multiple courses. And then we implemented the ID. E. Pieces or well, that's not the only reason you're being used and you were introducing Okay. That's a great question.

So the question was, did we already have the infrastructure there with the learning outcomes and the rubrics that some folks were already using, maybe in different schools or colleges, or did we have to build Yeah. We had to build it. Learning outcomes are not widely used at the University of Michigan yet, and so that was something that we introduced all of these folks to the different schools too. And we did it, for this particular project, it is the same rubric and the same assignment that is created by the script for every single course. As it so it the script goes in, identifies the course, creates the assignment, attaches the rubric, and then puts the right level of competency gained and contact our for that particular offering.

Yes, but is identified in that Google spreadsheet as an IPE course. It yeah. No. I think the script what we wrote is especially for the competency tracking. So all that SIS information, we are pulling from UDP, our unison data platform.

Now, all these students are in Canvas right now. Because all these are these offerings became canvas courses. And when you have the students, the UDP know that these are, these are the students, and we can usually pull up what are the carrier program background. So the script is just for competency tracking, but all the curriculum mapping, what is the background is through that, the the SIS component of UDP. Alright.

In the middle here. So did you use canvas outcomes? So did we use a canvas outcomes? Yes, we did. We used canvas learning outcomes in order to track those competencies. Uh-huh. When you said the and what tell me about how you set those competencies, where were the decisions made on on those five and and How often do you think you will be reviewing refresh modes to stay current? Okay.

So the question was how do we set the competencies and how often are they reviewed and refreshed? And we'll say that's out of our scope. That is the center for interprofessional education. They set those competencies. We used Canvas to find out how could we get the competencies in there. There was a, in fact, during the project, there was a lot of discussion of how exactly are we gonna word all this? How are we gonna and so We advise them like, let's not change this real often.

But yeah, I don't honestly don't know what their state was. Yeah. As of now, it is standard So in Canvas, we have an IP subaccount. So that's where they create this competencies, and they are tied up with a rubric. So all we do is, like, pull that information from the rubric.

So pretty much standard. But if if if you If if you have a requirement that it keep on changing, all you need to know is, like, which offering you need to decide with which learning outcome, and I think and that that's it. Yes. To your relevance, but how did you track the number of hours? How did you do that? Okay. So the question was, how do we track the number of hours? Actually, we've put a line in the rubric called contact hours that could go from one to one hundred.

And then whatever the whatever was going on. Whatever the group that she said, that's what we put in as that number. So it can look a little weird, like, and can show red on a rubric in Canvas. If you're only getting three contact hours, and the range says it can go from one to one hundred, but it when we talked to the center for IP, they said, you know what? This is a really nobody's gonna get more than a hundred contact hours, and they're probably not ever gonna get less than one. So this is a pretty safe range for us to use on the roo itself.

Yeah. I think with the so each offering has a set of competencies that student receive after taking. So we also define what is the contact hours. So it's unique for every offering. Yeah.

How is it received by faculty was the question? Which is a great question. The center for IPE works closely with the faculty who develop these courses. There's a whole project, a whole process they go through in order to get a competency or a course certified for competency. I think the faculty from the perspective of assigning the rubric they did have to do some training to say, Hey, after the term ends, you're gonna see an assignment show up in your course that won't be in the grade book. It doesn't provide it.

You know, there's no points or anything associated with it. It is just for IPE competency tracking. So we they just had to do some training to let faculty knows, so they weren't weirded out if there was an assignment. And the way that we've developed the script is that if a if a faculty then copies their course to the next term, Even if the assignment comes over, the script is smart enough to go, oh, this is an old one. We're gonna overwrite it with the new competencies if that's changed, you know, in the next term.

So we tried to make as minimal impact as possible on faculty in the way that we set up this process. I think, the when the center was formed, There was a need. The one of the goals they had is the shift in the cult the thinking of the faculty. So cultural shifting. That was also there.

It didn't came about easily for them. They really they were working with the faculty that shift need to happen. And at that I think it's been That's that's what we can, we can say. Health center at the university. I work at.

I mean, he might be that, curious when it comes to applications of just outside do that specific thing, say accreditation. When you're applying these learning outcomes in these rubrics, and the system that you created were identifies as rubric say you have one program that just needed to track a different set of competencies. Will this be easily, beautiful or capable to apply to a with that use case, or is this eye so closely that it Alright. So the question was, does this really only work for us in our particular world, or could this be scaled down a single program who wanted to do something like this in order to spread accreditation or for tracking their own competencies. And I think we had a slide up there.

This program is scalable to a single program. The whole approach can be taken like that. And I think that's what when we were like discussing with our faculty, and this This input came from them. You know, we have a scaled up version. This can definitely scale down, to single program.

Yeah. We didn't we definitely wanted this to be something other folks could use. Your datum sources may be different, but the process can just certainly scalable. Yeah. All the information, Ssis information, Canvas data.

It's all there. It's just that how do you put together? Any other colleges interested in maybe doing something similar to this? Essentially. So the the question was there any other colleges doing this, like, say, maybe the business college. We haven't heard from them. But, there are some information out there.

There are there kinked to it. As I said, this thing grew from being fire offering, and these ten school just instantly did not came. So there was incremental partnership that added, you know, one school joined, like the the other thing. And it's an ongoing thing. Who knows it might go to the business school as well? Alright.

Question in the back. So we don't use Canvas outcomes right now, so I know like this much about it. Do you have a end rubric for each of your experiences that then go up to a main I p e rubric or a kind of that structure Alright. So the question was, do we have a different rubric for each one of these experiences that somehow rolls up? The answer to that is no. There is one rubric.

And so it's each one of the competencies. And for each one of those in the rubric, there's the three levels. And then there's contact hours and the how many contact hours. So that was one of the things we had to do was we had to standardize a rubric that would work across all those offerings. That our script could easily apply it.

I think we kept it simple like that because it worked with our design. But if you have a situation where every experience is tied up with a rubric, all you need to is have that mentioned somewhere in that kind of relational schema as I said. If it's each offering is tied up with the competencies, You can also add a rubric. So basically a rubric's number, if in canvas. So all all the the data is just pulling that the script just pulls that.

Do you have any examples of how this data is being used drive change in the curriculum. Alright. Do we have any examples of how this data is being used to drive train change in the curriculum? I I don't think so that the outcomes of this approach. So we piloted this, in twenty twenty two, you know, They had this goal. You know, how do they track competencies? They had this goal.

And this is the first time last year they were able to do that. I think their next phase of the project is to stand how does this really have that four quadruple goal. Right? That's what they wanted to measure. And that that that's the next phase of the thing they wanted to working on. Any more questions? Okay.

Alright. If not Thank you very, very much for attending. We really appreciate it.