How to assess and boost your educators’ proficiency with your LMS

How should your institution rank the proficiency of your educators and trainers’ use of your learning management system (LMS)? Experience can be an expensive teacher, so why not learn from someone who’s already done the work?

The University of the East, in Manilla, the Philippines, has done just that. By e-surveying 674 of 795 teachers using the Canvas LMS at that institution, it developed a three-tiered system to gauge users’ self-reported ease of using Canvas’ many features. 

Roselle S Basa, Program Coordinator for Instructional Technology at the university’s Office of Curriculum Development and Instruction, shared her institution’s methodology and insights at the 2022 APAC InstructureCon. You can see the full presentation here.

She found that almost half of the surveyed Canvas users reached top proficiency in using the LMS. The results are informing targeted training to boost proficiency across the organisation.

Why gauge proficiency?

The impetus for assessing teachers’ Canvas proficiency was to build their capacity continuously, says Roselle.

“With the ever-changing challenge for faculty members to be equipped in handling various instruct modalities, it is a must that relevant capacity-building programs be designed and deployed.”

The University of the East offers K-12, tertiary and graduate-school courses with two major city campuses comprising a total of 12 colleges and two basic education departments. The university first signed onto Canvas LMS in 2018. It became mandatory for all users from senior high school to graduate school from the summer 2020 term.

Training to the test

To upskill staff in Canvas, users undertook workshops, face-to-face sessions, and for academic-unit heads, change management training. Initially, teachers collaborated to develop blueprints for nine general education core courses.

Roselle said: “Once these blueprints courses were deployed, they still allowed faculty members to exercise their academic freedom and creativity. They can still incorporate their own materials or build upon what’s available in the blueprint.”

A survey conducted by the University of the East was keen to establish a baseline of users’ understanding of Canvas. Just under a quarter of those surveyed had used Canvas for four years or more, 32% for three years, 42% for two years, and only 2.4% had used it for a year.

Roselle and her team began the task by identifying several key Canvas features that educators and trainers should be used with ease based on what their training covered. They were:

1. Pages
2. Modules
3. Announcements 
4. Quizzes
5. Assignments
6. Chats
7. Messages/inbox
8. Discussions
9. Attendance
10. Grades
11. New Analytics
12. People
13. Calendar

Next, they grouped those features into three levels – bronze, silver and gold - to rank proficiency. Rather than develop a test for each staff member, the University of the East used electronic surveys. Respondents self-reported their familiarity and ease of using each feature by ranking them on a Likert scale.

Going for gold

Roselle said: “To achieve gold-level proficiency skills, faculty members were expected to use the features to perform the tasks easily. They would have explored advanced features of Canvas or employed alternative strategies to achieve outcomes.”

More than eight in 10 users achieved bronze proficiency, about two-thirds scored silver, and a significant 47.2% achieved gold, said Roselle.

The University of the East is happy to share its criteria for each proficiency level – you can see them all in the presentation.

Here are the five activities that even gold-proficiency level users found difficult:

• Automatically apply deductions for late submissions
• Split student name into two columns: last name and first name
• Use New Analytics to address the concerns of struggling students 
• Use New Analytics to determine the performance of each student in the class, and
• Customise the colour indicators for late and missing submissions.

Goals for future training

The results help the University of the East develop targeted topics for Canvas training programs. This academic year, it will maximise the use of Canvas features and align those with its institutional goals. On the training agenda for faculty members are:

• Differentiated learning through mastery path
• Migration to new quizzes before sunset, and
• Set up and integrate outcomes into course components.

Students will learn how to build their learning portfolios, and college heads will be trained in how to monitor courses on Canvas.

Roselle says her university doesn’t currently have the full Canvas functionality; she hopes an upgrade to Canvas Impact is on the cards. 

“Currently, our IT department downloads the Canvas data for specific and limited use, such as looking at the Canvas logs for attendance and monitoring faculty during synchronous sessions. We hope to use Canvas data with more depth and breadth in the future,” Roselle said.

“We do not yet have a sophisticated Canvas data dashboard with visualisations like charts and graphs. It’s a dream that will hopefully be actualised.”

If you’re curious about the Canvas functions your institution may be missing out on, get in touch today for a demonstration.