From the Engineering Desk: Insight Into Our Current Experiments

As the SVP of Engineering at Instructure, AI integration and capability is top of mind. I’d like to share some of my thoughts on AI with you as we continuously explore this new technology and how it can benefit students and educators everywhere. 

With AI, Context Matters 

To get value out of AI like ChatGPT, sending the right prompt makes a big difference. An AI Prompt is any form of information–like text, a question or code–that tells AI what you are looking for. The better the prompt and context, the better the result. We want the tremendous benefits that AI promises to educators and students, but how do we balance data sharing with an AI with privacy rights, potential bias and data ownership? 

We have been running a bunch of experiments with OpenAI’s program interface for a while now. During these experiments, we have been impressed with the configurability of that interface to significantly tailor and tune the responses to be as useful as possible in the educational context. To successfully prompt these tools, however, we've found that they require the right context and data. A well-crafted prompt augmented with the right context produces correct outputs more often than just a prompt alone. Balancing this kind of data sharing against student and educator privacy must be first priority. As education technologists seeking opportunities to advance edtech, we must preserve data privacy and prevent bias in the use of AI.

Nuance vs. Bias 

As an example of this balancing act and its impact on system correctness and bias, let's consider one of our experiments in this space. As all educators know, one of the most labor intensive teaching activities is grading. One option we are exploring is an AI grading assistant. A tool like this could use a lot of data, for example:

• The original question
• The student’s response
• The grading rubric or ideal response
• Full or partial credit allowed
• Grade level
• Relevant teaching materials from the course
• English language proficiency level

As we increase the amount of data we send, we receive better, more nuanced answers from the tool. For example, if an instructor is asking for help creating a rubric about the Grand Canyon, what data can be sent to get a useful result?

It’s clear that if I were a 5th grade teacher, the example response for a 10 year old is much more appropriate. However, do we know what the AI considers to be a “10 year old”? What if that 10 year old is speaking English as a second language? Should we share that with the AI? What bias does that inject? Should edtech tools be in the position to collect and store that data?

Another Example in Action 

In another iteration of the experiment, we tried to get the most value with the lowest amount of context sharing to balance privacy, bias and effectiveness by sending just the student response and the grading rubric. 

The rubric sent to the model:

• Must mention that the Grand Canyon took around 6 million years to form.
• Must also mention something about the Colorado River.
• Must mention that there are 3 primary rock layer sets.
• Must describe the 4 states that the grand canyon runs through.

As you can see in these examples, additional context and extra data matters. It produces better responses and, in some cases, can defeat latent bias in the model. Before we send any piece of student, educator, or school data, though, we must ensure that we're proper stewards of your data and use it only in a transparent, ethical way with prior consent.

To add to what our Chief Privacy Officer, Daisy, shared in her post on the ethical use of AI, we also want to magnify the value that AI gives us by including context as we interact with the model. Our commitment is that when these features are enabled in our products, we will disclose all of the data that is included in our interaction with the AI. We believe that this disclosure will give confidence that 

1. No inappropriate bias is being applied
2. The AI has enough context to provide as useful a response as possible
3. We are committed first and foremost to protecting the data of students, educators and institutions.

As we continue our experiments and learn more, we look forward to sharing it with you. 

Check out more on Instructure’s product approach when it comes to AI here.