Personalized Feedback and Hints with Korbit

Montreal, Canada, Q1 2021 – Korbit Technologies has published a new research paper at the Educational Advances in Artificial Intelligence (EAAI-21) conference, demonstrating an innovative method for personalized feedback and hints based on techniques from machine learning and deep neural networks driving superior learning outcomes for students.  



As we progress into the 21st century, we delve more into an era dominated by innovative technology. The question we may want to ask ourselves is, how can this technology be used to better the future of our businesses, our workforce and our society as a whole?  Korbit chose to answer this question by taking advantage of state-of-the-art AI technology to take education to the next level with personalized learning paths and active learning (such as problem-based learning and project-based learning powered by AI), all while fulfilling its objective of delivering high quality, accessible education to everyone and anyone who needs it.


The National Academy of Engineering named the development of personalized learning systems a “Grand Challenge” for the 21st century, one that is destined to be transformative on a global scale. Korbit’s new solution for generating personalized feedback and hints automatically and in real-time guarantees its learners the most efficient and most effective learning experience. 


What is Personalized learning & How It Can Help Modern Day Professionals 

The U.S Department of Education defines personalized learning as “instruction in which the pace of learning and the instructional approach are optimized for the needs of each learner…learning activities are meaningful and relevant to learners, driven by their interests and often self-initiated.” So what does this translate to for training our future workforce? By targeting the learner’s specific needs, this approach allows learners to increase learning gains, acquire new skills faster and more effectively and further develop their careers. Not only this, but employees can now guarantee that less time is wasted as their employees’ training will be focused, precise and highly applicable to the company needs. Moreover, employers can ensure that more employees finish their training programs, as it has been shown that highly personalized curricula increase the learner’s motivation to finish by adjusting the path to the employees level and unblocking them when they are stuck.


Korbit’s Personalized Feedback and Hints & How it works

Korbit Technologies added to its existing hints system and developed a new feedback system which can break apart the learner’s answers into distinct concepts and provide separate feedback for each concept or part. Enabled by state-of-the-art machine learning techniques from deep neural networks, the new feedback system  analyzes and decomposes student solutions to decide which parts are correct or require guidance in two main ways. 


Firstly, Korbit’s new feedback system can provide suggestions about missing parts in a student’s answer attempt. A deep neural network decomposes the answer attempt and the expected solution into distinct concepts, after which it then tries to match the decomposed answer parts to the known solutions. If an expected concept or argument is missing from the student’s answer, Korbi can then suggest filling in the missing part, while also giving positive feedback about correct concepts and arguments.


Secondly, It can also detect if a part of the learner’s answer is incorrect. The model detects that the solution attempt only partly matches the expected solution, then reinforces which concepts and arguments are correct and provides hints about the incorrect concepts and arguments. Having multiple feedback sources means that students can get different perspectives on how to improve their answer leading to a better learning outcome. 


The Proven Positive Results

To determine how much these new hints help learners, two experiments were conducted: 1) the learning Gains Experiment which measured real-world learning gains with approximately 100 students, and 2) the Ranked Feedback Experiment which analysed feedback from domain experts. Within these experiments, we compared the new feedback system against two other approaches, the basic feedback model that simply tells students to try again, without elaborating further, and the correct-ideas feedback system, a system that tells students the correct parts in their answer, without any further guidance. 


Through the Learning Gains Experiments, we observed that with Korbit’s full personalized feedback system, students had average learning gains of 51%, compared to 25.6% with the basic feedback model and 23.5% with the correct-ideas feedback system. In other words, the new feedback system doubled the learning gains of students!


Through the ranked feedback experiment, we compared the feedback systems by ranking their feedback messages directly. In this experiment, we collected 91 feedback messages from the three systems. Domain experts then ranked them in a head-to-head comparison, allowing for ties. The results confirmed the observations from the first experiment, with the new feedback system being ranked best 66% of the time by domain experts, double that of the correct-ideas feedback system. 

[text: Huda Hafez]

Leave a Comment

Your email address will not be published. Required fields are marked *