Personalized Hints Improve Learning Gains in Korbit Study

Montreal, Canada, June 25 — Korbit reveals promising results in a new study published at the AIED conference highlighting how personalized hints improve learning.

In a new study published at the 21st International Conference on Artificial Intelligence in Education (AIED), Korbit reveals promising results for its new personalized hints feature. The scientific article shows how Korbit uses machine learning to generate automated hints tailored to each student’s learning needs, and how these personalized hints improve the overall learning experience.


Its quest to democratize education and offer an accessible alternative to personal human tutors, Korbit innovates once again. In collaboration with scientific advisors Ekaterina Kochmar (University of Cambridge) and Joelle Pineau (MILA and McGill University), the Korbit team developed a new feature that allows Korbi, a personal AI-powered tutor, to automatically produce personalized hints to help students while they attempt to solve exercises.

Belonging to an era where online education is as popular as ever, Korbit distinguishes itself from other passive, one-size-fits-all learning platforms by providing an interactive and personalized learning experience where students solve exercises and interact with their own personal tutor, Korbi.

This new personalized hints feature is an important milestone in mimicking personal human tutors with AI, and the results shown in the AIED study confirm that it leads to significant improvements in student learning outcomes and students’ appreciation of the learning experience.

Amidst the global COVID-19 pandemic, there has been an increase in online learning from all levels of education. Much attention has been brought to the deficiency of conventional online learning while hopes for personalized learning as a sustainable solution are increasing. Students across the world have been studying at home with Korbi the AI-tutor and taking advantage of this novel learning resource including the new personalized hints feature.

The Personalized Hints Features

Korbit’s new personalized hints feature incorporates three types of hints: text-based hints and explanations, Wikipedia-based explanations, and mathematical hints. These new tools will add to Korbi’s already diverse repertoire of feedback interventions.

1. Text-based hints and explanations:

Shown below is an illustration of how Korbi selects a personalized text-based hint after the student provides an incomplete answer for an exercise on linear regression.

Personalized Hints Improve Learning
Image: Text-based hints and explanation

To achieve this, the system goes through two steps. First, it analyzes the available set of correct solutions for the exercise with natural language processing (NLP) techniques, and then generates a large collection of possible hints based on the correct answers. Second, the system ranks all the generated hints based on their quality and appropriateness for the student. This is where the personalization happens. The system uses machine learning models that take into account the student’s performance and past interactions with Korbi to select the best possible hint.

2. Wikipedia-based explanations:

Wikipedia-based explanations provide an alternative way of helping students assimilate concepts and solve exercises. Below is an example of how Korbi uses a Wikipedia-based explanation to help a student through an exercise on artificial intelligence:

Personalized Hints Improve Learning
Image: Wikipedia-based explanations

The system achieves this by extracting key concepts from the question and the expected answers, and then generates explanations based on a large collection of related Wikipedia articles. Korbi then selects the most relevant Wikipedia-generated hint using machine learning and state-of-the-art NLP techniques.

3. Mathematical Hints:

The third new personalized hint feature added to Korbi’s feedback toolkit consists of mathematical hints. This type of hint is given for exercises that can be solved with mathematical equations. Shown below is an example of a mathematical hint given for an exercise on stochastic gradient descent (SDG), an algorithm used for training neural networks.

Personalized Hints Improve Learning
Image: Mathematical Hints
Image: Mathematical Hints

For instance, when a student inputs an incorrect equation, Korbi will give a hint on what could be changed in the student’s equation to get the correct solution. This hint is also personalized for the student with machine learning models.

Promising Results 

To test out the three new personalized hint features, the Korbit team ran an experiment on their platform involving 183 students and 796 student-tutor interactions.

The efficacy of the text-based hints given by Korbi were evaluated with the learning gains they produced. The learning gain for a specific hint is defined as “the proportion of times students answer correctly after receiving the hint”. The better the hint, the higher the learning gain.

The results for the personalized text-based hints are highly promising. The table below shows the average learning gains for personalized text-based hints in comparison with non-personalized hints.

  Average learning gains
Non-personalized hints 37.9%
Personalized hints 60.5%

This shows that Korbi’s personalization hints are very beneficial for students’ learning outcomes, as it increases learning gains by over 67%.

For the Wikipedia-based explanations, the students were asked directly for feedback. They found Korbi’s Wikipedia-based explanations to be helpful 83.3% of the time. This is encouraging and shows that personalized Wikipedia-based explanations are an effective tool to target students’ knowledge gaps.

To evaluate the quality of the mathematical hints, two domain experts were brought in and were asked to independently rate the mathematical hints given by Korbi as “Very useful”, “Somewhat useful”, and “Not useful”. With the results below, the feedback was overwhelmingly positive, with almost 90% of hints being at least somewhat useful, and half being very useful.

“Not useful” 9.3%
“Somewhat useful” 39.5%
“Very useful” 51.2%

With these new personalized hinting tools, Korbi continues to improve and moves closer towards its goal to make data science and machine learning accessible to all.

Join thousands of students and young professionals and start learning data science now! #LearnDataSciencewithAnAI #DataScience.

Contact for the press

Public Relations: Ansona Ching []