AI (artificial intelligence) has quickly shifted from science fiction to an integral part of our daily lives. It’s everywhere, from virtual assistants like Siri and Google Assistant to social media algorithms, making it an invisible yet essential part of modern life.
So it was inevitable that AI would also make its way into the classroom. But does AI help or hinder students’ performance? Studies suggest that the impact depends largely on how AI is implemented. When applied correctly, AI tutoring could achieve similar benefits as one-on-one tutoring, offering an advantage over traditional classroom settings.
Positive Impact of AI on Students
A 2024 randomized study published on Research Square found that AI, when designed effectively, can significantly enhance learning. The study showed that students who used AI tutoring learned more than twice as much in less time, while also reporting higher engagement and motivation levels. In fact, 83% of students said the AI tutor's explanations were as good as, or better than, those from human instructors in the classroom.
The Problem with Unguided Use
However, another study showed contrasting results, at least at first glance. This research found that nearly half of physics students’ solutions, when generated with ChatGPT assistance, were incorrect. Additionally, students using ChatGPT exhibited an increase in direct copy-pasting, which hindered deep learning. But the key difference here is how AI was used. These students were using ChatGPT directly rather than a customized AI tutor designed to support learning.
Why AI Implementation Matters: Custom Tutors vs. General Tools
This distinction is critical, and another study tested it directly. Researchers compared a general AI assistant (like ChatGPT) with a customized GPT tutor that encouraged learning by avoiding full solutions and instead guiding students step-by-step. Both groups showed short-term improvements over the control group, but the improvement was significantly higher for students using the custom GPT tutor.
What About Long-Term Impact?
While unguided AI use may offer short-term benefits, it could harm students in the long run. When AI access was removed, students who had used the basic GPT-like solution performed 17% worse on exams compared to those who never had AI assistance. This suggests that students relied on AI as a "crutch", failing to internalize the material. However, students who used the custom AI tutor performed about the same as the control group, indicating that a well-designed AI tutor can prevent learning loss.
The Key to Effective AI Tutoring
It seems that the key to AI tutoring success is designing platforms that follow the same evidence-based strategies proven effective in other environments, such as the traditional classroom.
These include
- Facilitating active learning
- Managing cognitive load
- Promoting a growth mindset
- Scaffolding content
- Ensuring accuracy of information and feedback
- Delivering timely, targeted feedback
- Allowing for self-pacing
The latter two are particularly relevant in AI tutoring, just like in one-on-one tutoring. By adhering to these best practices, AI tutoring systems can enhance learning outcomes and engagement.
Sources:
“AI Tutoring Outperforms Active Learning” (2024, Research Square)
“Generative AI Can Harm Learning” (2024, SSRN)
“Unreflected Acceptance -- Investigating the Negative Consequences of ChatGPT-Assisted Problem Solving in Physics Education” (2023, arXiv)
