Muscat – Artificial intelligence (AI) tools such as ChatGPT can explain financial concepts with clarity, but these still struggle with numerical reasoning and calculations, according to a new study by researchers at Sultan Qaboos University (SQU).
The research, conducted in collaboration with an international partner, examined whether AI tools can truly match student performance in a specialised subject like corporate finance. The findings suggest that while AI can support learning, it cannot replace human analytical skills.
The study was led by Dr Syed Mujahid Hussain, Associate Professor and Head of the Department of Economics and Finance at the College of Economics and Political Science, SQU, alongside Dr Nisar Ahmad, Dr Aqdas Malik and Dr Sheraz Ahmed of LUT Business School in Finland. The research has been published in the Journal of Educational Technology Systems.
To conduct the study, the researchers used 60 multiple-choice questions drawn from a standard corporate finance textbook, covering topics such as net present value, internal rate of return, time value of money and financial ratios. The questions were taken from real examinations taken by 67 undergraduate students at SQU. The same questions were then presented to ChatGPT-4 on three different occasions to assess consistency in performance.
The results showed a clear divide between theory and practice. ChatGPT performed strongly in descriptive, theory-based questions, scoring an average of 87%, compared to students’ 75%. However, in calculation-based questions, students significantly outperformed AI, achieving an average score of 82%, while ChatGPT scored just 32%.
Explaining the motivation behind the research, Hussain said the research team was prompted by growing classroom experience.
“As faculty members, we regularly check exams and assignments and have noticed increasing use, and sometimes over-reliance, on AI by students,” he told Muscat Daily. “We also use AI tools ourselves when designing exams or reviewing solutions, so we wanted to systematically examine how AI actually performs compared to students across different types of questions.”
According to the study, ChatGPT’s weakness lies in analytical and numerical reasoning. While the tool can retrieve and reproduce narrative explanations effectively, it frequently misapplies formulas, makes calculation errors or misunderstands the structure of financial problems.
“These findings show that current AI models are not yet fully trained to handle analytical reasoning and numerical problem-solving,” Hussain explained. “Finance requires logical sequencing, judgment and precision, which remain largely human strengths.”
The researchers cautioned students against over-reliance on AI tools, particularly when preparing for exams or entering the job market. While AI can help clarify concepts, it may offer only a superficial understanding if used as a primary learning source.
“Students must practise questions from textbooks, class notes and other sources, and develop hands-on skills,” Hussain said. “AI should supplement learning, not replace it.”
The study also raised broader questions for universities as AI becomes more embedded in education. The researcher team emphasised the need to protect and strengthen core human skills such as critical thinking, reasoning and contextual understanding.
“Universities should focus more on case studies, flipped classrooms and applied learning,” Hussain noted. “Equally important are soft skills such as communication, teamwork and problem-solving, which AI cannot replicate”.
Looking ahead, the researchers believe AI will continue to evolve and improve, including in numerical reasoning. However, they stressed that further research is needed to compare different AI models over time and assess how the capabilities of these develop.
“For now, the message is clear: AI is a powerful educational aid, but genuine learning, especially in technical disciplines, still depends on human effort, judgment and understanding,” Hussain said.
