10.11.2026 |  15:30  -  16:30 Uhr

Thema: Can Foundation Models Replace Traditional Machine Learning in Educational Data Mining?
A Case-Study on Small Educational Datasets

Beschreibung: Educational Data Mining (EDM) applications frequently rely on relatively small tabular datasets collected from individual courses, study programs, or institutions. Such datasets often contain only a few hundred or a few thousand instances, limiting the effectiveness of data-hungry machine learning approaches and increasing the risk of overfitting. Recent advances in foundation models for tabular data have introduced a new paradigm in which models are pre-trained on millions of synthetic learning tasks and can subsequently perform classificationand regression on previously unseen datasets with little or no task-specific training. Despite their promising results in various domains, their applicability to educational datamining remains largely unexplored. This research investigates whether foundation models can serve as a viable alternative to traditional machine learning algorithms in educational prediction tasks involving limited amounts of data. We conduct an empirical evaluation of TabPFN (Prior-Data Fitted Networks), a foundation model for tabular data, against widely used baseline methods, including Random Forest, XGBoost, CatBoost, Support Vector Machines, and Multi-Layer Perceptrons. Experiments are performed on freely available educational datasets covering student performance prediction, academic success forecasting, and dropout risk detection. In addition to predictive performance, we analyze robustness to small sample sizes, sensitivity to class imbalance, and model stability across repeated validation procedures. The results indicate that foundation models achieve competitive and often superior predictive performance on small educational datasets while requiring substantially less hyperparameter tuning than conventional approaches. Furthermore, the performance advantage becomes particularly pronounced in low-data scenarios where traditional models struggle to generalize. The findings suggest that foundation models may represent a promising new direction for educational data mining, especially for institutions with limited historical data.

Weitere Informationen

Ort:

Raum Z 312

Referent:

Prof. Branko Kavsek, University of Primorska (Koper, Slovenia)