Automation of Classification Tasks with Imbalanced Datasets using an Adapted CRISP-DM Methodology

The authors of this paper propose and evaluate the performance of an automated approach for solving classification tasks with imbalanced datasets using an adapted CRISP-DM methodology. Structured programming in Python was used to develop an object-oriented system that operationalises each phase of the CRISP-DM methodology - from business understanding to deployment - using specifically defined methods inside a class. Automation of the process is enabled by iteration over multiple datasets and by multiple oversampling techniques - SMOTE and ADASYN - with evaluation metrics such as balanced accuracy, F1 score for each class individually class and AUC-ROC score calculated in a standardised format. The results show consistent performance of the SMOTE method compared to ADASYN, with statistical validation using the McNemar test repeatedly confirming no significant difference (p > 0.05). The work highlights the importance of integrating statistical model testing into the CRISP-DM methodology and shows that a high degree of replicability, scalability and transparency in the analysis of imbalanced classification problems can be achieved using a modular approach. A Python implementation of an automated approach for solving classification tasks with imbalanced datasets using an adapted CRISP-DM methodology is available at the following GitHub link: https://github.com/smitroviefos/crisp_disbl_paper2025.