Machine Learning versus Deep Learning for Network Traffic Anomaly Detection: A Systematic Literature Review

Network intrusion detection systems (NIDS) are still an important defence against cyber threats, which become increasingly complex each year. Among the datasets used to evaluate machine learning (ML) and deep learning (DL) detectors, CICIDS2017 stands out for its realistic traffic, comprehensive collection of flow-level features, wide range of attack taxonomy, and has thus become one of the most commonly used benchmarks in the field. We present a PRISMA-guided systematic review of anomaly detection techniques tested on this dataset, covering traditional ML, deep learning, ensemble and hybrid models, feature selection, generative approaches, transfer learning, and federated learning. Of the 540 records initially found, 53 were peer-reviewed studies meeting all inclusion criteria. We build a direct comparison between thirteen model families of both ML and DL, select models based on inclusion criteria, and compare their results including their strengths, weaknesses, and bestsuited settings. The review reveals eight recurring limitations in the literature: temporal data leakage due to random splitting, limited attention to class imbalance, infrequent use of continual learning, limited explainability, limited reproducibility, very few reproducible results, almost no reporting of the false-positive rate, and absence of adversarial robustness testing. A series of baseline experiments under both random and temporal protocols were conducted. These observations are given empirical grounding through protocols and we conclude with a five-point research agenda.