Enhanced Text Classification Using Random Forest: Comparative Analysis and Insights on Performance and Efficiency

Abstract

Text classification is a critical task in natural language processing (NLP), with applications ranging from spam detection to sentiment analysis and document categorisation. This research investigates the application of the Random Forest (RF) algorithm for text classification, highlighting its performance relative to traditional classifiers, such as Naive Bayes (NB), Support Vector Machines (SVM), and Logistic Regression (LR). Experiments were conducted on the 20 Newsgroups dataset, which is a benchmark text dataset characterised by high-dimensional and sparse data. The methodology involved rigorous preprocessing steps, including tokenisation, stopword removal, stemming, and vectorisation, using TF-IDF. Random Forest achieved superior performance, with an accuracy of 89.3% and an F1-score of 88.1%, surpassing SVM (87.6% accuracy, 87.2% F1-score), LR (86.3% accuracy, 86.0% F1-score), and NB (82.4% accuracy, 81.9% F1-score). RF also demonstrated competitive computational efficiency, with a training time of 2.0 seconds, making it faster than SVM (3.8 s) and comparable to LR (2.5 s). The results underscore the robustness of RF in handling high-dimensional text data, offering a balance between predictive performance and computational efficiency. However, the study identified certain limitations, such as dependence on TF-IDF for feature representation and the scope of experiments being limited to a single dataset. Future work will focus on integrating RF with advanced text embeddings and expanding its evaluation to diverse datasets. These findings establish RF as a reliable, interpretable, and efficient classifier for text classification tasks, with significant potential for further enhancement in modern NLP applications.