An Efficient Deep Learning Approach for Text Classification of social media text

Abstract

Text classification, which involves assigning text to specific categories, is a crucial task in natural language processing (NLP) for applications like spam detection, sentiment analysis, and topic identification. While state-of-the-art models perform well, their reliance on large annotated datasets and high computational resources can be a significant limitation, particularly in low-resource environments. This paper presents a resource-saving framework for classifying social media texts by employing knowledge distillation, adapter modules, and improved data augmentation techniques like back-translation and synonym replacement. By using pre-trained transformer models, this approach facilitates effective learning from rich language resources. Experimental findings show that a Distilled Transformer model can achieve an accuracy of 86.5% , which is comparable to the Full BERT model's 88.7% accuracy , but with reduced training time, memory usage, and inference latency. This methodology, which leverages data augmentation and transfer learning, is suitable for operation on edge devices and in environments with limited resources.