Hate Speech Detection Using Transformer-Based BERT model

Abstract

Hate speech detection has emerged as a significant concern in recent years, particularly with the rise of online platforms where people frequently communicate in code-mixed or resource-scarce languages. Roman Urdu, being a widely used code-mixed language on social media, poses unique challenges due to its lack of standardized grammar and vocabulary, making the task of automated hate speech detection more complex. Traditional approaches often struggle in capturing the nuanced meaning of Roman Urdu expressions, as hate speech is highly context-dependent and cannot always be detected by surface-level word matching. To address these challenges, we utilize the pre-trained Bidirectional Encoder Representations from Transformers (BERT) model, fine-tuned specifically for hate speech classification in Roman Urdu. Our study contributes a novel methodology designed to handle both monolingual and multilingual aspects of Roman Urdu communication. Furthermore, we integrate the Profanity Check Technique (PCT), which combines a ReLU activation function with logistic regression, to effectively distinguish between hate and non-hate content in tweets, thereby improving detection accuracy.