Authors:
Alacam, Özge, oezge.alacam@uni-bielefeld.de, Computational Linguistics, University of Bielefeld
Hoeken, Sanne, shoeken@uni-bielefeld.de, Computational Linguistics, University of Bielefeld
Zarrieß, Sina, sina.zarriess@uni-bielefeld.de, Computational Linguistics, University of Bielefeld
Keywords: hate speech, gaze signals, pupil-size parameters, subjectivity
Abstract:
Hate speech is a highly complex and subjective phenomena, of which annotation/evaluation is dependent on the domain of hate as well as on the individual annotators’ backgrounds and biases How to deal with this subjective variation in human annotations and detection of hate speech has been a long-standing and notorious question for research in NLP.Such task-specific reading occurs in two stages: first, the sentence must be perceived and comprehended at both the syntactic and semantic levels. Then reasoning/decision making processes take place to complete the task. To date, existing research in NLP has been mainly inspired by reading studies and commonly utilized few gaze features related to information intake and cognitive load featuring fixations or saccades However, reading and evaluating hateful text also evokes intense emotions (e.g. feeling empathy, being personally affected by the hate speech). In this study, we analyze a wide range of fixation, saccade and pupil-size parameters, comparing their predictive power for subjective hate ratings using the Gaze4Hate dataset. This dataset includes gaze, hatefulness ratings and rationales collected from 43 participants evaluating 90 sentences that express positive, neutral or hateful statements towards women. At this workshop, we aim to discuss our findings in relation to two main research questions: - Do gaze features provide robust predictors for subjective hate speech annotations? Different eye-movement parameters capture different aspects of hate speech. While pupil-size parameters are sensitive towards the strength of the sentiment (towards very positive and very negative statements), the fixation-based parameters can be used as a predictor for differentiating hate versus no hate speech. - Are gaze features useful for enriching LMs for hate speech detection? We concatenate individual gaze features with sentence representations obtained from various pretrained and fine-tuned language models (BERT, fine-tuned BERT for hate speech classification, em-LLaMA2 and em-Mistral models). Our results indicate that incorporating gaze features enhances model robustness in detecting subjective hate speech, regardless of the language model’s size. References are not included in the short abstract version.