INDOMITA

Hate speech is one of the most central problems of online life, with real-life consequences: various hate crimes started as online hate. 1 in 4 users have been harassed online (Pew Research), 63% of the targets are women (Cox commission). The pandemic-related increase of online activity has only intensified this problem: over 500 million messages are sent each day. To address this problem, content providers and policymakers need automated assistance in spotting and addressing hateful comments. INDOMITA will provide those methods.

But hate speech is complex. What is considered offensive varies by social norms and user demographics. “Yo, a**hole!” is acceptable among friends, but problematic with strangers. But current hate speech detection only uses the words in a message to determine whether it is hate speech or not. It does not consider who says those words and to whom, potentially missing subtle forms of hate speech and mislabeling harmless interactions due to overreliance on keywords. This overly simplified approach is a significant limitation. Our user-based approach will address that.

But “better” detection is subjective: people have very different thresholds for what they find offensive. Current evaluation metrics do not allow for such nuance. Any tool that improves the overall detection rate will be judged sufficient. But a tool that works great for most users, but fails for some other groups might achieve good performance. It still fails in the task it was designed to do. Our fairness metrics will correct this.

But detection alone does not solve the problem. Interventions like counterspeech or education have a lasting impact on abusive users. It can be enough to alert them to the hurtful nature of their message. At other times, they will only respond if someone they perceive as authoritative engages in a discussion. This decision requires an understanding of the abusive user’s social context. Our user-based counterspeech approach facilitates this.

Our novel, user-centered approach will address hatespeech in three ways:

1. comprehensively modeling a complex issue to improve detection across input formats (text, images, and video), by incorporating socio-demographic context into the model.
2. developing methods to automate counterspeech and to address abusive users effectively.
3. developing evaluation metrics that assess fairness and performance and account for the subjective nature of hate speech.

In sum, our user focus will revolutionize existing research on hate speech detection, both in Italian and other languages, to give authorities and media providers better ways to assess content for immediate countermeasures. It will allow us to bridge language differences more easily than purely text-based methods, as we capture socio-behavioral patterns that generalize across languages. It will generate revolutionary insights of the complex dynamics between online actors and the generation of online hate.

INDOMITA is supported by a MUR FARE 2020 initiative under grant agreement Prot. R20YSMBZ8S.