publications

2024

1. NAACL 2024

   The Uli Dataset: An Exercise in Experience Led Annotation of oGBV

   Arnav Arora, Maha Jinadoss, Cheshta Arora, and 5 more authors

   🏆 Outstanding Paper Award

   In The 8th Workshop on Online Abuse and Harms at Annual Conference of the North American Chapter of the Association for Computational Linguistics, 2024

   Abs HTML PDF Website

   Outstanding Paper Award

   Online gender-based violence has grown concomitantly with the adoption of the internet and social media. Its effects are worse in the Global majority where many users use social media in languages other than English. The scale and volume of conversations on the internet have necessitated the need for automated detection of hate speech and, more specifically, gendered abuse. There is, however, a lack of language-specific and contextual data to build such automated tools. In this paper, we present a dataset on gendered abuse in three languages- Hindi, Tamil and Indian English. The dataset comprises of tweets annotated along three questions pertaining to the experience of gender abuse, by experts who identify as women or a member of the LGBTQIA+ community in South Asia. Through this dataset, we demonstrate a participatory approach to creating datasets that drive AI systems.

2022

1. TACL

   Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages

   Gowtham Ramesh, Sumanth Doddapaneni, Aravinth Bheemaraj, and 9 more authors

   Transactions of the Association for Computational Linguistics, 2022

   Abs HTML Website

   We present Samanantar, the largest publicly available parallel corpora collection for Indic languages. The collection contains a total of 49.7 million sentence pairs between English and 11 Indic languages (from two language families). Specifically, we compile 12.4 million sentence pairs from existing, publicly available parallel corpora, and additionally mine 37.4 million sentence pairs from the Web, resulting in a 4\mbox\times increase. We mine the parallel sentences from the Web by combining many corpora, tools, and methods: (a) Web-crawled monolingual corpora, (b) document OCR for extracting sentences from scanned documents, (c) multilingual representation models for aligning sentences, and (d) approximate nearest neighbor search for searching in a large collection of sentences. Human evaluation of samples from the newly mined corpora validate the high quality of the parallel sentences across 11 languages. Further, we extract 83.4 million sentence pairs between all 55 Indic language pairs from the English-centric parallel corpus using English as the pivot language. We trained multilingual NMT models spanning all these languages on Samanantar which outperform existing models and baselines on publicly available benchmarks, such as FLORES, establishing the utility of Samanantar. Our data and models are available publicly at Samanantar and we hope they will help advance research in NMT and multilingual NLP for Indic languages.