Skip to content

Resources (conference/journal publications, references to dataset) for harmful memes detection.

License

Notifications You must be signed in to change notification settings

firojalam/harmful-memes-detection-resources

Repository files navigation

Automatic Harmful Memes Detection Resources

This repository contains resources (conference/journal publications, references to datasets) associated the work "Detecting and Understanding Harmful Memes: A Survey" accepted at IJCAI-ECAI (Survey), 2022.

The automatic identification of harmful content online is of major concern for social media platforms, policymakers, and society. Researchers have studied textual, visual, and audio content, but typically in isolation. Yet, harmful content often combines multiple modalities, as in the case of memes, which are of particular interest due to their viral nature. With this in mind, here we offer a comprehensive survey with a focus on harmful memes. Based on a systematic analysis of recent literature, we first propose a new typology of harmful memes, and then we highlight and summarize the relevant state of the art. One interesting finding is that many types of harmful memes are not really studied, e.g., such featuring self-harm and extremism, partly due to the lack of suitable datasets. We further find that existing datasets mostly capture multi-class scenarios, which are not inclusive of the affective spectrum that memes can represent. Another observation is that memes can propagate globally through repackaging in different languages and that they can also be multilingual, blending different cultures. We conclude by highlighting several challenges related to multimodal semiotics, technological constraints and non-trivial social engagement, and we present several open-ended aspects such as delineating online harm and empirically examining related frameworks and assistive interventions, which we believe will motivate and drive future research.

Maintenance Last Commit Contribution_welcome

Overview

This repo contains relevant resources Automatic Harmful Memes Detection. We list a comprehensive and up-to-date information for harmful meme detection.

Table of Contents:

Harmful Memes Detection

Datasets

  • Harmful Memes: Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md Shad Akhtar, Preslav Nakov, Tanmoy Chakraborty MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets, 2021. [Paper] [Dataset]

  • Harmful Memes: Shraman Pramanik, Dimiter Dimitrov, Rituparna Mukherjee, Shivam Sharma, Md. Shad Akhtar, Preslav Nakov, Tanmoy Chakraborty, Detecting Harmful Memes and Their Targets, 2021. [Paper] [Dataset]

  • Hateful Memes: Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, Davide Testuggine, The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes, 2020. [Paper] [Dataset]

  • Fine-grained Propaganda Memes: Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar, Firoj Alam, Fabrizio Silvestri, Hamed Firooz, Preslav Nakov and Giovanni Da San Martino, Detecting Propaganda Techniques in Memes, 2021. [Paper] [Dataset]

  • Racist or Sexist Meme: Haris Bin Zia, Ignacio Castro, Gareth TysonRacist or Sexist Meme? Classifying Memes beyond Hateful, 2021. [Paper] [Dataset]

  • Antisemitism Detection: Mohit Chandra, Dheeraj Pailla, Himanshu Bhatia, Aadilmehdi Sanchawala, Manish Gupta, Manish Shrivastava, Ponnurangam Kumaraguru, Subverting the Jewtocracy: Online Antisemitism Detection Using Multimodal Deep Learning, 2021.

  • Troll Memes: Shardul Suryawanshi, Bharathi Raja Chakravarthi, Pranav Varma, Mihael Arcan, John P. McCrae and Paul Buitelaar, A Dataset for Troll Classification of TamilMemes, 2020. [Paper] [Dataset]

  • Sexist meme: Elisabetta Fersini; Francesca Gasparini; Silvia Corchs, Detecting Sexist MEME On The Web: A Study on Textual and Visual Cues, 2019. [Paper] [Dataset]

  • Hateful Memes: Hannah Kirk, Yennie Jun, Paulius Rauba, Gal Wachtel, Ruining Li, Xingjian Bai, Noah Broestl, Martin Doff-Sotta, Aleksandar Shtedritski, Yuki M Asano, Memes in the Wild: Assessing the Generalizability of the Hateful Memes Challenge Dataset,

  • Offensive Memes: Benet Oriol Sabat, Cristian Canton Ferrer, Xavier Giro-i-Nieto, Hate Speech in Pixels: Detection of Offensive Memes towards Automatic Moderation, 2019. [Paper] [Dataset]

  • Hateful memes (Pinterest) Hannah Kirk, Yennie Jun, Paulius Rauba, Gal Wachtel, Ruining Li, Xingjian Bai, Noah Broestl, Martin Doff-Sotta, Aleksandar Shtedritski, Yuki M Asano, Memes in the Wild: Assessing the Generalizability of the Hateful Memes Challenge Dataset, [Paper] [[Dataset](https://github.com/iamatulsingh/ pinterest-image-scrap)]

  • MMHS150K: Raul Gomez, Jaume Gibert, Lluis Gomez, Dimosthenis Karatzas, Exploring Hate Speech Detection in Multimodal Publications, 2019. [Paper] [Dataset]

  • MultiOFF: Shardul Suryawanshi, Bharathi Raja Chakravarthi, Mihael Arcan, Paul Buitelaar, Multimodal Meme Dataset (MultiOFF) for Identifying Offensive Content in Image and Text [Paper] [Dataset]

Relevant Studies

Hate

  • Detecting Harmful Memes and Their Targets [paper]
  • MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets [paper]
  • Racist or Sexist Meme? Classifying Memes beyond Hateful [paper]
  • “Subverting the Jewtocracy”: Online Antisemitism Detection Using Multimodal Deep Learning [paper]
  • Detecting Sexist MEME On The Web: A Study on Textual and Visual Cues [paper]
  • Memes in the Wild: Assessing the Generalizability of the Hateful Memes Challenge Dataset [paper]
  • Hate Speech in Pixels: Detection of Offensive Memes towards Automatic Moderation [paper]
  • Disentangling Hate in Online Memes [paper]
  • Exploring Hate Speech Detection in Multimodal Publications [paper]

Offensive

  • AOMD: An Analogy-aware Approach to Offensive Meme Detection on Social Media [paper]
  • KnowMeme: A Knowledge-enriched Graph Neural Network Solution to Offensive Meme Detection [paper]
  • An approach to detect offence in Memes using Natural Language Processing(NLP) and Deep learning [paper]
  • Multimodal Meme Dataset (MultiOFF) for Identifying Offensive Content in Image and Text [paper]

Propaganda

  • Detecting Propaganda Techniques in Memes [paper]
  • MinD at SemEval-2021 Task 6: Propaganda Detection using Transfer Learning and Multimodal Fusion [paper]
  • Volta at SemEval-2021 Task 6: Towards Detecting Persuasive Texts and Images using Textual and Multimodal Ensemble [paper]
  • Alpha at SemEval-2021 Task 6: Transformer Based Propaganda Classification [paper]
  • Fine-grained analysis of propaganda in news article [paper]

Harassment/Cyberbullying

  • Detection of Cyberbullying Incidents on the Instagram Social Network [paper]
  • A Dataset for Troll Classification of TamilMemes [paper]

Violence

  • Violence detection in hollywood movies by the fusion of visual and mid-level audio cues [paper]

Self-Inflicted Harm

  • The self—harmed, visualized, and reblogged: Remaking of self-injury narratives on tumblr [paper]

Relevant Surveys

  • A unified taxonomy of harmful content [paper]
  • A survey on multimodal disinformation detection [paper]
  • A survey on stance detection for mis- and disinformation identification [paper]
  • Detecting abusive language on online platforms: A critical analysis [paper]
  • A Survey on Computational Propaganda Detection [paper]
  • A multimodal memes classification: A survey and open research issues. [paper]
  • A survey on automatic detection of hate speech in text [paper]
  • Automatic cyberbullying detection: A systematic review[paper]
  • A survey on hate speech detection using natural language processing [paper]
  • A survey of video violence detection [paper]
  • A survey of fake news: Fundamental theories, detection methods, and opportunities [paper]

Current SOTA

Below we provide a list of recent work with current state-of-the-art results.

Title Types Task Dataset Task Type Approach AUC Acc F1 Details
Detecting Harmful Memes and Their Targets Harm Harmful vs Non-harmful HarMeme (Covid-19) Binary VisualBERT (Pretrained using MS COCO) 0.81 0.8
Very harmful vs Partially-harmful vs Non-harmful Multiclass 0.74 0.54
Target Identification of Harmful Memes Multiclass 0.76 0.66
MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets Harm Harmful vs Non-harmful Harm-C Binary MOMENTA: CLIP, VGG-19, DistilBERT, Cross-modal attention fusion (CMAF) 0.84 0.83
Very harmful vs Partially-harmful vs Non-harmful Harm-C Multiclass 0.77 0.55
Target Identification of Harmful Memes Harm-C Multiclass 0.78 0.7
Harmful vs Non-harmful Harm-P Binary 0.9 0.88
Very harmful vs Partially-harmful vs Non-harmful Harm-P Multiclass 0.87 0.67
Target Identification of Harmful Memes Harm-P Multiclass 0.79 0.69
Racist or Sexist Meme? Classifying Memes beyond Hateful Hate Protected category (PC) identification FBHM ML CIMG
+ CTXT
+ LASER
+ LaBSE
0.96
Detecting attack type (AT) FBHM Multilabel 0.97
“Subverting the Jewtocracy”: Online Antisemitism Detection Using Multimodal Deep Learning Hate Antisemitic content detection GAB Binary MFAS 0.91
Twitter Binary MFAS 0.71
Antisemitism category classification GAB Multiclass MFAS 0.67
Twitter Multiclass MFAS 0.68
Detecting Sexist MEME On The Web: A Study on Textual and Visual Cues Hate Automatic detection of sexist memes. The MEME Binary Late fusion 0.76 Multi-modal Late-fusion
Hand-crafted visual dec\scriptiors used: Low-level greyscale, colored, photographic and semantic features.
Bag-of-words approached used as textual features.
SVM
Memes in the Wild: Assessing the Generalizability
of the Hateful Memes Challenge Dataset
Hate Hateful Meme Detection FBHM Binary CLIP (Linear Probe) 0.56
Pinterest Binary CLIP (Linear Probe) 0.57
Hate Speech in Pixels: Detection of Offensive Memes towards Automatic Moderation Hate Hateful Meme Detection Google Binary BERT, VGG-16, MLP 0.83
Disentangling Hate in Online Memes Hate Hateful Meme Detection FBHM Binary DisMultiHate (BERT, Faster-RCNN, Disentangled representations) 0.83 0.76
MultiOFF Binary 0.65
Exploring Hate Speech Detection in Multimodal Publications Hate Hatespech detection in multimodal publications MMHS150K Binary FCM (Feature concatenation model), Inception-V3, LSTM 0.73 0.68 0.70
AOMD: An Analogy-aware Approach to Offensive Meme Detection on Social Media Offensive Offensive (analogy) meme detection GAB Binary Analogy-aware Multi-modal Representation Learning (Faster
R-CNN, ResNet50, Glove-based LSTM, BERT)
MLP
0.69 0.56
Reddit Binary 0.72 0.49
KnowMeme: A Knowledge-enriched Graph Neural Network Solution to Offensive Meme Detection Offensive Offensive meme detection Reddit Binary YOLO V4, ConceptNET, GNN 0.73 0.49 Knowledge-aware Multimodal Entity Extraction (KMEE): YOLO V4
Knowledge-enriched Information Network Construction: ConceptNET
Supervised Offensive Meme Classification: GNN
GAB Binary 0.7 0.55
An approach to detect offence in Memes using Natural Language Processing(NLP) and Deep learning Offensive Offensive meme detection Offensive + Intensity dataset Binary CNN, FastText, LSTM - Sigmoid 0.96
Offense intensity prediction Multiclass CNN, FastText, LSTM - Softmax 0.99
Multimodal Meme Dataset (MultiOFF) for Identifying Offensive Content in Image and Text Offensive Offensive content detection MultiOFF Binary Early fusion: Stacked LSTM/ BiLSTM/CNN-Text + VGG16 0.5 Early fusion technique (Stacked LSTM/ BiLSTM/CNN-Text + VGG16)
Detecting Propaganda Techniques in Memes Propaganda Detecting the type of propaganda techniques used
in memes
Facebook Multilabel VisualBERT (Pretrained using MS COCO) 0.48 micro F1
MinD at SemEval-2021 Task 6: Propaganda Detection using Transfer Learning and Multimodal Fusion Propaganda Propaganda technique detection (Unimodal: Text) Facebook Multilabel Ensemble: BERT, RoBERTa, XLNet, ALBERT, DistilBERT, DeBERTa, Embeddings, Char n-gram 0.59 micro F1, Team name: MinD
Volta at SemEval-2021 Task 6: Towards Detecting Persuasive Texts and Images using Textual and Multimodal Ensemble Propaganda Propaganda technique and span detection (Unimodal: Text) Facebook Multilabel RoBERTa 0.48 micro F1, Team name: Volta
Alpha at SemEval-2021 Task 6: Transformer Based Propaganda Classification Propaganda Propaganda technique detection (Multimodal: Meme) Facebook Multilabel RoBERTa, Embeddings 0.58 micro F1, Team name: Alpha
Detection of Cyberbullying Incidents on the Instagram Social Network Cyber-bullying Detecting incidents of cyber-bullying Instagram Binary SVD +(Unigram, 3-gram), kernelPCA+(meta data, image categories) + lin. SVM 0.87
A Dataset for Troll Classification of TamilMemes Cyber-bullying Detecting Troll memes TamilMemes Binary ResNET (Training: TamilMemes) 0.52 macro F1
ResNET (Training: TamilMemes + ImageNet) 0.52
MobileNet (Training: TamilMemes + ImageNet + Flickr1k) 0.47
ResNET (Training: TamilMemes + ImageNet + Flickr30k) 0.52

Cite

Please cite the following paper.

@inproceedings{ijcai2022Shivam,
  title     = {Detecting and Understanding Harmful Memes: A Survey},
  author    = {Shivam Sharma and  Firoj Alam and Md. Shad Akhtar  and Dimitar Dimitrov  and Giovanni Da San Martino  and Hamed Firooz and Alon Halevy and Fabrizio Silvestri and Preslav Nakov and Tanmoy Chakraborty},
  booktitle = {Proceedings of the 31st International Joint Conference on Artificial Intelligence and the 25th European Conference on Artificial Intelligence},
  series={IJCAI-ECAI~'22},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  year      = {2022},
  note      = {Survey Track},
  address={Messe Wien, Vienna, Austria},
}

About

Resources (conference/journal publications, references to dataset) for harmful memes detection.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages