RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication
Contribución de congreso (ConferenceOutput)
Multimodal Multi-image Fake News Detection
[EN] Recent years have seen a large increase in the amount of false information that is posted online. Fake news are created and propagated in order to deceive users and manipulate opinions and subsequently have a negative impact on the society.
The automatic detection of fake news is very challenging since
some of those news are created in sophisticated ways containing
text or images that have been deliberately modified. Combining
information from different modalities can be very useful for
determining which of the online articles are fake. In this paper,
we propose a multimodal multi-image system that combines
information from different modalities in order to detect fake
news posted online. In particular, our system combines textual,
visual and semantic information. For the textual representation
we use the Bidirectional Encoder Representations from Transformers
(BERT) to better capture the underlying semantic and contextual meaning of the text. For the visual representation we extract image tags from multiple images that the articles contain using the VGG-16 model. The semantic representation refers to the text-image similarity calculated using the cosine similarity between the title and image tags embeddings. Our experimental results on a real world dataset show that combining features from the different modalities is effective for fake news detection. In particular, our multimodal multi-image system significantly outperforms the BERT baseline by 4.19% and SpotFake by 5.39%., Anastasia Giachanou is supported by
the SNSF Early Postdoc Mobility grant under the project
Early Fake News Detection on Social Media, Switzerland
(P2TIP2 181441). Guobiao Zhang is funded by China Scholarship Council (CSC) from the Ministry of Education of
P.R. China. The work of Paolo Rosso is partially funded by
the Spanish MICINN under the research project MISMISFAKEnHATE on Misinformation and Miscommunication in
social media: FAKE news and HATE speech (PGC2018-
096212-B-C31).