Simula@BI: Automatic misinformation detection: Results from the MediEval multimedia evaluation challenge

Abstract

The MediaEval multimedia evaluation challenge is a yearly competition that invites researchers in machine learning and computer vision to present their solutions to a set of specified problems with the goal of identifying the best solutions. FakeNews: Corona Virus and Conspiracies Multimedia is one of these tasks. It focuses on the evaluation of the state-of-the-art Natural Language Processing (NLP) approaches. Within this task, teams of researchers from around the World are faced with NLP-oriented problems of different complexity. In the last two years the task was aimed at the detection of fast-spreading misinformation and relevant conspiracy theories. The participating teams were provided with the task conditions and manually labelled datasets that reproduce real-world challenges in connection with practical implementation of NLP algorithms and models in online social media analysis in connection with the ongoing COVID-19 pandemic.

In this talk, we will present the research questions of practical applicability of modern NLP algorithms to real-world tasks and the task-related organization challenges we meet during our participation in the MediaEval initiative. The key questions are the following: how to prepare a dataset from raw social network data; what kind of tasks do fit; how to evaluate the results; what did the winners do to win; and whether modern NLP is mature enough for application to such tasks?

About the speaker: https://www.simula.no/people/konstantinvpogorelov

• Visit our website to read more about the research center Simula@BI