News Media Profiling using Large Language Models

Author

Zain Mujahid, Mohamed bin Zayed University of Artificial IntelligenceFollow

Date of Award

4-30-2024

Document Type

Thesis

Degree Name

Master of Science in Natural Language Processing

Department

Natural Language Processing

First Advisor

Prof. Preslav Nakov

Second Advisor

Prof. Shangsong Liang

Abstract

In an age defined by the proliferation of information and the dominance of digital media, understanding the bias and factual reporting levels of news outlets is of utmost importance. This thesis presents a novel methodology tailored for news media profiling, centering on crafting custom prompts to solicit responses from Large Language Models (LLMs) to detect bias and factual reporting levels. Through systematic experimentation and thorough data curation, we explore two distinct approaches. Approach 1 involves crafting tailored prompts for LLM responses by querying LLMs against gold labels from Media Bias/Fact Check for data collection and subsequent processing and structuring for analysis. Meanwhile, Approach 2 incorporates systematic guidelines into Approach 1, akin to that used by professional journalists. We perform extensive experiments utilizing machine learning models such as Logistic Regression, Support Vector Machine, and Transformer-based models to predict bias and factuality levels of news media outlets without relying on specific news articles. Leveraging the inherent characteristics and knowledge encapsulated within LLMs, our methodology enables informed predictions and offers valuable insights into the bias and factual reporting levels of news outlets. Our results suggest that the proposed methodologies hold promise for accurate and efficient detection of bias and factual reporting levels in news media outlets. Through innovative approaches and utilizing LLM capabilities, this research lays the groundwork for future advancements in media analysis and information dissemination.

Comments

Thesis submitted to the Deanship of Graduate and Postdoctoral Studies

In partial fulfilment of the requirements for the M.Sc degree in Science in Natural Language Processing

Advisors:Preslav Nakov , Shangsong Liang

Online access available for MBZUAI patrons

Recommended Citation

Z. Mujahid, "News Media Profiling using Large Language Models,", Apr 2024.