Uncovering the essence of diverse media biases from the semantic embedding space

The study addresses the challenge of identifying and quantifying media bias, which shapes public perceptions through selective event coverage and subjective wording (tone, framing, word choice). Traditional approaches are often manual, subjective, and narrow in scope, focusing on single bias types. The authors propose an objective, scalable framework to analyze media bias from two complementary perspectives: macro-level event selection (which events/topics outlets choose to report) and micro-level linguistic bias (attitudes conveyed by word choices and contexts). Grounded in Cognitive Miser and Semantic Differential theories and enabled by embedding models, the research aims to capture varying magnitudes, connections, and dynamics of multiple biases across outlets and topics, providing deeper insight into how media bias emerges and operates.

The paper surveys social science and computational literature on media bias. In social science, research has historically been qualitative or involved manual coding (editorials, endorsements, content analysis), with some quantitative work counting keywords or computing sentiment and similarity. These methods can be labor-intensive, subjective, and limited in semantic depth. In computer science and NLP, efforts often focus on a single bias type or on bias within pretrained language models rather than media bias directly. Prior approaches to media bias have examined partisan and ideological slant via word frequencies or per-article aggregation, which may miss deeper semantics and holistic relations across texts. The authors position their framework as a general, automated, and semantically rich alternative that integrates psychological theories (Semantic Differential, Cognitive Miser) with embedding techniques to capture both event selection (macro) and wording bias (micro).

Datasets: (1) GDELT Mention Table provides global news event mentions across sources and languages. Using GlobalEventID and MentionSourceName, the authors construct a media-event matrix where each entry counts how often a media outlet reported an event. They caution that automated extraction and categorization in GDELT may miss or misinterpret some events. (2) MediaCloud-based corpus: over 1.2 million articles (2016–2021) from 12 mainstream U.S. outlets were collected; URLs were retrieved via API and content crawled. Macro-level (event selection) bias via media embeddings: The media-event matrix A (media × events) is decomposed via truncated SVD (LSA-style) to obtain low-dimensional media embeddings. Outlets reporting similar sets of events (i.e., similar topical focus) are close in this embedding space, reflecting event selection bias. For analysis, embeddings are clustered (e.g., K-means), and similarities between sets of media (e.g., by country) are computed using a Word Mover Distance–based similarity (converted from distance to a 0–1 similarity score via 1 − optimal transport cost). Monthly media-event matrices (e.g., Feb–Apr 2022) enable temporal analysis of cross-country similarity dynamics. Micro-level (wording) bias via word embeddings and Semantic Differential: To capture biases in news language, the authors build per-outlet word embedding models using Word2Vec. They first up-sample each outlet’s corpus to equalize article counts (to avoid dominance in pretraining), aggregate all 12 corpora to pretrain a base model W_base, then fine-tune W_base separately on each outlet’s corpus to obtain outlet-specific models W_i. Bias for a target term x on topic T is defined as the difference between the average cosine similarity of x to a set of positive-pole words P and the average similarity to an opposite-pole set −P (antonymic/opposed semantics per Semantic Differential). Topic word lists (e.g., gender: male/female terms; income: rich/poor; political: Republican/Democrat associations by state) are from prior work, search engines, and author judgment. To ensure robustness to up-sampling randomness, experiments are repeated five times with different seeds; results are averaged, and two-tailed t-tests confirm no significant differences across repeats. Implementation/analysis details: For macro analysis, 247 outlets from 8 countries (US, UK, CA, AU, IE, NZ, IN, CN) were embedded and clustered (K=10). For cross-country temporal similarity, media from 25 countries were compared over Feb–Apr 2022 using WMD-based set similarity. For micro analysis, three topics were examined across 12 U.S. outlets: (a) gender bias by occupation (8 occupations), (b) income bias by race/ethnicity (Asian, African, Hispanic, Latino), (c) political bias by U.S. state (top ‘red’ and ‘blue’ states by CPVI).

• Macro/event-selection bias is regional and organizational:
  • Media from the same country cluster together in the embedding space, indicating similar event selection. AP and Reuters cluster together despite geography, consistent with international coverage overlap.
  • Within organizations, outlets show similar selection: 16 of 21 Fox-affiliated outlets form a distinct cluster.
• Temporal convergence driven by international events:
  • Country-level similarity to Ukrainian media peaked in March 2022, aligning with escalation of the Russia–Ukraine conflict. Russia had highest similarity to Ukraine in March; Poland in April, reflecting wartime focus and refugee flows. Media from Egypt and China showed lower similarity to others, possibly reflecting language and national-image objectives.
• Micro/wording biases across topics (12 U.S. outlets, 2016–2021):
  • Gender by occupation: All outlets associate “teacher” and “nurse” with women; most associate “police,” “driver,” “lawyer,” and “scientist” with men; “director” and “photographer” show weaker male-leaning bias. These patterns align with U.S. Bureau of Labor Statistics gender distributions.
  • Income by race/ethnicity: Most outlets associate Asians with “rich” (ESPN is an outlier). Biases toward Hispanic and Latino are similar, both richer than African Americans in most outlets. This roughly matches long-term U.S. income statistics (Asians highest, Hispanics next, African Americans lowest).
  • Political by U.S. state: Most red states lean Republican and most blue states lean Democrat in coverage, but some blue states (e.g., Hawaii, Maryland) lean Republican in the embeddings. The authors attribute this to corpus timeframe (2016–2021) dominated by Republican administration coverage under President Trump.
• Robustness: For gender, income, and political topics, repeated experiments with five random seeds showed no significant differences (two-tailed t-tests).

The framework directly addresses the challenge of scalable, objective media bias analysis by modeling two complementary dimensions: macro-level event selection and micro-level wording bias. Embedding representations provide a holistic, data-driven way to capture similarities in outlets’ topical focus and linguistic associations that shape audience perceptions (consistent with Cognitive Miser theory). Integrating Semantic Differential offers interpretable bias scales based on opposed semantic poles. Findings validate that media selection is strongly shaped by region and organizational affiliation, and that international crises can align coverage across countries. Micro-level analyses reveal that media language often mirrors and may reinforce existing social patterns (e.g., gender distributions by occupation; income stratification by race/ethnicity) and political alignments, while also reflecting temporal political context. Together, these insights demonstrate how the approach can illuminate where and how biases emerge, helping researchers and the public triangulate coverage across outlets and time to mitigate partial perspectives.

The paper introduces a unified, generalizable framework that uncovers media bias at scale using embedding techniques informed by psychological theory. By combining truncated SVD–based media embeddings for event selection with Word2Vec-based Semantic Differential measures for wording bias, the approach captures both macro and micro dimensions across millions of events and over a million articles. Empirical analyses show regional and organizational clustering in event selection, convergence during major international events, and consistent topic-specific biases in line with external statistics. The framework can be extended to more outlets, languages, time periods, and bias types. Future work includes enhancing interpretability of media embeddings to map dimensions to explicit topics, expanding to more complex semantic relations beyond simple antonymic scales, and applying the method to broader domains and real-time monitoring.

• Interpretability of media embeddings: Continuous vectors are effective but not inherently explainable; mapping dimensions to explicit topics remains challenging.
• Relativity of selection bias: Without an absolute ground truth of ‘all events that should be covered,’ event selection bias is best understood as relative topic coverage.
• Semantic scale constraints: For complex topics not well captured by antonymic poles, Semantic Differential-based scales may be insufficient and need extension.
• Data caveats: GDELT’s automated extraction and categorization may miss or misclassify events; MediaCloud corpora (2016–2021) reflect a specific political period (largely under a Republican administration), which may influence political bias findings.