The FIAT emblem showing blurred circles
Atlantic Council Logo Digital Forensic Research Lab Logo

Interference 2020

Foreign Interference Attribution Tracker

A Project of the Digital Forensic Research Lab (DFRLab) of the Atlantic Council

The DFRLab's Foreign Interference Attribution Tracker (FIAT) is an interactive, open-source database that captures allegations of foreign interference relevant to the 2020 election. This tool assesses the credibility, bias, evidence, transparency, and impact of each claim. Explore by scrolling through the timeline and map below. Hover over a circle to see details about a particular case.

Case Selection

In order to be included, cases must fulfill three criteria.

First, cases must involve allegations of foreign interference by primarily digital means. The Australian Government Department of Home Affairs defines foreign interference as activity by a foreign actor that is "coercive, corrupting, deceptive, or clandestine" in nature, distinguishing it from the more benign phenomenon of foreign influence. By focusing expressly on digital activity, this definition denotes a range of interference activities—including disinformation, media manipulation, and cyber intrusion—that are conducted by foreign actors to affect political outcomes.

Second, cases must be novel. A novel case is one which involves a fresh foreign interference claim or which reveals new evidence to reinvigorate an old one. A novel case is also one in which significant newsworthiness is attached to the individual or organization making the claim. In general, a president or ex-president’s claim is novel regardless of the evidence presented. Meanwhile, an op-ed or report by a mid-level U.S. official is only novel if it contains previously undisclosed information.

Third and finally, cases must be relevant to the 2020 U.S. election. This focuses case selection on alleged foreign interference that seems intended to influence voting behaviors, denigrate particular candidates, or engage in political or social issues of direct relevance to the election. It also bounds case selection to foreign interference claims that have occurred around or following the 2018 U.S. midterm elections.

Attribution Impact

Attribution Impact measures the spread of case-related articles and content over the seven days following a foreign interference allegation. It is a sum of the cumulative Facebook engagements, Twitter shares, and Reddit engagements of relevant articles, intended to capture the digital impact of a given case. It is best used as a comparative, rather than absolute, measure.

This sum is computed by a standardized query process that utilizes SerpApi (which enables "Google dorks" using advanced Google search operators), BuzzSumo (which enables the query of a large database of social media content and collection of engagement data), and CrowdTangle (which enables the collection of engagement data for content not indexed by BuzzSumo).

The following steps are followed in order to determine the Attribution Impact. This process is automated by Python and run by means of a Google Collab notebook:

  1. A case's Link of Attribution is used by BuzzSumo to compute a list of backlinks, which includes social media engagement data.
  2. To account for content not indexed by BuzzSumo, SerpAPI is used. The Google dork is standardized by way of several variables: Source, Disinformation, Disinformant Nation, and Date of Attribution. The standard Google dork is “intext:Source AND Disinformant AND Disinformant Nation.” (In some cases, alternate names are substituted. These substitutions are documented in the raw dataset.) The resulting Google dork is reviewed by coders and irrelevant results are deleted.
  3. The edited Google dork is automatically reconciled with the list of BuzzSumo backlinks.
  4. An additional BuzzSumo search is performed on remaining web links, importing social media engagement data where matches are found.
  5. Where matches are not found, web links are visited manually and their social media engagement data is recorded via the CrowdTangle link checker extension for Google Chrome.
  6. Social media engagement data is tallied from these three sources—BuzzSumo backlinks, the BuzzSumo search of web links, and manual CrowdTangle input—to arrive at the final Attribution Impact.

The fixed timeframe, standardized Google dork formula, and opacity of the tools themselves means that relevant social media engagement data may be excluded. Therefore, the Attribution Impact should be treated as a rough estimate, most useful for comparing between cases.

The political leanings of 600+ media entities – using political data from the AllSides Media Bias Ratings – was applied to the web links found in Attribution Impact. Since not all web links had matching AllSides data, the polarization data filter shows only cases where there were ten or more AllSides-matching articles and/or cases where 25 percent or more of articles matched with AllSides data. The stem lines on the polarization data filter are colored based on the mean political polarization of that case.

It is the intention of the FIAT team to introduce additional dimensions of Attribution Impact as the tool evolves.

Attribution Score

The Attribution Score is a framework of eighteen binary statements (true or false) that assess foreign interference claims made by governments, technology companies, the media, and civil society organizations. The measure is intended to capture the reliability of the attribution as discernible through public sources rather than to serve as a fact-check of the attribution itself. If a statement is deemed applicable, a point is awarded. If a statement is deemed inapplicable or irrelevant, no point is awarded. Initial coding was reviewed by the FIAT research team and shaped by iterative discussion.

This scoring system is based on the experience of DFRLab experts in assessing—and making—such allegations. It is also based on a review of work produced by the wider disinformation studies community, and particularly resources compiled by attribution.news.

The Attribution Score is composed of four subsections:

Credibility
  • The source of the attribution does not have a direct financial interest in a certain attribution outcome.
  • The source of the attribution has a diversified and transparent funding stream.
  • The source of the attribution does not strongly endorse a specific political ideology.
  • The source of the attribution is in no way affiliated with a political campaign.
  • The source of the attribution has not previously promoted mis- or disinformation.
Objectivity
  • The attribution avoids using biased wording. The attribution avoids high-inference or emotive language.
  • The headline accurately conveys the content of the attribution.
  • The attribution clearly distinguishes factual information from argumentative analysis.
Evidence
  • The attribution provides a clear illustration of the methods, tactics, and platforms involved in the alleged information operation.
  • The attribution contextualizes the engagement with, and impact of, the alleged information operation.
  • The attribution identifies actors and states allegedly responsible.
  • The attribution clearly explains the strategic goal and rationale of the actors who conducted the alleged information operation.
  • The attribution relies on information which is unique to, or can only be procured by, the relevant actor. (e.g. classified information for U.S. federal agencies, back-end/developer information for technology companies)
Transparency
  • The attribution provides open access to a dataset or archived links of alleged assets.
  • The attribution methodology is clearly explained.
  • The attribution is replicable through open-source evidence.
  • The attribution acknowledges relevant limitations or mitigating factors in its assessment.
  • The attribution has been corroborated by a third party or independent investigation.

An attribution that scores fifteen points or higher is especially reliable. An attribution that scores six points or fewer should be carefully scrutinized.

Allegations of foreign interference in U.S. elections that met the case selection criteria were recorded by DFRLab coders using a codebook of variables. Five text variables, 36 binary variable options, and 5 other variables were used to describe who made the allegation of interference against who, what the attribution was, when it occurred, the platforms where it occurred, and how the interference was conducted. New binary variables for platforms and methods were added based on case review.

Additional variables were used to measure the impact of the allegation on online media discourse, and to assess the credibility, bias, evidence, and transparency of allegations.

Who is making the attribution, against whom?
  • Source of Allegation (free text). The original source of the interference allegation.
  • Source Nation (country). The country where the source of the interference allegation originates. Since the scope of this dataset is interference in the U.S., the most common source nation for allegations is the United States. The source nation does not necessarily denote the actor was associated with a national government.
  • Source Category (binary, select all that apply).
    • Civil Society Organization. A nonprofit, non-governmental, non-media entity, typically a university or think tank.
    • Foreign Government Body. A non-U.S. government entity.
    • Government. Government agencies, elected representatives, and officials, even if quoted anonymously.
    • Influential Individual. A noteworthy individual, not currently affiliated with another category, who is deemed nationally recognizable or operating in the public sphere.
    • Media. Only applies if a news organization makes the allegation on the basis of its own investigation. A media organization reporting on an allegation made by someone else (e.g. an anonymous government official) is not included.
    • Private Consultancy. A company engaged in private monitoring and risk consulting, typically in the field of cybersecurity.
    • Technology Company. A company that operates a social media platform or offers a technology service.
  • Disinformant (free text). Brief description of the actor purportedly responsible for the interference attempt.
  • Disinformant Nation (free text). The country where the interference originates, according to the source. When an allegation comes from a non-state political actor, this field is the nation of origin of that non-state political actor. This does not necessarily denote an actor is associated with the national government.
  • Attribution Type (binary, select all that apply).
    • Direct Attribution. The source directly accuses the disinformant of malicious political behavior.
    • Proxy/Inferred Attribution. The source does not make a direct attribution, but clearly states that the activity is likely associated with the disinformant or strongly implies the accusation is directed at the disinformant.
    • Non-Aligned Commercial Activity. The interference consists of malicious commercial activity rather than a politically motivated information operation.
  • Short title (free text).
  • Short description (free text). One to three sentence description of the allegation, alleged activity, and attribution.
  • Link to attribution (link).
  • If the attribution is to Facebook and was removed by Facebook for coordinated inauthentic behavior, additional data is available. These variables are an updated version of Sima Basel and Matt Suiche’s OSINT analysis of Facebook removals for coordinated inauthentic behavior. See Basel’s data compilation for variables and definitions.
When did the interference and attribution occur?
  • Date(s) of Activity. Date or range of purported activity..
    • Start (date). Input if start date is known; if not, omit.
    • End (date). Input if end date is known, if not, omit.
  • Date of Attribution (date). Date corresponds to date of link of attribution. Particular attention should be given to this date by coders because it is used to determine the time span for collecting Attribution Impact metrics.
On what platforms did the interference purportedly take place?
  • Open Web (binary, select all that apply).
    • State Media. A media outlet controlled by a government or government proxy, which is not editorially independent.
    • Independent Media. Media outlets that are generally regarded as reputable, balanced, and independent of direct government control.
    • "Junk News" Media. Unreliable, skewed, openly propagandistic, or fringe media outlets that lack discernable government ties.
  • Social Media Platform (binary, select all that apply). Platform(s) on which alleged interference occurred.
    • Facebook
    • Instagram
    • Twitter
    • YouTube
    • LinkedIn
    • Reddit
    • VK
    • Forum Board (binary)
  • Messaging Platforms. (binary, select all that apply). Platform(s) on which alleged interference occurred.
    • WhatsApp
    • Telegram
    • Signal
    • WeChat
    • SMS
  • Advertisement (binary).
  • Email (binary).
How was the interference purportedly conducted?
  • Method. Methods used in both the creation and the amplification of content related to the alleged foreign interference (binary, select all that apply).
    • Brigading. Authentic social media accounts but evidence of coordinated amplification or harassment.
    • Sockpuppets. Inauthentic social media accounts; evidence suggests a high likelihood of human operation.
    • Botnets. Inauthentic social media accounts; evidence suggests a high likelihood of automation.
    • Search Engine Manipulation. Manipulation of search queries and results; typosquatting.
    • Hacking - DDoS. Distributed denial-of-service attack; malicious attempt to disrupt server traffic.
    • Hacking - Data Exfiltration. Unauthorized movement of data; spearphishing; hack-and-release.
    • Deceptive Content Manipulation. Deceptively edited content; deceptive co-option of existing brands; does not include use of deep learning processes.
    • Deep Learning Processes. Augmented or fabricated content produced using deep learning processes; "deep fakes"; textual generation. Sometimes referred to as "synthetic media," although this term does not adequately distinguish between the use of deep learning and use of more basic manipulative techniques.
What was the impact of the allegation on online media discourse?
  • Attribution Impact. Measures the spread of case-related articles and content over the seven days following a foreign interference allegation. It is a sum of Facebook engagements, Twitter shares, and Reddit engagements. Methodology is described above to acquire online articles about cases and social media engagement of articles; social media data is defined and sourced by Buzzsumo and Crowdtangle.
    • Article count (quantitative). The number of web links about the allegation in the week after the date of attribution of the case.
    • Facebook engagement (quantitative). Aggregate shares, comments, and reactions of a web link.
    • Twitter engagement (quantitative). Aggregate shares of a web link.
    • Reddit engagement (quantitative). Aggregate shares and interactions of a web link.
    • Total engagement (quantitative). Aggregate Facebook, Twitter, and Reddit engagement.
    • Polarization (quantitative). Measures the political leaning of case-related articles using polarization data from the AllSides Media Bias Ratings. AllSides is a media technology company which provides multiple perspectives on news outlets. Articles from outlets with AllSides data were divided into five political categories: left, lean left, center, lean right, and right.
How credible, biased, legitimate, and transparent is the allegation?
  • Attribution Score. Methodology is described above; the goal of this score is to critically assess the validity of the allegation from multiple perspectives.
    • Credibility
    • Bias
    • Evidence
    • Transparency

About This Project

The core FIAT research team is composed of Emerson T. Brooking (lead), Alyssa Kann, Max Rizzuto, Jacqueline Malaret, and Helen Simpson.

The tool was designed by Matthias Stahl (higsch | data & design).

This project was directed by Graham Brookie and Emerson T. Brooking.

Invaluable counsel and coordination was provided by Nicholas Yap, Kelsey Henquinet, Andy Carvin, Anna Pellegatta, Devin Chavira, Eric Baker, and Zarine Kharazian.

This project is made possible thanks to the generosity and commitment of Craig Newmark Philanthropies.

About the DFRLab

The Atlantic Council's Digital Forensic Research Lab—whose work resides at the unique intersection between government, media, and technology—aims to establish accountability and transparency online as a means to secure democracy and restore trust in public discourse.

DFRLab is spread across five continents, and its work is rooted among three pillars:

About the Atlantic Council

The Atlantic Council promotes constructive leadership and engagement in international affairs based on the Atlantic Community’s central role in meeting global challenges. The Council provides an essential forum for navigating the dramatic economic and political changes defining the twenty-first century by informing and galvanizing its uniquely influential network of global leaders. The Atlantic Council—through the papers it publishes, the ideas it generates, the future leaders it develops, and the communities it builds—shapes policy choices and strategies to create a more free, secure, and prosperous world.