AI "reporters" spread disinformation about Ohio train derailment
AI "reporters" spread disinformation about Ohio train derailment
Conspiratorial claims about chemical contamination can be traced back to "reporters" who aren't real.
The Ohio train derailment and subsequent chemical release has spawned a flurry of conspiratorial claims about the event being akin to “America’s Chernobyl,” with some users saying the accident has poisoned the water supply over a vast area and warning residents up and down the Ohio and Mississippi Rivers that they’re in danger if they drink the water.
On Monday, I pointed out the beginning of what looked like a coordinated campaign on Twitter in which users were spreading conspiratorial claims about the Mississippi River being contaminated — despite its vast distance from the scene of the accident, and in contradiction to experts who say there’s “no chance” the contaminants from the accident scene would cause harm to those along the Mississippi River. All of these users were sharing the exact same map, which was unrelated to the current incident, and none provided a link or a source for their claims about how the contamination would spread, in what concentration, or how it would pose a danger to human and animal health. Generally, repetitive posting of outdated and out-of-context visuals as seen in these tweets is a sign of possible coordinated activity and/or disinformation campaigns.
But the claims in those tweets weren't just thinly sourced — some of them were actually sourced from a website that purports to be a news outlet but is really a disinformation website whose writers are fake people with AI-generated faces. This was first discovered by Twitter user Michael Paulauski, who stumbled upon the website while fact-checking unfounded claims about contamination spreading upstream and affecting a wide swath of the midwest.
Fake news websites and AI-generated “writers"
The website using fake writers with AI-generated pictures is called “Eden Reports”. The site was created on Dec. 27, 2022, and is registered with a Lithuanian-based registrar. A reverse IP search shows that only seventeen other domains are hosted on the same server — including multiple domains that are clearly meant to mimic the URL’s of real news websites, like “cbsnews.tk”. As you can see below, these websites haven’t been set up yet and are essentially just blank templates waiting to be turned into deceptive “fake news” websites.
Eden Reports claims to be “your source for reliable and accurate news coverage in Seattle and beyond,” and says it provides coverage of local issues like “the latest city council meeting,” yet a brief look through the content on the website reveals not a single local news article. Rather, the website features a variety of articles about controversial and/or trending national and international topics including Bill Gates, the State of the Union address, the Nord Stream pipeline, and at least nine articles about Russia’s war in Ukraine, including three published on the same day (Jan. 25). The content is often inflammatory and/or click-bait style, such as one article citing a Russian government spokesperson calling President Joe Biden a “terrorist” and promoting (at least in the headline) the idea that the U.S. might have intentionally sabotaged the pipeline.
Presenting this content under the guise of local news may be a strategy to gain trust among viewers, as research shows that people trust local news more than national news sources, and trust in local news has remained high even as trust in news more generally has plummeted. Hence, disinformation disguised as local news may be more likely to be believed and perceived as legitimate.
Eden Reports describes its writers as a “team of experienced reporters dedicated to bringing you the latest and most important stories from around the region,” yet the reporters pictured on the website are not real people — they’re AI-generated faces, produced through a machine learning process known as generative adversarial networks (GANs). The same technology has been used widely by disinformation actors on social media, and many of the deceptive accounts with GAN profile pictures have presented themselves as journalists, while others have claimed to be employees of fake businesses.
Eden Reports’ website doesn’t appear to get much traction, but its content is aggregated through Newsbreak, which greatly increases the reach of Eden Reports. On Newsbreak, Eden Reports has 16,000 followers.
Polluting the information environment
In the aftermath of the train derailment and subsequent chemical leak in East Palestine, Ohio, Eden Reports has produced numerous stories about the potential fallout, including one about the possibility of chemicals spreading across the east coast and causing widespread health and environmental problems. These articles were among those shared on social media and used as “proof” that the derailment is a disaster on par with Chernobyl. The articles on Eden Reports also contributed to the early rumor that the chemicals would spread upstream and cause harm to animals and humans thousands of miles away, which has been refuted by reputable sources including an AP fact-check.
Another article published by Eden Reports criticized Transportation Secretary Pete Buttigieg for his supposed absence during the crisis — a narrative that can be traced back to Fox News, which intensely criticized Buttigieg for being absent when he took parental leave last fall. In fact, Eden Reports’ article criticizing Buttigieg is a pretty clear ripoff of a Fox News article covering the exact same event.
Eden Reports has also produced a number of articles recently about other train derailments and crashes involving toxic chemicals, as well as an article based on a Tucker Carlson segment about intentional sabotage of the food supply — an article that feeds into a racist sub-narrative surrounding the train derailment (I will have an article up about this shortly). These articles have been shared on social media, where they are contributing to the pollution of the information environment and causing confusion about the actual crisis. Despite claims that the recent cluster of train derailments is a sign of sabotage or other intentional attacks, these incidents are actually surprisingly common and happen all the time — we just usually don’t hear about them because we aren’t looking for news reports about them. It’s not exactly reassuring that train derailments, sometimes involving toxic chemicals, are common in the U.S., but it does refute the emerging conspiracy theories claiming that recent derailments are part of a nefarious plot. It also refutes the idea that the current Transportation Secretary is responsible for America’s neglected rail infrastructure, as the problem started long before the current presidential administration.
When articles about train derailments and similar accidents are presented in episodic frames — a type of news framing that presents each incident in an isolated, disconnected context rather than situating it in the context of trends over time — it can create misperceptions about whether a phenomenon is actually occurring more frequently, and what the cause(s) of the problem really are. Compared to thematic framing, which situates individual news events in their larger context, episodic framing creates an information environment that is more conducive to misinformation and misperceptions. In this case, it may seem like these incidents of train derailments are happening more frequently now, but that’s because we’re just paying more attention to them. This is a type of cognitive bias known as the Baader–Meinhof phenomenon or frequency illusion, which happens when, after noticing something for the first time, we start to notice it more often and believe that it is increasing in frequency.
Taken together, the tactics used by Eden Reports — including writers with AI-generated faces presented as real people, plagiarizing or ripping off content from other news sources, falsely describing itself as a local news source, and publishing misinformation about an ongoing crisis making headlines around the nation — point strongly in the direction of intentional deception, rather than unintentionally misleading content. Indeed, this may represent an early example of an effective use of AI-generated facial images in a widespread influence campaign using the guise of local news during a crisis to disseminate messaging designed to maximize engagement.
With recent advances in artificial intelligence, including the generation of realistic fake human faces and believable text in the style of news articles, it is likely that we will see more websites like Eden Reports, only more sophisticated and harder to detect. In fact, some AI-generated text is so good at mimicking news writing that it can produce fake and misleading news articles that are more believable than those written by humans. This technology has the potential to be used to produce believable and highly influential misinformation campaigns on a massive scale. This is frightening enough on its own, but the idea of such a campaign being deployed during a crisis to mislead and confuse people about matters involving health and safety takes this concern to a whole new level.