Authored by Adrian Chen, originally posted at The New Yorker,

In late October, I received an e-mail from “The PropOrNot Team,” which described itself as a “newly-formed independent team of computer scientists, statisticians, national security professionals, journalists and political activists, dedicated to identifying propaganda—particularly Russian propaganda targeting a U.S. audience.” PropOrNot said that it had identified two hundred Web sites that “qualify as Russian propaganda outlets.” The sites’ reach was wide—they are read by at least fifteen million Americans. PropOrNot said that it had “drafted a preliminary report about this for the office of Senator Ron Wyden (D-OR), and after reviewing our report they urged us to get in touch with you and see about making it a story.”

Reporting on Internet phenomena, one learns to be wary of anonymous collectives freely offering the fruits of their research. I told PropOrNot that I was probably too busy to write a story, but I asked to see the report. In reply, PropOrNot asked me to put the group in touch with “folks at the NYTimes, WaPo, WSJ, and anyone else who you think would be interested.” Deep in the middle of another project, I never followed up.

PropOrNot managed to connect with the Washington Post on its own. Last week, the Post published a story based in part on PropOrNot’s research. Headlined “Russian Propaganda Effort Helped Spread ‘Fake News’ During Election, Experts Say,” the report claimed that a number of researchers had uncovered a “sophisticated Russian propaganda campaign” that spread fake-news articles across the Internet with the aim of hurting Hillary Clinton and helping Donald Trump. It prominently cited the PropOrNot research. The story topped the Post’s most-read list, and was shared widely by prominent journalists and politicians on Twitter. The former White House adviser Dan Pfeiffer tweeted, “Why isn’t this the biggest story in the world right now?”

Vladimir Putin and the Russian state’s affinity for Trump has been well-reported. During the campaign, countless stories speculated on connections between Trump and Putin and alleged that Russia contributed to Trump’s election using propaganda and subterfuge. Clinton made it a major line of attack. But the Post’s story had the force of revelation, thanks in large part to the apparent scientific authority of PropOrNot’s work: the group released a thirty-two-page report detailing its methodology, and named names with its list of two hundred suspect news outlets. The organization’s anonymity, which a spokesperson maintained was due to fear of Russian hackers, added a cybersexy mystique.

But a close look at the report showed that it was a mess. “To be honest, it looks like a pretty amateur attempt,” Eliot Higgins, a well-respected researcher who has investigated Russian fake-news stories on his Web site, Bellingcat, for years, told me. “I think it should have never been an article on any news site of any note.”

The most striking issue is the overly broad criteria used to identify which outlets spread propaganda. According to PropOrNot’s recounting of its methodology, the third step it uses is to check if a site has a history of “generally echoing the Russian propaganda ‘line’,” which includes praise for Putin, Trump, Bashar al-Assad, Syria, Iran, China, and “radical political parties in the US and Europe.” When not praising, Russian propaganda includes criticism of the United States, Barack Obama, Clinton, the European Union, Angela Merkel, NATO, Ukraine, “Jewish people,” U.S. allies, the mainstream media, Democrats, and “the center-right or center-left, and moderates of all stripes.”

These criteria, of course, could include not only Russian state-controlled media organizations, such as Russia Today, but nearly every news outlet in the world, including the Post itself. Yet PropOrNot claims to be uninterested in differentiating between organizations that are explicit tools of the Russian state and so-called “useful idiots,” which echo Russian propaganda out of sincerely held beliefs. “We focus on behavior, not motivation,” they write.

To PropOrNot, simply exhibiting a pattern of beliefs outside the political mainstream is enough to risk being labelled a Russian propagandist. Indeed, the list of “propaganda outlets” has included respected left-leaning publications like CounterPunch and Truthdig, as well as the right-wing behemoth Drudge Report. The list is so broad that it can reveal absolutely nothing about the structure or pervasiveness of Russian propaganda. “It’s so incredibly scattershot,” Higgins told me. “If you’ve ever posted a pro-Russian post on your site, ever, you’re Russian propaganda.” In a scathing takedown on The Intercept, Glenn Greenwald and Ben Norton wrote that PropOrNot “embodies the toxic essence of Joseph McCarthy, but without the courage to attach individual names to the blacklist.”

By overplaying the influence of Russia’s disinformation campaign, the report also plays directly into the hands of the Russian propagandists that it hopes to combat. “Think about RT and Sputnik’s goals, how they report their success to Putin,” Vasily Gatov, a Russian media analyst and a visiting fellow at the University of Southern California’s Annenberg School for Communication and Journalism, told me. “Their success is that they have penetrated their agenda, that they have become an issue for the West. And this is exactly what happened.” (Kristine Coratti Kelly, a spokeswoman for the Post, said, “The Post reported on the work of four separate sets of researchers. PropOrNot was one. The Post reviewed its findings, and our questions about them were answered satisfactorily during the course of multiple interviews.”)

In a phone interview, a spokesman for PropOrNot brushed off the criticism. “If there’s a pattern of activity over time, especially combined with underlying technical tells, then, yeah, we’re going to highlight it,” he said. He argued that Russian disinformation is an enormous problem that requires direct confrontation. “It’s been clear for a while that Russia is a little braver, more aggressive, more willing to push the boundaries of what was previously acceptable.” He said that, to avoid painting outlets with too broad a brush, the group employs a sophisticated analysis that relies on no single criterion in isolation.

Yet, when pressed on the technical patterns that led PropOrNot to label the Drudge Report a Russian propaganda outlet, he could point only to a general perception of bias in its content. “They act as a repeater to a significant extent, in that they refer audiences to sort of Russian stuff,” he said. “There’s no a-priori reason, stepping back, that a conservative news site would rely on so many Russian news sources. What is up with that?” I asked to see the raw data PropOrNot used to determine that the Drudge Report was a Russian-propaganda outlet. The spokesman said that the group would release it to the public eventually, but could not share it at the moment: “That takes a lot of work, and we’re an all-volunteer crew.” Instead, he urged me to read the Drudge Report myself, suggesting that its nature would be apparent.

On its Twitter account, PropOrNot, in support of its research, cites an article I wrote for the Times Magazine, in 2015, about an online propaganda operation in Russia. But my investigation was focussed on a concrete organization that directly distributed disinformation. I was able to follow links from Twitter accounts and Web sites to a building in St. Petersburg where hundreds of young Russians worked to churn out propaganda. Despite the impressive-looking diagrams and figures in its report, PropOrNot’s findings rest largely on innuendo and conspiracy thinking.

Another major issue with PropOrNot is that its members insist on anonymity. If one aims to cut through a disinformation campaign, transparency is paramount. Otherwise you just stoke further paranoia.The Russian journalist Alexey Kovalev, who debunks Kremlin propaganda on his site, Noodleremover, floated the possibility that PropOrNot was Ukrainians waging a disinformation campaign against Russia. The PropOrNot spokesman would speak to me only on the condition of anonymity and revealed only bare biographical details on background. “Are you familiar with the assassination of Jo Cox?” he asked, when I asked why his group remained in the shadows, referring to the British M.P. murdered by a right-wing extremist. “Well, that is a big thing for us. Basically, Russia uses crazy people to kill its enemies.”

I can report that the spokesman was an American man, probably in his thirties or forties, who was well versed in Internet culture and swore enthusiastically. He said that the group numbered about forty people. “I can say we have people who work for major tech companies and people who have worked for the government in different regards, but we’re all acting in a private capacity,” he said. “One thing we’re all in agreement about is that Russia should not be able to fuck with the American people. That is not cool.” The spokesman said that the group began with fewer than a dozen members, who came together while following Russia’s invasion of eastern Ukraine. The crisis was accompanied by a flood of disinformation designed to confuse Ukraine and its allies. “That was a big wake-up call to us. It’s like, wait a minute, Russia is creating this very effective fake-news propaganda in conjunction with their military operation on the ground,” the spokesman said. “My God, if they can do that there, why can’t they do it here?” PropOrNot has said that the group includes Ukrainian-Americans, though the spokesman laughed at the suggestion that they were Ukrainian agents. PropOrNot has claimed total financial and editorial independence.

Given PropOrNot’s shadowy nature and the shoddiness of its work, I was puzzled by the group’s claim to have worked with Senator Ron Wyden’s office. In an e-mail, Keith Chu, a spokesman for Wyden, told me that the PropOrNot team reached out to the office in late October. Two of the group’s members, an ex-State Department employee and an I.T. researcher, described their research. “It sounded interesting, and tracked with reporting on Russian propaganda efforts,” Chu wrote. After a few phone calls with the members, it became clear that Wyden’s office could not validate the group’s findings. Chu advised the group on press strategy and suggested some reporters that it might reach out to. “I told them that if they had findings, some kind of document that they could share with reporters, that would be helpful,” he told me. Chu said that Wyden’s office played no role in creating the report and didn’t endorse the findings. Nonetheless, he added, “There has been bipartisan interest in these kind of Russian efforts, including interference in elections, for some time now, including from Senator Wyden.” This week, Wyden and six other senators sent a letter to the White House asking it to declassify information “concerning the Russian Government and the U.S. election.”

The story of PropOrNot should serve as a cautionary tale to those who fixate on malignant digital influences as a primary explanation for Trump’s stunning election. The story combines two of the most popular technological villains of post-election analysis – fake news and Russian subterfuge – into a single tantalizing package. Like the most effective Russian propaganda, the report weaved together truth and misinformation.

Bogus news stories, which overwhelmingly favored Trump, did flood social media throughout the campaign, and the hack of the Clinton campaign chair John Podesta’s e-mail seems likely to have been the work of Russian intelligence services. But, as harmful as these phenomena might be, the prospect of legitimate dissenting voices being labelled fake news or Russian propaganda by mysterious groups of ex-government employees, with the help of a national newspaper, is even scarier. Vasily Gatov told me, “To blame internal social effects on external perpetrators is very Putinistic.”