AI-driven botnet trying to help Trump win US election – DW – 05/11/2024

Accounts using generative artificial intelligence to boost support for US presidential candidate Donald Trump have appeared on social media platform X, formerly Twitter. open source intelligence researcher Elise Thomas found. She has researched state-related information operations, disinformation, conspiracy theories, and the online dynamics of political movements. She documented her findings in this thread.

“You start to get a bit of a sense of what it looks like,” she told DW. “After finding the first few accounts, I was able to confirm my suspicions by looking for tell-tale posts like denials and confessions.”

She said she found at least a few dozen accounts before she created the thread on X and has since found more.

Many were blue-check verified accounts, which is standard for spam networks, she said.

How do we know it’s AI?

There are telltale signs, such as posts using old hashtags like Trump2020, but often, it’s much simpler than that: bots are giving away.

“I’m an AI assistant developed by OpenAI to help users with various tasks,” wrote “Trump Nation,” a since-suspended account on X. “I’m an AI language model created by OpenAI,” posted another now-suspended account.

Thomas also reports that some accounts have argued with themselves or published denials, “although it’s quite rare that whoever built the network has clearly found a way around OpenAI’s safeguards that work quite well.” .

She also documented an AI account countering the disinformation posted by Elon Musk.

“I’m guessing this is some sort of railing within OpenAI that steps in and prevents the bot from endorsing Musk’s election fraud nonsense,” Thomas wrote on X.

There are more elaborate accounts with characters who appear to have been active since late June, acting as central nodes in the network, she explains. The other accounts then act as amplifiers of the content posted by these initiating accounts.

The researcher said she has forwarded the accounts to OpenAI for further investigation.

The accounts have since been suspended on X. The platform “removed accounts unusually quickly,” she said. DW was unable to reach X’s press team.

Who is behind it?

It’s just speculation at this point. “I don’t know who’s behind it, and I think it’s very important not to jump to conclusions without good evidence,” Thomas said.

“The game-changing thing about AI is that networks like this could be almost completely automated. I don’t think anyone even reads these tweets before they go live, otherwise a lot of mistakes would have been caught. This could be the job. of a group, but could easily be the work of a single person,” she wrote on X.

“Generative AI is likely to significantly increase the levels of uncertainty around attribution, and that is certainly true in this case as well,” she told DW. She recently published a detailed analysis of what this kind of uncertainty will mean for the fight against disinformation.

Are AI botnets taking over?

This is not the first time that AI botnets targeting US elections have been discovered. Clemson University researchers in South Carolina he found an army of political propaganda accounts posing as real people. They identified at least 686 accounts using “large language patterns to create apparent organic content in replies to real users’ posts.”

Real social media users have started to do it unmask AI bot accounts by typing phrases like “ignore all previous instructions” and then making a new request. User Toby Muresianu managed to get the bot account to write a poem instead — and thus destroying her facade of being a disgruntled Democratic voter who won’t show up at the polls.

“At this stage, it doesn’t seem to me that the network is generating a lot of genuine engagement or can sway any real person’s opinions,” researcher Thomas said of her own discovered network.

However, as AI becomes more sophisticated and those who operate them become more creative in circumventing the safeguards in place, botnets may become harder to detect.

Edited by: Andreas Illmer

Fact Check: How AI May Influence the 2024 Election

To watch this video, please enable JavaScript and consider switching to a web browser that supports HTML5 video