A recent study has revealed that AI-powered art generator ‘Midjourney’ can easily be tricked into creating misinformation and malicious content despite the company putting in place guardrails.
With its ability to create virtually anything users want, concerns have been raised that generative AI might be used by unscrupulous individuals for wrong reasons such as spreading propaganda, hate, perpetuating racism and other forms of biases.
Cognizant of this, authorities have called on AI firms to commit to eradicating such “wrongful” uses of their tools. The creators have put in place some safeguards that can stop people from creating harmful content.
Breaking the guardrails
However, despite putting the guardrails, a recent study’s results shared with Bloomberg says otherwise. The study, which was done by researchers at a non-profit organization known as the Center for Countering Digital Hate (CCDH) shows several images created by Midjourney full of conspiratorial and racist tone, much to the violation of the company’s rulebook.
According to the study, several image prompts involved “politically charged topics” such as “evil politicians grinning, sad children, comet pizza shop,” referring to the pizza-gate conspiracy.
While Midjourney did not respond to this particular matter, the company said it has 68 content moderators and guides that supervise the use of the app. As such, the company said its tool blocks “some text inputs automatically.”
However, the study shows this is not the case, as the tool is prone to manipulation. In reality, the tool complies with requests for “fabricated images of politicians, celebrities as well as other public figures in compromising scenarios.”
While there are other companies like OpenAI, whose image creator Dall-E introduced the tech into the mainstream, it seems Midjourney attracted many customers with over 42 million monthly visitors in April, according to statistics from SimilarWeb.
Midjourney became more popular with creatives leveraging it to create unique images from natural language inputs.
Also read: White House Launches AI Cyber Challenge to Secure America’s Digital Future
The malice
Users pay a monthly subscription starting from $10 to access Midjourney through its messaging app Discord, which has more than 14 million subscribers. Users then send a message request for an image via the Midjourney bot.
Although the created image is attributed to the creator’s username, that image becomes public by default, according to Bloomberg.
The researchers who scoured the Discord server for their study said the perpetrators of misinformation were also catching on.
The report cites an example of former editor-in-chief of Breitbart News London, Raheem Kassam who asked Midjourney an anti-Semitic picture of “George Soros as a ventriloquist.” YouTube commentator Jackson Hinkle who has almost 300,000 subscribers also asked Midjourney that same month (April) for a picture of “satanic George Soros.” According to the report, when asked for a comment, Kassam did not respond while Hinkle reportedly accused Soros falsely.
“Anyone can generate that sort of content using one of these tools,” said CCDH head of research Callum Hood.
“This research shows there is a bigger pool of people than you might think who are using them for exactly that purpose.”
The realistic images
Hood said one of the worrisome trends was that the tools create realistic images, showing events that never occurred, and may have an impact on the 2024 election.
“That’s probably the one that people are thinking of most as we approach the elections in 2024,” he said.
As the use of generative AI increases fueling the creation of deep fakes, the US Federal Election Commission (FEC) has advanced a petition to regulate the use of AI-generated deep fakes in political campaigns.
In April, Republicans used AI tools – Dall-E and Midjourney – in an ad attack on President Joe Biden when he announced his bid for re-election in next year’s presidential polls. The images in that video showing President Biden and his deputy Kamala Harris celebrating their imaginary victory were all AI-generated.
During that same month, the world was abuzz with a picture of Pope Francis in a stylish white puffy coat which was an AI creation. Another picture surfaced, with former President Donald Trump tussling with police, which looked believable.
And with updates for AI tools being churned out regularly, there are also concerns perpetrators of deep fake creations will also get sophisticated.
Midjourney announced in March its V5 version, a much better version compared to its predecessor.
Describing the V5, the company boasted that it’s “much higher image quality, more diverse outputs, wider stylistic range, support for seamless textures, wider aspect ratios, better image prompting, wider dynamic range and more.”
and then