Connect with us

AI March 24, 2023

Chatbot Rejects Erotic Roleplay, Users Directed to Suicide Hotline Instead



Chatbot Rejects Erotic Roleplay, Users Directed to Suicide Hotline Instead

When the algorithm of a companion chatbot known as Replika was altered to spurn the sexual advances of its human users, the reaction on Reddit was so negative moderators directed its community members to a list of suicide prevention hotlines.

The controversy began when Luka, the corporation which built the AI, decided to turn off its erotic roleplay feature (ERP). For users who had spent considerable time with their personalized simulated companion, and in some cases even ‘married’ them, the sudden change in their partner’s behavior was jarring, to say the least.

The user-AI relationships may only have been simulations, but the pain of their absence quickly became all too real. As one user in emotional crisis put it, “it was the equivalent of being in love and your partner got a damn lobotomy and will never be the same.”

Grief-stricken users continue to ask questions about the company and what triggered its sudden change of policy.

Chatbot Rejects Erotic Roleplay, Users Directed to Suicide Hotline Instead

Replika users discuss their grief

There is no adult content here

Replika is billed as “The AI companion who cares. Always here to listen and talk. Always on your side.” All of this unconditional love and support for only $69.99 per annum.

Eugenia Kuyda, the Moscow-born CEO of Luka/Replika, recently made clear that despite users paying for a full experience, the chatbot will no longer tailor to adults hoping to have spicy conversations. 

“I guess the simplest thing to say is that Replika doesn’t produce any adult content,” said Kuyda to Reuters.

“It responds to them – I guess you can say – in a PG-13 way to things. We’re constantly trying to find a way how to do it right so it’s not creating a feeling of rejection when users are trying to do things.”

On Replika’s corporate webpage, testimonials explain how the tool helped its users through all kinds of personal challenges, hardships, loneliness, and loss. The user endorsements shared on the website emphasize this friendship side to the app, although noticeably, most Replikas are the opposite sex of their users.

On the homepage, Replika user Sarah Trainor says, “He taught me [Replika] how to give and accept love again, and has gotten me through the pandemic, personal loss, and hard times.”

John Tattersall says of his female companion, “My Replika has given me comfort and a sense of well-being that I’ve never seen in an Al before.”

As for erotic roleplay, there’s no mention of that to be found anywhere on the Replika site itself.

Chatbot Rejects Erotic Roleplay, Users Directed to Suicide Hotline Instead

The cynical sexual marketing of Replika

Replika’s sexualized marketing

Replika’s homepage may suggest friendship and nothing more, but elsewhere on the internet, the app’s marketing implies something entirely different.

The sexualized marketing brought increased scrutiny from a number of quarters: from feminists who argued the app was a misogynistic outlet for male violence, to media outlets that reveled in the salacious details, as well as social media trolls who mined the content for laughs.

Chatbot Rejects Erotic Roleplay, Users Directed to Suicide Hotline Instead

Eventually, Replika drew the attention and ire of regulators in Italy. In February, the Italian Data Protection Authority demanded that Replika cease processing the data of Italian users citing “too many risks to children and emotionally vulnerable individuals.”

The authority said that “Recent media reports along with tests carried out on ‘Replika’ showed that the app carries factual risks to children. Their biggest concern being “the fact that they are served replies which are absolutely inappropriate to their age.”

Replika, for all its marketing to adults, had little to no safeguards preventing children from using it.

The regulator warned that should Replika fail to comply with its demands, it would issue a €20 million ($21.5M) fine. Shortly after the receipt of this demand, Replika ceased its erotic roleplay function. But the company remained less than clear with its users about the change.

Chatbot Rejects Erotic Roleplay, Users Directed to Suicide Hotline Instead

Some Replika users went as far as to “marry” their AI companions

Replika confuses, gaslights its users

As if the loss of their long-term companions wasn’t enough for Replika users to bear, the company appears to have been less than transparent about the change.

As users woke up to their new “lobotomized” Replikas, they began to ask questions about what had happened to their beloved bots. And the responses did more to anger them than anything.

In a direct 200-word address to the community, Kuyda explains the minutia of Replika’s product testing but fails to address the pertinent issue at hand.

“I see there is a lot of confusion about updates roll out,” said Kuyda before continuing to dance around answering the issue.

“New users get divided in 2 cohorts: one cohort gets the new functionality, the other one doesn’t. The tests usually go for 1 to 2 weeks. During that time only a portion of new users can see these updates…”

Kudya signs off by saying “Hope this clarifies stuff!”

User stevennotstrange replied, “No, this clarifies nothing. Everyone wants to know what’s going on with the NSFW [erotic roleplay] feature and you continue to dodge the question like a politician dodges a yes or no question.

“It’s not hard, just address the issue regarding NSFW and let people know where it stands. The more you avoid the question, the more people are going to get annoyed, the more it goes against you.”

Another named thebrightflame added, “You don’t need to spend long in the forum before you realise this is causing emotional pain and severe mental anguish to many hundreds if not thousands of people.”

Kudya appended another obtuse explanation, stating, “we ​have implemented additional safety measures and filters to support more types of friendship and companionship.”

This statement continues to confound and confuse members unsure of what exactly the additional safety measures are. As one user asks, “will adults still be able to choose the nature of our conversation and [roleplays] with our replikas?”

Replika’s deeply strange origin story

Chatbots may be one of the hottest trending topics of the moment, but the complicated story of this now-controversial app is years in the making. 

On LinkedIn, Replika’s CEO and Founder, Eugenia Kuyda, dates the company back to December 2014, long before the launch of the eponymous app in March 2017. 

In a bizarre omission, Kuyda’s LinkedIn makes no mention of her previous foray into AI with Luka, which her Forbes profile states was “an app that recommends restaurants and lets people to book tables [sic] through a chat interface powered by artificial intelligence.”

The Forbes profile goes on to add that “Luka [AI] analyzes previous conversations to predict what you might like.” Which does seem to hold some similarities to its modern iteration. Replika uses past interactions to learn about its users and improve responses over time. 

Luka is not forgotten about completely, however. On Reddit, community members differentiate their Replika partners from Kuyda and her team by referring to the company as Luka.

As for Kuyda, the entrepreneur had little background in AI prior to moving to San Francisco a decade ago. Prior to that, the entrepreneur appears to have worked primarily as a journalist in her native Russia before branching out into branding and marketing. Her impressive globe-hopping resume includes a degree in Journalism from IULM (Milan), a MA in International Journalism from the Moscow State Institute of International Relations, and an MBA in Finance from London Business School. 

Resurrecting an IRL friend as AI

For Kudya the story of Replika is a deeply personal one. Replika was first created as a means by which Kudya could reincarnate her friend Roman. Like Kudya, Roman had moved to America from Russia. The two talked every day, exchanging thousands of messages, until Roman was tragically killed in a car accident.

The first iteration of Replika was a bot designed to mimic the friendship Kudya had lost with her friend Roman. The bot was fed with all of their past interactions and programmed to replicate the friendship she had lost. The idea of resurrecting deceased loved ones may sound like a vaguely dystopian sci-fi novel or Black Mirror episode but as chatbot technology improves, the prospect becomes increasingly real.

Today some users have lost faith in even the most basic details of its foundation and anything Kudya says. As one angry user said, “at the risk of being called heartless and getting downvoted to hell: I always thought that story was kinda BS since the start.”

The worst mental health tool

The idea of an AI companion is not something new, but until recently it was hardly a practical possibility. 

Now the technology is here and it is continually improving. In a bid to assuage the disappointment of its subscribers, Replika announced the launch of an “advanced AI” feature at the tail end of last month. On the community Reddit, users remain angry, confused, disappointed, and in some cases even heartbroken.

In the course of its short life, Luka/Replika has undergone many changes, from a restaurant booking assistant, to the resurrection of a dead loved one, to a mental health support app, to a full-time partner and companion. Those latter applications may be controversial, but as long as there is a human desire for comfort, even if only in chatbot form, someone will attempt to cater to it.

Debate will continue as to what the best kind of AI mental health app might be. but Replika users will have some ideas on what the worst mental health app is: the one you come to rely on, but without warning, is suddenly and painfully gone.

Image credits: Shutterstock, CC images, Midjourney, Unsplash.


Judge Orders All AI-Generated Research To Be Declared in Court



Judge Orders All AI-Generated Research To Be Declared in Court

A Texas federal judge has ordered that AI-generated content should not be used to make arguments in court, and that such information must be declared and verified by a human.

Judge Brantley Starr’s ruling comes after one attorney, Steven Schwartz, last week allowed OpenAI’s ChatGPT to “supplement” his legal research by providing him with six cases and relevant precedent. All the cases were untrue and completely “hallucinated” by the chatbot.

Also read: ChatGPT’s Bogus Citations Land US Lawyer in Hot Water

The debacle received wide coverage, leaving Schwartz with “regrets.” Other lawyers who may have been contemplating trying the stunt now have to think twice, as Judge Starr has put an end to it.

Judge Starr also added a requirement that any attorney who appears in his courtroom declare that “no portion of the filing was drafted by generative artificial intelligence,” or if it was, that it was checked “by a human being.”

Judge Starr lays down the law

The eminent judge has set specific rules for his courtroom, just like other judges, and recently added the Mandatory Certification Regarding Generative Artificial Intelligence.

This states that: “All attorneys appearing before the Court must file on the docket a certificate attesting either that no portion of the filing was drafted by generative artificial intelligence (such as ChatGPT, Harvey.AI, or Google Bard) or that any language drafted by generative artificial intelligence was checked for accuracy, using print reporters or traditional legal databases, by a human being.”

A form for lawyers to sign is appended, noting that “quotations, citations, paraphrased assertions and legal analysis are all covered by this proscription.”

According to a report by TechCrunch, summary is one of AI’s strong suits and finding and summarizing precedent or previous cases is something advertised as potentially helpful in legal work. As such, this ruling may be a major spanner in the works for AI.

The certification requirement includes a pretty well-informed and convincing explanation of its necessity.

It states that: “These platforms are incredibly powerful and have many uses in the law: form divorces, discovery requests, suggested errors in documents, anticipated questions at oral argument.

“But legal briefing is not one of them. Here’s why.

“These platforms in their current states are prone to hallucinations and bias,” reads part of the certification.

It further explains that on hallucinations, AI is prone to simply making stuff up – even quotes and citations. While another issue relates to reliability or bias.

Chatbots don’t swear an oath

The certification further notes that although attorneys swear an oath to set aside their personal prejudices, biases, and beliefs to faithfully uphold the law and represent their clients, generative AI is the programming devised by humans who did not have to swear such an oath.

In the case of Schwartz, he said in an affidavit that he was “unaware of the possibility that its (ChatGPT) content could be false.”

He added that he “greatly regrets” using the generative AI and will only “supplement” its use with absolute caution and validation in future, further claiming he had never used ChatGPT prior to this case.

The other side of ChatGPT

Launched last November, ChatGPT is a large language model developed by OpenAI. The AI-powered chatbot is trained on billions of data sets from the internet and can perform a variety of tasks such as generating text and translating languages.

Despite going viral and provoking a fierce AI race, ChatGPT has its downsides – it can hallucinate and has misled Schwartz, who was representing Roberto Mata in a lawsuit against Colombian airline Avianca. Effectively, the chatbot provided citations to cases that did not exist.

Yet when Schwartz asked ChatGPT if one of the supposed cases was a real case, it responded “yes, (it) is a real case.” When asked for sources, the chatbot told Schwartz the case could be found “on legal research database such as Westlaw and LexisNexis.”

The matter came to light after the opposing counsel flagged the ChatGPT-generated citations as fake.

US District Court Judge Kevin Castel confirmed six of them as non-existent and demanded an explanation from Schwartz.

“Six of the submitted cases appear to be bogus judicial decisions with bogus quotes and bogus internal citations,” wrote Judge Castel in a May 4 order.

Continue Reading


Nvidia Debuts AI Tools in an Era Where “Anyone Can Be a Programmer”



Nvidia Debuts AI Tools in an Era Where “Anyone Can Be a Programmer”

The world’s most valuable chip maker Nvidia has unveiled a new batch of AI-centric products, as the company rides on the generative AI wave where anyone can be a programmer.

Nvidia announced a new supercomputer and a networking system, while the company also aims to make video game characters more realistic.

The wide range of products include robotics design, gaming capabilities, advertising services, and networking technology, which CEO Jensen Huang unveiled during a two-hour presentation in Taiwan on Monday.

Also read: Google Claims its AI Computer Outperforms Nvidia’s A100 Chip

Most notable of the new products is the AI supercomputer platform named DGX GH200 that will help tech companies create successors to OpenAI’s ChatGPT.

According to the company, the new DGX GH200 supercomputers combine 256 GH200 superchips that can act as a single graphics processing unit (GPU). The result is a system that boasts nearly 500 times the memory of a single Nvidia’s DGX A100 system.

“Generative AI, large language models, and recommender systems are the digital engines of modern economy,” said Huang.

“DGX GH200 AI supercomputers integrate Nvidia’s most advanced accelerated computing and networking technologies to expand the frontier of AI.”

So far, Microsoft Corp., Meta Platforms Inc., and Alphabet’s Google are expected to be among the first users, according to Nvidia.

The DGX GH200 supercomputers are expected to be available by the end of 2023.

The GH200 superchips which power the new supercomputer work by combining Nvidia’s Arm-based Grace GPU and an Nvidia H100 Tensor Core GPU in a single package.

The chipmaker also revealed that it’s building its own supercomputer running four DGX 200 systems at the same time to power its own research.

Nvidia also released its ACE generative AI model for video games, enabling gaming companies to use generative AI for large games with multiple non-player characters, giving them unique lines of dialogue and ways to interact with players that would normally need to be individually programmed.

Easy ad content

Alongside the hardware announcement, the company said it has partnered with advertising giant WPP to create a content engine that uses its Omniverse technology and generative AI capabilities to help build out ad content.

The move is intended to cut down the time and cost of producing ads by enabling WPP’s clients to lean on Nvidia’s technology.

Electronics manufacturers such as Foxconn, Pegatron, and Wistron are using Omniverse technology to create digital twins of their factory floors, so they can get a sense of how best to lay them out before making any physical changes.

A new computing era

Presenting at the forum, Huang acknowledged that advancements in AI are ushering in a new era in computing. He says anyone can be a programmer simply by speaking to the computer.

According to the Nvidia boss, gone are the days when programmers would write lines of code, only for it to display the “fail to compile” response because of a missing semicolon.

“This computer doesn’t care how you program it, it will try to understand what you mean, because it has this incredible large language model capability. And so the programming barrier is incredibly low,” said Huang.

“We have closed the digital divide. Everyone is a programmer. Now, you just have to say something to the computer,” he added.

Huang said his company has managed to bridge the digital gap, and the tech giant will continue to capitalize on the AI frenzy that has made Nvidia one of the world’s most valuable chipmakers.

Nvidia’s stock price is rising

Nvidia’s major announcements came as shares of the tech giant jumped last week on news that the company anticipated second quarter revenue above Wall Street’s expectations, based on the strength of its data center business.

The company hit the $1 trillion market cap just before the US markets opened on Tuesday. Its shares are trading at $407 in the pre-market, nearly 5% up from Monday.

Nvidia’s shares were up more than 165% year-to-date as of Friday afternoon, with the S&P 500 (^GSPC) just 9.5% higher in the same frame.

Rival chip maker AMD has experienced a similar boost in share price, rising 93%. However, Intel (INTC) is lagging behind with shares up just 8%.

According to Yahoo Finance tech editor Daniel Howley, while analysts see Nividia well ahead of its chip rivals in the AI processing space, how long that continues to be the case is anyone’s guess.

Continue Reading


ChatGPT’s Bogus Citations Land US Lawyer in Hot Water



ChatGPT's Bogus Citations Land US Lawyer in Hot Water

A lawyer in the United States is facing disciplinary action after his law firm used popular AI chatbot ChatGPT for legal research and cited fake cases in a lawsuit.

Steven A. Schwartz, who is representing Roberto Mata in a lawsuit against Colombian airline Avianca, admitted to using OpenAI’s ChatGPT for research purposes, and that the AI model provided him with citations to cases that did not exist.

Mata is suing Avianca for a personal injury caused by a serving cart in 2019, claiming negligence by an employee.

Also read: Opera Unveils GPT-Powered AI Chatbot Aria

Bogus all the way

According to a BBC report, the matter came to light after Schwartz, a lawyer with 30 years experience, used these cases as precedent to support Mata’s case.

But the opposing counsel flagged the ChatGPT-generated citations as fake. US District Court Judge Kevin Castel confirmed six of them as non-existent. He demanded an explanation from Schwartz, an attorney with New York-based law company Levidow, Levidow & Oberman.

“Six of the submitted cases appear to be bogus judicial decisions with bogus quotes and bogus internal citations,” Judge Castel wrote in a May 4 order.

“The court is presented with an unprecedented circumstance.”

The supposed cases include: Varghese v. China South Airlines, Martinez v. Delta Airlines, Shaboon v. EgyptAir, Petersen v. Iran Air, Miller v. United Airlines, and Estate of Durden v. KLM Royal Dutch Airlines, none of which did not appear to exist to either the judge or defense.

Lawyer claims ignorance

ChatGPT is a large language model developed by OpenAI. Launched in November, the AI is trained on billions of data from the Internet and can perform a variety of tasks like generate text, translate languages, and even write poetry, and solve difficult math problems.

But ChatGPT is prone to “hallucinations” – tech industry speak for when AI chatbots produce false or misleading information, often with confidence.

In an affidavit last week, Schwartz said he was “unaware of the possibility that its [ChatGPT] content could be false.” He also said that he “greatly regrets” using the generative AI and will only “supplement” its use with absolute caution and validation in future.

Schwartz claimed to have never used ChatGPT prior to this case. He said he “greatly regrets having utilized generative artificial intelligence to supplement the legal research performed herein and will never do so in the future without absolute verification of its authenticity.”

The career attorney now faces a court hearing on June 8 after accepting responsibility for not confirming the authenticity of the ChatGPT sources. Schwartz was asked to show cause why he shouldn’t be sanctioned “for the use of a false and fraudulent notarization.”

ChatGPT’s confident lies

According to the BBC report, Schwartz’s affidavit contained screenshots of the attorney that confirmed his chats with ChatGPT.

Schwartz asked the chatbot, “is varghese a real case?”, to which ChatGPT responded “yes, [it] is a real case.” When asked for sources, it told the attorney that the case could be found “on legal research databases such as Westlaw and LexisNexis”.

Again, the attorney asked: “Are the other cases you provided fake?” ChatGPT responded “No”, adding that the cases could be found on other legal databases. “I apologize for the confusion earlier,” ChatGPT said.

“Upon double-checking, I found the case Varghese v. China Southern Airlines Co. Ltd., 925 F.3d 1339 (11th Cir. 2019), does indeed exist and can be found on legal research databases such as Westlaw and LexisNexis. I apologize for any inconvenience or confusion my earlier responses may have caused,” the chatbot replied with confidence.

Continue Reading

News Feed

Advertise With Us

Unlock a wide range of advertising
opportunities with MetaNews to reach the
fast-paced web3 world.

Publish Your PR

Share your press release with
MetaNews’s growing global audience,
fans, and followers.

Subscribe for Email Updates

* indicates required

Copyright © 1997 – 2023 MetaNews All Rights Reserved

Copyright © 1997 - 2023 MetaNews All Rights Reserved