Love story with AI: Who is listening to your secrets?

In the movie Her, the protagonist begins a romantic relationship with an advanced artificial intelligence system, a relationship that turns out to be doomed. In 2013, this scenario seemed like pure science fiction. “Today, with the rapid development of genetic artificial intelligence and large language models (LLMs), it no longer seems so far-fetched,” warns Phil Muncaster from the team at cybersecurity company ESET.

AI companion apps are proliferating at a rapid pace. As Valentine's Day approaches, more and more people are turning to digital companions for communication and intimacy.

See more articles from iGuRu.gr when you search for news on Google.

iGuRu.gr preferred source

However, the close connection with an artificial intelligence chatbot comes with serious questions. Where are personal conversations stored? Who has access to them? Can they be leaked to third parties or become the target of cyberattacks?

The answers aren't always clear. Romance in the age of artificial intelligence can be exciting. The question is whether you're ready to take the risks that come with it.

Looking for (digital) love

AI companion apps are responding to a growing market demand, explains ESET’s Muncaster. “AI girlfriends” and “AI friends” leverage the power of large-scale language models and natural language processing to interact in a highly personalized way that simulates natural conversation. Services like Character.AI, Nomi, and Replika aim to meet the psychological and, in some cases, romantic needs of their users. It’s not hard to see why more and more developers are looking to enter the space.

Even big tech platforms are trying to meet the demand. OpenAI recently announced that it will launch “erotic content for verified adults” and may allow the creation of “adult” apps based on ChatGPT. Elon Musk’s xAI has also moved to make romantic AI companions available through the Grok app.

A survey published in July found that nearly three-quarters of teens have used AI companions, with half using them regularly. Even more worryingly, a third choose AI bots over humans for serious conversations, and a quarter have shared personal data with them, Muncaster said.

These findings are particularly concerning as the first warning signs are starting to emerge. In October, researchers revealed that two AI companion apps, Chattee Chat and GiMe Chat, had inadvertently exposed highly sensitive user data.

A misconfiguration in an Apache Kafka broker left the apps’ streaming and content distribution systems without access controls. As a result, anyone could potentially gain access to more than 600.000 user photos, IP addresses, and millions of private conversations belonging to more than 400.000 users.

The dangers of a relationship with a bot

Cybercriminals may exploit this new opportunity for extortion and fraud. Information shared by victims in romantic conversations is ideal material for blackmail, while images, videos and audio can be used in deepfake tools for sextortion purposes. Personal data can be sold on the dark web, and depending on the level of security of the app, hackers may also gain access to stored credit card information. According to Cybernews, some users spend thousands of dollars on in-app purchases.

Many developers seem to prioritize revenue over cybersecurity. This creates opportunities for exploitable vulnerabilities or misconfigurations. Some attackers may even create their own malicious companion apps, designed to steal data or convince users to reveal sensitive information.

Even a “safe” app can threaten your privacy. Some developers collect as much data as possible to sell to advertisers. Opaque privacy policies often make it difficult for the user to understand exactly how their data is being used. In addition, your conversations can be used to train or improve LLM, which further increases the risks.

How to protect your family

Whether you use AI companion apps yourself or are concerned about your children, the basic advice is the same: assume that these apps don’t have built-in security or privacy measures. Don’t share information that you wouldn’t share with a stranger. This is especially true for photos or videos that could be revealing.

Better yet, if you’re planning to try one of these apps, do your research first to see which ones offer the best protection. Read their privacy policies to understand how your data is collected and used, and avoid apps with unclear or ambiguous practices. Once you’ve chosen an app, enable all available security features, such as two-factor authentication, and adjust your privacy settings accordingly, such as disabling the use of conversations for model training.

If you are concerned about safety, privacy, or the psychological impact on teens, start an open dialogue with them. Explain the dangers of oversharing and emphasize that these apps are in the business of profit, not the best interests of the users. You may need to set time limits or implement parental controls, especially if age verification and content filtering mechanisms are not adequate.

It remains to be seen whether regulators will impose stricter rules on what developers in this space are allowed to do. For now, romance AI bots operate in a “gray zone,” although the upcoming EU Digital Justice Law could ban highly addictive and personalized experiences.

Until developers and regulators fill the gap, it's probably safest not to treat AI companions as trusted friends or emotional supports.

newsbot

Although the press releases will range from very select to rare, I said I'd pass...because sometimes the editors hide.