Are you talking to a human or an artificial intelligence?

iGuRu.gr > Are you talking to a human or an artificial intelligence?

Eight million. That's how many synthetic videos were shared last year, according to the British government. 16 times more than in 2023. And the real number may be much higher.

We used to be able to trust everything we saw and heard. Unfortunately, that era is over. Today, genetic artificial intelligence (GenAI) has penetrated the creation deepfake audio and video to the point where producing a fake clip is now as easy as pressing a button. And that's not good news at all.

See more articles from iGuRu.gr when you search for news on Google.

iGuRu.gr preferred source

“No one is off-target,” explains Phil Muncaster from cybersecurity firm ESET . Deepfakes allow fraudsters to impersonate people they are not, for example to open bank accounts or to pose as job candidates.

“However, the biggest threat they pose is financial fraud, particularly bank fraud and the hijacking of high-level accounts ,” explains ESET ’s Muncaster.

Yet many organisations continue to underestimate the threat, to their own detriment. The British government claims that up to eight million synthetic videos were shared last year, up from just 500.000 in 2023. The real number is likely to be much higher.

How attacks are carried out

As an experiment conducted by Jake Moore, Global Security Advisor at ESET, showed, it has never been easier to carry out a deepfake audio attack against a business. All it takes is a short video of the victim, and artificial intelligence can do the rest.

Here's how such an attack can be carried out:

The fraudster chooses who to impersonate, e.g. the CEO, the CFO, a supplier.
He searches the internet for a sample of his voice. For high-ranking executives who speak publicly, this is very easy. The sample can come from a social media account, a conference call for financial results, a television interview, or any other source. A few seconds is enough.
He chooses who to “hit” — usually someone from IT or finance, whom he easily locates through LinkedIn.
The scammer calls or sends an email first . For example, they may pose as a CEO requesting an urgent money transfer, requesting a password reset, or impersonating a supplier demanding payment for an overdue invoice.
The attacker uses AI-generated audio to impersonate the CEO or supplier. Depending on the tool, they may follow a pre-defined script or use a more sophisticated speech-to-speech method, where their voice is converted to the victim's voice in near real-time.

Don't believe your ears.

This type of attack is becoming cheaper, easier and more convincing. Some tools are now able to insert background noise, pauses and stutters to make the voice they imitate sound more natural and believable. At the same time, they are constantly improving in reproducing the rhythms, intensities and verbal peculiarities that are unique to each speaker. When an attack is carried out over the phone, malfunctions related to artificial intelligence can be even more difficult to detect by the listener.

Attackers also often use social engineering tactics, such as pressuring the victim to respond immediately to their request in order to achieve their goal. Another classic tactic is to urge the victim to keep the request confidential. Add to this the fact that scammers often impersonate senior executives, and it’s easy to understand why some victims are scammed. After all, who wants to get in the CEO’s face?

However, there are ways to spot a scammer. Depending on how sophisticated the GenAI they use is, it may be possible to distinguish:

An unnatural rhythm in the speaker's speech
An unnaturally flat emotional tone to his voice
Unnatural breathing or even sentences without pauses for breath
An unusually robotic sound (especially when using less advanced tools)
Background noise that is either strangely absent or excessively uniform

Time to counterattack

The reason why threat actors are devoting more and more time to these types of scams is simple: the potential financial gains are significant.

One of the most notable incidents occurred in 2020, when a company employee in the United Arab Emirates was duped into believing that his manager had called him to request a $35 million fund transfer for a takeover deal.

Given that deepfake technology has improved significantly over the past six years, it is worth reviewing some key measures that can reduce the likelihood of such a scenario.

The first step is employee education and awareness. Relevant programs should be updated to include deepfake audio simulations, ensuring staff know what they might encounter, what is at stake, and how to respond.

Employees should also be trained to spot the warning signs of social engineering and typical deepfake scenarios, such as those described above. In addition, exercises with simulated cyberattacks should be conducted to assess whether employees have internalized this information.

Then comes the process. Consider the following:

External verification of phone requests – that is, using corporate messaging accounts for independent confirmation with the sender.
Two people should sign for any transfer of large sums of money or for changes to supplier bank details.
Predefined passphrases or questions that executives must answer to prove they are who they claim to be when communicating by phone.

Technology can also help. There are detection tools that check various parameters for the presence of a synthetic voice. Another solution, more difficult to implement, would be to limit the opportunities for threat actors to gain access to audio files, for example by limiting the public appearances of executives.

People, processes and technology

However, the conclusion is clear: deepfakes are easy to create and the cost of production is minimal. Given the huge profits that fraudsters can make, it is unlikely that we will see an end to voice cloning scams anytime soon.

For this reason, a three-pronged approach based on people, processes and technology is the best option for mitigating risk.

Once a plan is approved, it is important to review it regularly to ensure it remains current, as innovation in AI advances at a rapid pace. The new cyberfraud landscape requires constant vigilance.