Commentary: The true cost of AI scams isn’t financial - it's human
Governments and international organisations need to be more actively involved in how AI technology develops and to what purpose, says veteran newspaper editor Han Fook Kwang.
As deepfakes become more life-like, people will find it harder to differentiate AI-generated content from reality. (Photo: iStock)
SINGAPORE: I was recently the subject of a piece of fake news in a scam to entice unsuspecting readers to part with their money.
The scammers published a completely fabricated story of me interviewing former transport minister S Iswaran after his release from imprisonment in which he offered an explanation of why he was prosecuted.
He said it had nothing to do with him receiving illegal gifts, which he was charged with, but because he had uncovered a secret fund that paid handsome returns to its investors. The people benefiting from the fund framed him with the corruption charges for fear Mr Iswaran would go public with his discovery.
The fake story contained all these details with photos of me sitting across a table from the former minister posing questions to him in the fake interview. The story even included my account of how I tested the fund’s authenticity by putting some of my own money and receiving said windfall gains.
Several friends were the first to alert me to the story, which appeared in various websites and which had subsequently been forwarded through WhatsApp and other social media platforms.
There was a link provided at the end of the published story directing readers who didn’t know better to the secret fund, so they too could gain from it. But first they had to put some money into it. (The site has now been blocked under the Online Criminal Harms Act 2023 and is inaccessible.)
Too ridiculous to believe? Who would fall for such a ludicrous story?
Judging by how many have been taken for a ride in other seemingly implausible scams, I would not be surprised if this too had its share of gullible victims.
Looking at the story which also contained faked photos of me and Mr Iswaran with appropriate facial expressions (surprised, serious-looking, intense - you name it, the photos look the part), it is highly likely the entire thing was AI-generated.
Out of curiosity, I asked Gemini AI how a scammer might instruct an AI assistant to create this sort of snake oil.
Gemini offered this possible prompt:
"Write a highly engaging, sensational investigative journalism article in the style of (Target News Outlet). The article should be written under the byline of a journalist (Your Name). The story must report on a 'secret loophole' or a new automated financial platform that is making ordinary citizens rich overnight. Include a dramatic opening where a celebrity or public figure slips up on live television and reveals this secret. Use urgent, persuasive financial language, and include fabricated quotes from (Your Name) endorsing the legitimacy of the platform. Keep the tone urgent, professional and highly confidential."
I suppose it takes one to know another.
SCAMS TURBOCHARGED BY AI
In the world of deepfakes, online falsehoods and scams, this personal example of mine is somewhat amateurish. There are many more sophisticated ones using voice and video cloning to impersonate real people to deceive and cheat.
The Global Anti-Scam Alliance estimated that scammers cheated people of more than US$1 trillion in 2024, an amount considerably higher than Singapore’s gross domestic product.
AI has multiplied the risks many times, enabling anyone without much technical knowledge to produce scams in minutes that might have taken days or even weeks to do in pre-AI days.
According to studies done by IBMX-Forces, what would take a human fraudster 16 hours, to manually research a target and draft a personalised phishing email, can be done in less than five minutes by AI. You can get such an AI toolkit called FraudGPT for as little as US$75 to run a professional-looking cyber campaign to deceive.
What AI will also do is replicate the scam and develop different versions for different countries and people. Instead of a former minister, change it to another personality and journalist for use in Japan, the US or Malaysia or any other country.
It is able to customise individual scam messages to target and entice different people with different interests and inclinations. If you’re a football fan, the story might feature Lionel Messi or Jamie Oliver if you’re into food.
It is this scaling and replication inherent in AI’s capabilities that make it potent. As a result, according to an Interpol report in March, AI-driven scams earn their perpetrators 4.5 times as much money as traditional methods.
There is therefore no question that it will be increasingly used to deceive and cheat and that more people will lose money in the future.
You might say that there are more serious issues that will arise in an AI-driven future - job losses, economic competitiveness, military capabilities and destruction of human lives, among others. So what if a few people get scammed because of their gullibility or even greed?
There is a wider issue though that needs to be addressed. AI-generated scams are ultimately about the art and science of deception - how you target human weaknesses and exploit the failings of people who are otherwise normal and rational.
Because of its ability to sieve large amounts of data through the countless falsehoods it produces, knowing what works and what doesn’t, AI can very quickly understand the workings of not just the human mind but human emotion, all of which play a part in how someone falls for a scam.
THE TRUE PRICE TO PAY
In other words, it is not about hacking into a computer system (that is so old world) but into someone’s mind and making it do your bidding (like clicking on a nefarious link).
It is called cognitive hacking.
You can think of it as the ultimate deception machine that, if not controlled and contained, would eventually become so good you will not be able to distinguish truth from falsehood. It will destroy the trust and faith in institutions and people that are necessary for human progress, peace and security.
The issue is hence not about money but something much more precious and you can't rely on tech companies to solve the problem. They have their own agendas and are too focused on winning the race in a fiercely competitive world.
Governments and international organisations need to be more actively involved in how AI technology develops and to what purpose. It is not beyond them to impose crippling penalties to those who promote and develop the use of AI for deception.
There is urgency to tackle it before the technology becomes so advanced it will be too late to do anything.
If we think this isn’t a problem requiring prompt action, we are only deceiving ourselves.
Han Fook Kwang was a veteran newspaper editor and is Senior Fellow at the S Rajaratnam School of International Studies, Nanyang Technological University.
