Scammers are producing passable artifacts with AI

Scammers are using freely available artificial intelligence tools to increase credultity amongst a broader, more discriminating echelon of…

Scammers are using freely available artificial intelligence tools to increase credultity amongst a broader, more discriminating echelon of the online public.

With language models (such as ChatGPT), generative adversarial neural networks (such as this-person-does-not-exist), and synthographic image generation (such as Midjourney), many of the telltale patterns of common scammer communication have already disappeared.

Poor grammar, spelling mistakes, and low-production quality were once hallmarks of fraudsters, making those traits an easy way to identify scams. While not all scams demonstrated dead-giveaway linguistic signals, banks never did. So identifying low-quality visual insignia and sloppy writing were at least useful as a first-pass filter to sort away contact attempts that obviously were thrown together by a person not supported by professional standards of care or a capable enterprise infrastructure.

Some security investigators have even suggested that these flaws in the substance quality of the contact attempts might be intentially left as a way to entrap only the most gullible. Their logic was that scammers rely on self-selection to reduce false positives for themselves: only the most credulous would respond to these emails in the first place, while those with enough discernment to identify a scam at a glance would be filtered out for the attacker by not responding at all.

I won’t say it’s now out-of-the-question that some scammers may have considered the advantages of demonstrating only elementary English-language fluency. However, I will share some evidence that scammers have recently begun wielding the cutting edge of publicly available language and image AI tools to appear more professional and to prop up the believability of their claims.

Logos

Here’s the email signature from a scam I recently got:

The FIUNION logo did not hit any matches when I searched for it using Google Lens reverse image search, leading me to believe it is an original design. Once upon a time, one would need access to a vector graphics program and the skills to use it to produce a logo like this. Or, one would need to hire a marketing design team, or at least a freelance digital illustrator. Now, with image generation tools like DALL-E, Craiyon, or Stable Diffusion, one can generate corporate-looking logos with little or no artistic skill or financial investment.

Descriptions

The scammer emailed me the following description of her organization.

FIunion Consulting Incorporation is the international body that brings together the world’s securities regulators and is recognized as the global standard setter for the securities sector. Fiunion develops, implements and promotes adherence to internationally recognized standards for securities regulation. It works intensively with the G20 and the Financial Stability Board (FSB) on the global regulatory reform agenda. Fiunion was established in 2015. Its membership regulates more than 95% of the world’s securities markets in more than 130 jurisdictions: securities regulators in emerging markets account for 75% of its ordinary membership.

The Fiunion Objectives and Principles of Securities Regulation have been endorsed by both the G20 and the FSB as the relevant standards in this area. They are the overarching core principles that guide Fiunion in the development and implementation of internationally recognized and consistent standards of regulation, oversight and enforcement. They form the basis for the evaluation of the securities sector for the Financial Sector Assessment Programs (FSAPs) of the International Monetary Fund (IMF) and the World Bank. By providing high quality technical assistance, education and training, and research to its members and other regulators, Fiunion seeks to build sound global capital markets and a robust global regulatory framework.

When it comes to the financial ombudsman, many clients are not fully aware or informed of the functions of this entity. They are also unaware of their rights when dealing with said institutions. We help the client in knowing his rights and how to make sure they are properly acted upon when dealing with financial institutions such as these. We assist the client in the preparation and processing of the package of documents with the ombudsman depending on which organization you are entering into a dispute with, in what jurisdiction, type of complaint, and what stages of settlement you have already gone through.

None of the above is true. It might sound believable enough though. I have entered this block of text into several GPT-detection sites which were not able to detect the copy as GPT-written. However, copy that is a blend of human writing and GPT generation can be hard to spot.

There is no FI Union website, which was a strong indication that fraud was afoot.

This organization claimed to be working at the behest of Interpol to help vicitims of cryptocurrency fraud. They shared with me a document on official-looking Interpol letterhead. I emailed the Interpol press office, which quickly confirmed to me that they do not work with or contract any organization by the name FI Union.

Below is a screenshot of the official-looking notice the scammer, who called herself a “recovery specialist,” sent me as an attachment:

Credentials

My contact shared her CV with me in order to establish her identity. She had no LinkedIn profile, and her organization had no LinkedIn page either.

What I especially want to point out here is the image on the CV. It is a GAN-generated image of a face — not a real person. Three bits of evidence in the AI-generated photo support this conclusion:

1. Out-of-place flesh-on-flesh contact in the lower left corner.

2. Absent or indistinct background

3. Drop-like earring which blends into the ear

I do not believe you can rely on the fact that AI-generated profile photos such as these always have dead, lifeless eyes. There are also many real human beings who seem to me to have soulless-looking eyes, despite having real flesh-and-blood bodies. Judging by the look of the eyes may not be reliable.

While it’s much easier to generate a fake CV with large-language models, one may still need familiarity with actual non-fraudulent work in order to ask for convincing work history details in a language-model prompt. In this case, the person mentioned real organizations on their CV, but there are no details about their accomplishments at those organizations, and the prestigious career tenure does not match up with their vague skillset. However, ChatGPT is able to format text into markdown, html, or other templating formats that can make a bare outline look stylistically like a professional resume.

Conclusion

If you are contacted by someone you suspect to be a scammer, do some background research. Do not let them direct the flow of the conversation. If you believe you’re talking to a scammer, do not continue the conversation. Stop talking to them.

We can no longer rely on scammers to have shady organizational summaries or poorly-made logos. Scammers appear to be using AI tools in order to fool more people. These particular scammers contacted me by phone first before emailing me, and they already knew my name and my gmail address, probably through some publicly available data breach. Be wary of unsolicited offers to recover funds or requests for payment in advance.

Stay safe out there!