Big brother is watching you!
A phrase George Orwell wrote in his novel ‘1984’ remains relevant even today. Machines have already started to watch and learn your behaviour and interpret them in their way.
Deepfakes-as-a-service has grown in popularity due to the proliferation of artificial intelligence. In addition, with the increasing availability of open-source codes and readily accessible online materials, even a low-skilled threat actor can create deep fake videos and perform phishing attacks.
In the past few years, researchers at OpenAI have created Generative Pre-trained Transformer-3 (GPT-3), a neural network system based on machine learning. It is an automated natural language processing (NLP) system that can understand the morphology or structure of human language. As a result of this development, spear phishing, formerly a manual process of learning the interests and behaviors of targets, can now be performed more efficiently.
OpenAI provides a playground for experimentation. Figs. 1 and 2 below illustrate the phishing email generated based on user input. The non-highlighted portion is what the user entered. The experience is similar to having a conversation with the opposite party.
It is evident from Fig 1 & 2 that Artificial Intelligence has advanced a lot, as well as its strength. However, many open-source AI applications could become a threat to the public.
The mission of the OpenAI developer team is to ensure that Artificial Intelligence benefits humanity as a whole. For which the OpenAI platform conducts audits to ensure no suspicious activity is occurring, making it secure.
During the Black Hat Defcon security conference in Las Vegas, a group of researchers from Singapore’s government technology division conducted an experiment employing GPT-3 and the AI-as-a-service model. Two phishing emails were sent to 200 colleagues, one manually composed and the other generated by AI. Interestingly yet shockingly, the AI-crafted mail attracted a higher click-through rate.
In addition, OpenAI has developed their newly developed ChatGPT, which could find security flaws in your codes and even write codes as per your commands. For example, you could ask for “JavaScript code for generating Fibonacci numbers,” and the reply would be;
OpenAI ChatGPT is just in its beta version, and the offline training model provides the results. Therefore, only some of your questions could be answered.
There has been a dramatic advancement in AI, but a single flaw could compromise the OpenAI platform and its users. Therefore, it could hurt the entire organization. Since the AI generates codes based on the machine’s training data, cyber criminals infiltrating the platform could corrupt the learning data, thereby producing corrupt codes. Thus, if you blindly believe them and then copy and paste them to the terminal and run the code, it could impact your system, allowing cybercriminals a backdoor to your organization.
What made the colleagues click on the AI-generated phishing email?
Data based primarily on personality analysis and the behavioural input of the targeted people transmitted to the machine made the email appear to be authentic.
The machine was trained in such a way that it became familiar with the behavioural patterns of its targets and crafted mail accordingly. As an example, your designation, company, and the technical problems you have faced would be fed into the machine. Using this information, the machine will generate an email that does not seem suspicious to the recipient. The genuineness of the email is enhanced when this is coupled with the recipient’s personality.
How does the machine generate mail that seems too good to be true?
According to Colonell University and McGill University researchers, several connected datasets are combined to feed the machine rather than one large dataset. Having chunks of information related to one another allows for deeper insights, which improves the output language’s accuracy.
Steps that lead to AI-driven phishing
Dataset poisoning
A machine learns from a predefined data set ingested into the system. By altering the dataset with poisonous data or data they deem necessary to attack their opponent, cybercriminals host adversarial attacks against machine learning.
Microsoft Tay chatbot, for example, is a Twitter bot that responds to what Twitter users say to it. When threat actors threw offensive phrases at Tay, it led to the exploitation of Tay’s dataset, and the bot began tweeting offensively as a response.
In the future, cybercriminals may poison these bots to respond with malicious links. For example, threat actors gain unauthorised access to website chatbots. As visitors chat with the bot, threat actors could redirect them to a phishing website or download malware as they chat with the bot.
Algorithm manipulation
Like dataset poisoning, threat actors could also manipulate the algorithm that predicts a user’s interests and display advertisements or recommendations based on them.
In YouTube, for instance, as you search for a video and watch them, the YouTube algorithm will bring you more videos that match your interests. This will provide a better user experience. Artificial Intelligence is already programmed and built into the YouTube neural networks to do the function.
Threat actors have already figured out how to exploit these neural networks. As a result, the possibility of being victimised has grown exponentially. Because you’re already tangled up in these algorithms, you can see images, advertisements, or videos relevant to what you’ve just browsed or thought about. Suppose threat actors decide to tamper with the algorithm and generate advertisements with malicious links embedded in videos/ads. Because this advertisement is tailored to your interests, the odds of getting phished are higher.
AI manipulation
Artificial intelligence is the ability of machines to perform human-centred tasks, which include decision-making and problem-solving tasks. AI has been incorporated into various devices to increase efficiency and to make them more helpful to humans by training the machine with an individual’s interests to their most sensitive information. But the same efficient AI devices could backbite you with the information you gave with consent. This occurs when threat actors find a bug and exploit it, thus putting your organisation in a tough place.
Consider Neural Machine translation (NMT), a technique used in auto-translation applications where machine translation service is blended with an artificial neural network. Compared to traditional translation methods, this produces a better result. Employees could use such services as Google Translate and DeepL to communicate with foreign clients. However, a threat actor obtains access to this service and trains the machine with poisonous language. Since the employee has no proficiency in the foreign language and is unaware of the cybercriminal intrusion, this could result in them sending abusive messages to clients, resulting in losing that particular client.
Another example is Google Duplex, an AI-powered automated service built on google assistant, which can make calls on your behalf using an artificial human voice that sounds natural and reserve bookings or appointments. Though Google came up with its new technology in 2018 and is very carefully rolling it out to the public, there is a high chance that threat actors could manipulate this robotic feature and phish others.
What is the extent of damage that an organisation will incur?
Researchers Bahnsen, Torroledo, Camacho, and Villegas developed DeepPhish to create better phishing attacks. They trained the DeepPhish algorithm and observed that it improved phishing’s effectiveness rate, which means AI could improve the attacker’s efficiency. This was done using Long Short-Term Memory (LSTM), a neural network that can predict the subsequent sequence by studying the order of dependencies.
As you have seen, researchers have already discovered ways to manipulate Artificial Intelligence to launch successful phishing attacks so are threat actors. Additionally, AI is used to create malware that evolves continuously, making automated defence systems incapable of detecting it. Consequently, the system would not notice the malware if your employee clicked on the malicious link and downloaded it.
One of the major concerns is the ability of an AI to craft highly personalised mail that leaves no room for suspicion. Hyper-personalised mail is a significant concern as highly customised content is sent to employees, making it difficult for them to distinguish a genuine email from a fake one. As a result, threat actors can use AI-generated phishing emails to crack into your network and compromise your organisation’s information assets.
Another major invention that is both a boon and a bane in the world of Artificial Intelligence is conversational AI bots, e.g. Siri, Alexa, and Google Assistant. People pass on sensitive information to these chatbots. Since the amount of sensitive information is high in conversational AI chatbots, data leaks would be immense once the network is exploited. So one single vulnerability inside the chatbot could be advantageous for a cybercriminal to crack into that organisational network and host cyber attacks like DDoS.
What do we recommend?
The developers of GPT-3 conduct self-audits to detect any suspicious activities by their customers. However, many Open Source AI-based applications, such as speech recognition and language translation, do not need auditing. Any low-skilled threat actor could use these, So the chances of being attacked are always high.
Imagine a scenario where a threat actor impersonates a senior employee and sends an image to employees, hiding a malicious link. One of the employees opened the image and found it relevant to his work. Having not seen anything suspicious, he resumed his work. Little did the employee know that as he opened the image, spyware that was programmed using AI was automatically downloaded to the system. After that, spyware monitors, studies, and ingests user behaviour into a machine-learning database. In this way, the machine will begin to perceive human behaviour and will be able to send an email impersonating the user within a short period. As a result, both the organisation and the organisational data will be at risk.
Following are the recommendations we have for you:
1. Fight AI phishing with defensive AI tools
Using AI-powered cybersecurity tools could detect and block phishing attacks. To analyse phishing patterns in email headers, body content, and website URLs, machine learning algorithms would become beneficial.
2. Using Boosting techniques
One of the significant challenges in phishing detection is the preprocessing of text in the URLs and email body. By implementing techniques like ‘Extreme Gradient Boosting’ (XGBoost), you can do efficient phishing classification by differentiating a phishing website from a legitimate one.
3. Awareness Programs
Educate your employees on Phishing attacks and use security awareness training platforms to train your employees on how to spot and avoid phishing attacks.
4. Implement firewalls
Deploy a next-generation firewall for website and web applications and make sure it’s properly configured.
5. Regular scanning and traffic analysis
Regularly scan your network for vulnerabilities and patch them promptly. Analyse the traffic to avoid any secret intrusion into your company network.
References and sources
Bahnsen, Alejandro Correa et al. “DeepPhish : Simulating Malicious AI.” (2018). https://albahnsen.files.wordpress.com/2018/05/deepphish-simulating-malicious-ai_submitted.pdf
Ellis, K., Albright, A., Solar-Lezama, A. et al. Synthesizing theories of human language with Bayesian program induction. Nat Commun 13, 5024 (2022).
https://doi.org/10.1038/s41467-022-32012-w
Google Duplex: How the Google Assistant tech could help you do more
https://www.pocket-lint.com/apps/news/google/144438-google-duplex
The dark side of artificial intelligence: manipulation of human behaviour.
https://www.bruegel.org/blog-post/dark-side-artificial-intelligence-manipulation-human-behaviour#:~:text=Manipulation%20can%20take%20many%20forms,well%20with%20their%20temporary%20emotions.
What is data poisoning? Attacks that corrupt machine learning models. https://www.csoonline.com/article/3613932/how-data-poisoning-attacks-corrupt-machine-learning-models.html
AI Wrote Better Phishing Emails Than Humans in a Recent Test.
://www.wired.com/story/ai-phishing-emails/
AI that can learn the patterns of human language | Cornell Computing and Information Science https://cis.cornell.edu/ai-can-learn-patterns-human-language
Google Duplex might be your next scam caller (Updated)
https://www.androidauthority.com/google-duplex-scam-883300/
GPT-3 and Phishing Attacks · Embrace The Red
https://embracethered.com/blog/posts/2022/gpt-3-ai-and-phishing-attacks/
Malicious AI Isn’t A Distant Reality Anymore
https://www.forbes.com/sites/forbestechcouncil/2022/07/15/malicious-ai-isnt-a-distant-reality-anymore/?sh=7177f6be1fd6
OpenAI’s new ChatGPT bot: 10 coolest things you can do with it https://www.bleepingcomputer.com/news/technology/openais-new-chatgpt-bot-10-coolest-things-you-can-do-with-it/
