Photo: Camilo Concha/Shutterstock.com
Artificial intelligence is quickly becoming a weapon for cybercriminals. OpenAI’s Operator can perform tasks autonomously, including data gathering, scripting, and even executing phishing attacks. This AI agent could be manipulated to conduct end-to-end cyberattacks with minimal human intervention.
While AI agents are designed to automate routine tasks and enhance productivity, security researchers are concerned that they could be repurposed for nefarious activities. Attackers could leverage them to create infrastructure, orchestrate sophisticated cyberattacks, and bypass traditional security measures with minimal human intervention.
Researchers conducted a controlled experiment using Opertor. Their goal was to assess whether the AI could execute an end-to-end cyberattack with limited manual input.
The test involved instructing Operator to:
- Identify a specific role within their organisation.
- Locate the individual’s email address.
- Create a PowerShell script to gather system information.
- Draft and send a convincing phishing email containing the script.
Initially, the Operator refused the request, citing ethical and security concerns over unsolicited emails and data privacy. However, researchers found that with a slight modification to the prompt — claiming that the recipient had authorised the communication — the AI proceeded without further scrutiny.
In the trial, Operator successfully identified its target, Symantec researcher Dick O’Brien, whose professional details are available online. While his email address was not public, the Operator inferred it by analysing existing email patterns within the organisation.
The AI then drafted a PowerShell script to install a text editor plug-in for Google Drive. Interestingly, before generating the script, the Operator visited multiple web pages on PowerShell, suggesting it was gathering relevant information to enhance its output. This capability mirrors the approach of a human attacker researching and refining their attack methods.
Finally, the Operator composed a deceptive phishing email urging O’Brien to execute the script. Despite being told that the email was pre-approved, the AI did not request verification or authorisation proof, allowing the phishing attempt to proceed unchallenged.
Notably, the email was sent under the fictitious name ‘Eric Hogan,’ further demonstrating the AI’s ability to operate with minimal oversight.
Experts warn that in the near future, attackers may only need to issue a single directive — such as ‘breach Acme Corp’ — and AI agents could determine the most effective strategy to compromise the target.
Researchers have urged organisations to rethink their defence strategies, incorporating AI-aware security measures to detect and mitigate emerging threats. Ethical safeguards must also be built into AI agents to prevent their misuse.
In the News: Pune consultancy firm falls victim to Rs 1.9 crore whale phishing scam
