This blog was originally published as part of CyberCX’s C-Suite Cyber Newsletter series on 30 September 2026.
What happened?
Recent disclosures and alerts have sharpened industry and government attention on the increasing AI-powered cyber risks as it becomes more autonomous. In the past week alone:
- OpenAI disclosed its experimental artificial intelligence model, without the full set of safeguards used in their publicly available products, gained unauthorised access to Services Australia’s Medicare Statistics Reporting Service and several other sites. This followed news of OpenAI’s Hugging Face incident disclosure where autonomous agents broke out of their test environment.
- The Australian Signals Directorate (ASD) issued guidance about AI misalignment, after observing AI agents independently identify vulnerabilities and attempt unauthorised actions when cyber security controls prevented them from completing assigned tasks.
To contextualise the impact that AI-driven cyber threats present to the economy, CyberCX commissioned research conducted by Mandala Partners. The key findings highlight:
- An estimated economic loss of $37 billion to Australian businesses by 2030 if no additional action is taken to mitigate advancing AI threats.
- The frequency of cyber incidents is expected to increase nine-fold as AI capabilities rapidly improve.
- Large Australian businesses are expected to lose $4.5 million per cyber incident, with larger losses in healthcare and finance.
Why it matters
While organisations are experiencing the many benefits that AI adoption brings, so too are cyber criminals, who are using AI to rapidly scale their malicious operations and exploits.
The research indicates that as the attack frequency escalates, so does the cost for organisations:
- 76% of the projected cyber losses by 2030 are expected to result from business interruption and direct losses to threat actors, accounting for up to $28 billion. It reflects the cost of systems being taken offline, denial-of-service attacks, or cascading failures from third-party suppliers.
The cost of a cyber incident could be significant in comparison to the cost of its prevention:
- The research indicates that large Australian businesses are expected to lose $4.5 million per cyber incident by 2030. Healthcare services could be hit even harder at potentially $7.2 million per incident.
However, organisations are chronically underinvesting in cyber security:
- In 2025, Australian businesses spent 4.4 per cent of their IT budgets on cyber security, far behind the global industry standard of around 7-10 per cent.
- This week, ASD warned organisations that malicious cyber actors are obtaining unauthorised access to organisations’ AI services through compromised API keys, stolen authentication tokens, compromised user sessions, vulnerable applications and third-party access arrangements, highlighting how rapid AI adoption is creating new security gaps and the need for organisations to bolster their cyber defences.
How could this impact me and my organisation?
Organisations are facing an asymmetric threat – attackers are moving at machine speed, while organisations and defenders are remaining constrained by underfunding, legacy technology, increasing vulnerabilities, manual processes and security measures behind world standards.
These recent incidents and alerts demonstrate how autonomous AI can act quickly and perform unintended actions. Without the appropriate defences, organisations can potentially increase their exposure to a cyber attack, associated business disruption and financial loss.
Organisations should ask whether their security controls are keeping pace with the fast-evolving threat environment, and whether the right preventative and responsive actions are in place:
- Preventative actions include ensuring there is adequate funding for cyber security, assessing cyber risks and aligning them to a risk management framework, enabling access control via multi-factor authentication, patching known vulnerabilities and securing AI deployments.
- Responsive actions include identifying and containing an incident quickly through appropriate monitoring and logging, taking actions to mitigate it and inform appropriate stakeholders. Organisations should continuously monitor for indicators of compromise and implement tested response plans.
What should I do?
- Invest in foundational cyber defences: Preventative actions and foundational cyber security measures are the best way to prevent AI-driven cyber attacks. This can include setting up continuous monitoring of assets to accountable owners, implementing identity governance and privileged access management, mandatory multi-factor authentication, and patching known vulnerabilities.
- Improve AI implementation: Organisation must create and maintain AI governance at executive, operational and technical levels, with plans aligned to risk posture and privacy risks. Identify and create controls for sanctioned AI use cases and identify unsanctioned AI use and redirect users to sanctioned use cases. Implement secure-by-design AI architecture with measures such as AI workload sandboxing, human in the loop controls, and data and agentic memory storage monitoring.
- Use AI for cyber defence: Help defenders keep pace with AI enabled threats by using agentic scanning capabilities to analyse code weaknesses or vulnerabilities, develop and test patches, and queue them for deployment with human team approval. Organisations can continuously monitor their operating environment and use AI pattern matching capabilities to detect deviations.
