Defence in the agent economy
During the July intrusion into its infrastructure, Hugging Face found that the commercial models it first tried would not complete much of the forensic work. Its technical account says their guardrails treated reverse engineering an exploit as though it were assistance to an attacker. The team then deployed GLM-5.2 on its own infrastructure and rerouted the analysis.
Hugging Face ended its account with a practical recommendation: have a capable model that can run on your own infrastructure “vetted and ready before an incident.” Preparing that option in advance means deciding where the model can run and what evidence it may examine, before those decisions become part of an active investigation.
Automated defence can work quickly. In the final round of DARPA’s AI Cyber Challenge, autonomous systems analysed more than 54 million lines of code, found 54 synthetic vulnerabilities and patched 43. Patches were submitted in an average of 45 minutes. The vulnerabilities were planted and the competition was not an incident response exercise, but the result demonstrates automated vulnerability detection and repair under controlled conditions.
An APRA-regulated entity also faces a deadline for notifying its regulator. CPS 234 requires notification as soon as possible, and no later than 72 hours after the entity becomes aware of an information security incident that meets the standard’s materiality test. The 72-hour limit concerns notification; it does not require the investigation to be complete.
The institution also needs to know how long its own approval process would take before a model could analyse sensitive incident evidence. That depends on its data-handling rules and who can authorise access. An exercise can establish whether the response team has an approved option or would have to seek permission while investigating.
CPS 234 does not prescribe a model review. It does require information assets to be classified by criticality and sensitivity, protections appropriate to the importance and sensitivity of those assets, and response plans that cover the relevant stages of an incident. APRA’s April 2026 letter on AI also sets an expectation of risk and information security assessment before deployment and throughout the lifecycle.
Incident evidence may contain credentials, customer information, exploit material and details of live systems. A model accessed through an external provider may refuse to analyse exploit material, or the institution’s rules may prevent that evidence from being sent to it. Running a model on the institution’s own infrastructure avoids some of those restrictions, but the institution must still decide how to deploy it securely, control access and check that it is suitable for the work.
Choosing a capable model is only part of the preparation. The institution must also decide which evidence it may receive, where it may run, who can authorise its use and how that use will be recorded.
ASD and the AICD’s frontier AI guidance for boards asks whether incident response and business continuity plans have been updated and tested for frontier AI threats. It also recommends suitable AI models for cyber defence where their use is secure, controllable and supervised by people. Each institution still has to establish whether its own response team can put that guidance into practice with real incident evidence.
At the next scheduled risk committee meeting, I would put one question on the agenda:
If an incident happened next month, could our response team use a suitable AI model on the evidence with the necessary approvals in place?
An institution may have an approved hosted service, a model approved to run on its own infrastructure, or no current option. If the answer is no current option, the committee has found a concrete readiness gap while there is still time to decide how to close it. Test the arrangement in the next incident-response exercise: give the team realistic sample evidence, simulate its usual AI service refusing the analysis, and establish whether it can use an approved alternative.
Keith Cheng
Arkna
The incident and its evidence limits are set out in Anatomy of an agent escape and The agent’s version of events.