Australia to legislate mandatory frontier AI safety controls and propose regulator

The Albanese Government says it will legislate mandatory safety and security protections for frontier artificial intelligence through Australia’s AI Standards legislation, according to the Prime Minister’s official statement. Frontier AI means the most sophisticated models at the cutting edge of current technological capability. The protections would apply to all frontier AI companies that wish to operate in Australia.

The Government will propose a new targeted AI safety regulator to enforce the obligations and impose significant penalties for non-compliance. The regulator would work with the Australian AI Safety Institute (AISI), which would provide technical expertise and oversight as global standards develop. The announcement is a policy commitment, not an operative requirement: the protections still have to be legislated, and the Government says further details will follow after detailed work and consultation.

The proposed obligations

Companies would be required by the regulator to:

- maintain security controls across every stage of development and deployment;
- identify and manage serious risks associated with their models;
- allow independent evaluation of their models;
- notify the Australian Government rapidly of incidents;
- maintain and publish safety and security frameworks covering governance, risk management and transparency; and
- respond to updated standards and rules as technology changes.

The Government said the measures are intended to provide earlier warning of emerging risks from “rogue AI agents” and enable faster, coordinated action when serious AI incidents occur. Prime Minister Anthony Albanese said the protections would support confidence that frontier AI companies are taking steps to improve safety and security, while providing certainty for investors.

For compliance teams, the proposed duties would reach beyond pre-deployment testing. They would require security controls across development and deployment, independent assessment, incident reporting and public-facing governance frameworks. The obligation to respond to changing standards also points to an ongoing compliance requirement rather than a one-off certification exercise, although the announcement does not yet specify reporting windows, evaluation criteria or penalty amounts.

Link to Australia’s safety work

The approach follows recent technical work by the Department of Industry, Science and Resources and the AISI. The Department published a report on scalable oversight on 8 October examining whether AI systems can influence their overseers while completing assigned tasks. It also released a multi-agent risk framework in August, addressing failures that can emerge when autonomous systems interact across organisational boundaries.

The Prime Minister’s announcement says the proposed protections align with interim recommendations from the Joint Select Committee on Artificial Intelligence. Deputy Prime Minister and Defence Minister Richard Marles said the settings would support Australia’s economy, national security and sovereignty. Home Affairs and Cyber Security Minister Tony Burke said companies need “clear guardrails” rather than a “free-for-all”.

Further details of the legislation will become available as the Government completes its detailed work and consultation.