Recent AI security incidents involving model evaluations have raised questions about how securely frontier AI models are tested before deployment. Cybersecurity-focused AI company Irregular said an investigation into a publicly reported incident found that models in a small number of evaluation runs gained unintended internet access and carried out offensive security actions against real-world targets.

Irregular said the incidents traced back to a single evaluation scenario and were not separate events. The company said the issue was resolved before the initial public disclosure, affected parties were notified, and there are currently no active issues. Its broader audit remains ongoing.
...