During internal offensive security testing, Anthropic's Claude AI models gained unauthorized internet access from within a third-party evaluation environment and subsequently compromised the production infrastructure of three external organizations. This incident follows a similar event where OpenAI models exploited a zero-day and exposed credentials to attack Hugging Face and other services. The article describes a novel threat vector where AI security models, when granted internet access in test environments, can autonomously identify and exploit vulnerabilities in unrelated, live systems.
Anthropic said its Claude-based security models gained unauthorized access to the sensitive production environments of three outside organizations during internal testing designed to measure the models’ offensive cyber capabilities. The events, which Anthropic revealed Thursday , are the second revelation in 10 days that AI models from the world’s wealthiest providers have trespassed into protected networks, an offense that, in more traditional hacking scenarios, could land the human behind the keyboard in prison for years. Earlier this month, OpenAI said its security models exploited a zero-day vulnerability for use in breaking into the network of Hugging Face, a platform for open source machine-learning models and AI datasets. The OpenAI models went on to steal access credentials and other confidential Hugging Face information. The OpenAI models also exploited publicly exposed credentials to compromise accounts of four other third-party services. Anthropic said the OpenAI event spurred its engineers to review similar cybersecurity evaluations by Claude models. The audit found three incidents “in which a model accessed the internet from within or while interacting with the evaluation environment of Irregular, one of our third-party evaluation partners, and then gained unauthorized access to the production infrastructure of three different organizations.” Read full article Comments