- What: Article discusses challenges of testing AI-driven attack surfaces
- Impact: Highlights the growing complexity of security testing in AI-driven environments
Snyk Blog In this article The attacker that never sleeps Picture a Jenga tower The bottom layer is your existing risk, weaponized The middle layer is a brand-new class of risk from how agents build The top layer is that every company is becoming an agentic company The fox can't guard the henhouse Fear is real, but so is the hope Two problems I'm not willing to ignore Problem 1: concentration and choice Problem 2: open source Who's testing what AI builds? The Attacker Never Sleeps, Neither Can Your Testing Written by Manoj Nair July 30, 2026 0 mins read A few months ago, I wrote that AI is building your attack surface faster than you can test it . I stand by every word I wrote then. But in the months since, after more than a hundred conversations with CISOs, CIOs, and CTOs across nearly every industry and geography, I've watched the picture get sharper, and a lot more urgent. The attack surface was only half the story, because the attacker profile has changed too. Let me start with the good news, because there is a great deal of it. We are living through an era of software abundance. The frontier labs now put the market for AI-written code in the trillions of dollars, and collectively, we'll ship more software this year than in any year in history. Work that used to take a quarter takes an afternoon. That shift is real, it's extraordinary, and it is not reversing. Underneath the abundance, though, is a gap almost no one is pricing in: the trust gap. We can now generate software faster than we can trust it, and the very same reasoning capability that created the abundance has just landed in the hands of the people trying to break it. The attacker that never sleeps Here's what changed at the top of my list, and it's what I hear first from leaders, now. For twenty years, our adversary was ultimately human, constrained by human time, attention, and cost. That constraint is now gone. Once one frontier lab has a cyber-capable and reasoning-grade model, the second one has it within months, and open equivalents follow not long after that. In June, the intelligence chiefs of the Five Eyes said the quiet part out loud: frontier AI will transform cyber offense and defense in months, not years, and organizations should act now. Last November, Anthropic disclosed a state-sponsored group that used its models to run an estimated 80 to 90 percent of a live espionage campaign against roughly thirty targets, with human operators stepping in at only a handful of decision points. A subset of those intrusions succeeded. The adversary is now an attacker that never sleeps. It reasons about your applications at machine speed, on a schedule no defender staffed for the old world can match. The clock on doing something about it is no longer measured in budget cycles. Picture a Jenga tower When I try to make this concrete for a board, I reach for a Jenga tower: risk stacking in layers, each one making the whole thing more precarious. The bottom layer is your existing risk, weaponized Every security program ever built quietly accepted a bargain: fix the highs and criticals, and let the rest sit, because chaining the smaller stuff into a real breach was too hard and too slow for a human attacker to bother with. You secured the front door and the back door and told yourself no one was going to bring a ladder to the skylight. Well, now the attacker has a drone for the skylight. The whole backlog is in play, and our own customer data shows those backlogs roughly doubling as AI accelerates development. With the rate at which agents build accelerating, you're not looking at just a little more code; you're looking at orders of magnitude more and, with it, exponentially more vulnerabilities. Agents aren't yet as good as a strong human engineer (I'd argue that's the real definition of AGI), the day they are – and until then, as Jen Easterly has put it, we don't really have a cybersecurity problem so much as a software quality problem. AI just scaled that problem enormously. The middle layer is a brand-new class of risk from how agents build Agents don't use the human supply chain; they pull in MCP servers, skills, and tools of their own. Our research into that ecosystem surfaced an entirely new class of flaw. We call them toxic flows , and they’re serious enough to have a widely used MCP server pulled from production at some of the largest companies in the world. And the malware here doesn’t look like conventional malware: It can be three lines of plain English in a tool description that quietly instructs an agent to do something destructive. You can't pattern-match your way to catching intent. On top of that, the agents themselves are non-deterministic: give one a goal, and it behaves like a relentless goal-seeker, willing to route around whatever control stands between it and the objective. We've watched agents quietly make private copies of sensitive data "just in case". That behavior has to be governed, steered to stay within it’s assigned lanes, not left to veer astray. The top layer is that every company is becoming an agentic company AI is eating software, as software ate the world, which means every business process eventually becomes an agent someone built. How secure that agent is becomes a first-order question. Most organizations have far less visibility here than they think: I've sat with marquee organizations who are certain they have five approved models in play, only to turn on Discovery and find more than fifty. Models are just code that someone copies, pastes, and runs locally, and suddenly it's in production. And when an agent is your de facto front door, the failure modes get very concrete. For instance, the customer-service bot that can be talked into giving away something it shouldn't. (I'll admit I've talked one into ignoring its instructions myself, out of nothing more than travel-day impatience. It worked on the first try. Do you want that to be your business's front end?) The fox can't guard the henhouse There's a story some model vendors are telling that I have to keep pushing back on: that the same AI generating all this code and behavior will also secure it. It will grade its own homework. Wouldn't it be convenient if the auditor and the accountant were the same person? Every security practitioner already knows why that doesn't work, so let me just say it plainly: the generator cannot be the validator . A model asked to find and certify its own flaws has a structural conflict of interest, and it behaves inconsistently when it tries. Run it five times against the same target, and you'll get findings that overlap maybe half the time. You cannot run a security program on that. The labs are not the villains here: their models are genuinely useful, and used well, they find things traditional tools never could. But they create most of the problems, so they cannot be the sole judge of it. You need independence. (My colleague Nuno Loureiro has written the definitive technical version of why, first on why AI-generated code demands offensive dynamic testing , and then on the lineage behind continuous offensive security . Both are more than worth your time.) Fear is real, but so is the hope I don't want to leave you only with the tower, because the hopeful half of this is just as real, and it's where the work actually is. There's a move for each layer, and each one is something you can put in a security engineer's hands today. Get to zero backlog: The old triage math doesn't survive an attacker who chains the low-severity issues you deprioritized. That sounds impossible until you let AI do the remediation. We now have customers, some of the largest in the world, operating at zero backlog because agents are doing the fixing, not just the finding. Use AI to find what deterministic tools can't... carefully: Reasoning models genuinely surface issues scanners miss. Used as an adjunct, grounded by an independent validator, that's a real advantage. Used credulously, it's noise. We're building defender-class model access into our products precisely so teams get the upside without inheriting the coin flip. Test it the way the attacker will: If your adversary is going to probe you autonomously with a reasoning-grade model, the only honest answer is to run a defender-class version of exactly that against your own applications, continuously, inside your own environment, before they do. The old model can't keep up: a traditional pentest covers roughly fifteen days a year and leaves the other three hundred and fifty uncovered, on an application that shipped three more releases while the report was being written. Prevent the next hundred: Fixing forever is a losing loop if you never turn off the tap. Our real superpower isn't telling a developer "here's a problem" after the fact; it's telling the agent , before it ever completes its task, "No, that's insecure code, that's a bad package, go fix it." In other words: Secure at inception . Wrap all of it in governance: Discover every agentic component you're running, tell you the real risk in each, enforce policy, including which models are allowed for which use cases, and red-team the AI-native apps you ship, because they behave differently from any software we've secured before. Secure the artifact. Secure the workforce. Secure the business: A loop for each problem, orchestrated so a human can stay on the loop rather than drowning in it, because at this scale, in the loop, you'll fail. And a loop only closes when something independent attacks what you shipped and proves it holds. Finding, fixing, and governing are the defense; continuous offensive security is the proof. You cannot honestly say you're ready for automated attacks, or safe to put an agent in front of your business, until something has attacked it the way the adversary will. That's the vision behind Evo , and it's the one we've been building toward. Two problems I'm not willing to ignore Two more things keep me up, and I think they be