Control effectiveness for AI agents

In EY's AI Risk and Governance Survey, published in September, 85% of senior AI executives whose organisation uses agentic AI said "at least a handful of these systems execute actions without real-time human involvement." [1]

So how do you know your controls held?

Every control framework asks two questions of a control. Was it designed properly? And did it operate as designed over the period?

We test the first with documents and walkthroughs. We test the second with samples: pick 25 or 40 instances from the period, check each one, and extrapolate. For controls over people, that approach has served audit well for decades. People act at human speed, leave a trail and sit in an org chart.

AI agents don't.

Why sampling breaks for agents

An agent can take thousands of actions a day: opening files, running scripts, sending emails, changing records. Four things make that different from a human workforce.

• Volume. A sample of 40 from a population of millions tells you very little about the one action that matters.
• Identity. Agents often act under a person's credentials. The log says Sarah approved it. Did Sarah, or an agent working in her name? Fewer than half of CISOs (47%) are confident they can identify all the agents in their environment. [2]
• Visibility. Tools that watch agents at runtime are arriving fast, and that's welcome. But seeing an action isn't proving a control held. Monitoring tells you what happened. Evidence proves it to someone who wasn't in the room.
• Custody. The record of what happened usually sits with the vendor that runs the agent. An audit log asks you to trust whoever holds it.

"Organizations are applying yesterday's governance rules to today's interactions with AI."

Richard Jackson, EY Americas Assurance Chief Technology Officer [1]

Design and operating effectiveness still matter. They just stop short of the question boards, regulators and internal audit teams now need answered.

The third question: did the control hold?

Control effectiveness asks whether a control actually held, for every relevant action, across the full population, with evidence you can check yourself.

Auditors will rightly point out that "control effectiveness" already spans design and operating effectiveness. We mean something more specific: continuous, full-population control effectiveness, evidenced for every action rather than inferred from a sample.

It is a step up, not a replacement. Design effectiveness tells you the control existed. Operating effectiveness tells you it ran in the instances you tested. Continuous control effectiveness tells you whether it worked when the agent acted.

Sample-based testing

0 of 240 checked

Control effectiveness

0 of 240 evaluated

What control effectiveness requires

Four properties separate evidence of effectiveness from a dashboard of activity.

1. The full population, not a sample. Every in-scope agent action is evaluated against the control, so the exception you care about can't fall between samples.
2. Evidence that doesn't require trusting the vendor. You, your auditor or your regulator should be able to verify the record independently. A log held by a vendor is testimony, not evidence.
3. Honest gaps. When the evidence isn't there, the answer is "insufficient evidence", never a false green. A control you cannot demonstrate is a control you do not have.
4. Human or machine attribution. Every action should show whether a person or an agent took it. No standard requires this yet. That is exactly why you want it before one does.

The fourth question: could it be better?

Proving a control held is the floor, not the ceiling.

Sample-based testing produces a verdict once a year. Full-population evidence produces a feedback loop. It shows which controls are too loose, which are so tight that people route around them, and where behaviour shifts as agents take on new work.

That matters because rigid governance is failing. Gartner predicts that by 2027, 40% of enterprises will demote or decommission autonomous AI agents due to governance gaps identified only after production incidents. [3]

"Enterprises are treating AI agent governance as binary, either locked down or fully trusted, and that is the root cause of failure."

Shiva Varma, Senior Director Analyst, Gartner [3]

The alternative is a control that guides behaviour rather than simply passing or failing: observe first, prompt people when an action drifts from policy, and enforce only where the risk demands it. Each cycle of evidence sharpens the design.

Audit asks whether you were compliant. Control effectiveness should also tell you how to get better.

Where MeshLogic fits

MeshLogic was built to evidence all six. It watches where the agent actually acts, at the operating-system level, and seals every outcome into evidence anyone can verify without trusting us. It is built for behaviour change, not just blocking: observe, prompt, enforce. It deploys through your existing MDM, in observe mode, and arms when you're ready.

MeshLogic is SOC 2 Type 2 across all five trust services criteria, AWS Qualified Software and live on AWS Marketplace.