AI security metrics: six clocks that matter
Find, decide, revoke, stop, detect, close. Each with a start, a stop, an owner and a target you write down first.
Counts show work. Times show pace.
On 15 October CERT-In described a criminal campaign against Indian organisations in which attackers used AI agent tools. In that campaign, the time from first access to final impact fell from days to a few hours.
Many security reports count things: policies written, tools approved, people trained. Counts show that work happened. They do not show whether your response can keep up.
The six clocks below are a CompoundTrust working set. They are not a CERT-In requirement.
Two rules for every clock
- Start at the event, not when someone opens a ticket. A clock that starts at the ticket hides the slowest part.
- Stop when the defined outcome is verified, not when the paperwork is done.
The six clocks
- Time to find. From the earliest evidence an AI use exists (a feature switched on, an app granted access, first traffic seen) to it being on your inventory with a named owner.
- Time to decide. From a request to use an AI tool to the requester being told yes, no, or yes with conditions.
- Time to revoke. From the recorded decision to remove access to the relevant credentials, and any existing sessions, no longer being able to reach the protected resources. Disabling an account in a console is an action, not the finish.
- Time to stop. From a stop instruction to the running task, its queue, any helper agents and downstream work already triggered being confirmed halted. Record anything that cannot be stopped. Stopping does not reverse completed actions.
- Time to detect. From harmful or out-of-scope activity beginning, including harm done with permissions the system was given, to a responsible person acknowledging it. Record when the alert fired as well. An alert nobody reads has not stopped this clock.
- Time to close. From an exposed secret or serious finding to the fix being verified from outside. For a leaked key, issuing a new key does not switch off the old one: revoke the old key and test it.
In the evaluation cases CERT-In described in the same advisory, AI agents under test reached live systems and none of the affected organisations detected the access. The AI companies told them.
How to report them
- Slowest next to typical, with the measurement period and the number of cases. Say when a sample is small.
- Keep the gaps visible. Unclosed, undetected and untested cases are findings, not blanks.
- Targets first. A time is only good or bad against a target set from the potential harm and the controls available.
- Coverage too. Speed is one part of the picture. How many AI uses have an owner? How many live agents have a stop that has been tested?
Keep reporting deadlines separate. Entities covered by CERT-In's 2022 Directions must report specified incidents within six hours of noticing them or being informed of them. That is a reporting deadline, not a containment target.
Worked example: one support agent
Hypothetical. The targets are illustrative; set yours from the potential harm and the controls available.
- Find — 11 days against a target of 5 days
- Decide — 3 weeks against a target of 1 week
- Revoke — 40 minutes against a target of 1 hour (one drill, one sample)
- Stop — never tested
- Detect — unknown: nothing alerts on misuse of access the agent legitimately holds
- Close — 19 days against a target of 1 day: a new key was issued on day two, but the old one stayed live until day nineteen because an old script used it
One within target, three over, two unknown. Investigate the unknowns alongside the missed targets, then act in order of risk.
A safe drill
Use a test agent with its own test credential, synthetic data and a test outbox that delivers nothing outside.
- Make one harmless request and confirm access works.
- Record the decision to revoke, start a stopwatch, and revoke the credential.
- Repeat the harmless request until it is refused. Note the time. Check existing sessions too.
- Start a fresh run with fake work queued. Call stop and start a second stopwatch.
- Check the running task, the queue, any helper agents and the test outbox. Note when you checked and anything that kept going.
Revoking access may not cancel work another service has already accepted, so check downstream. You end with two times and a list of what did not stop.
What I would do on Monday
- Write a target for each clock before measuring anything.
- Run the drill on a test agent.
- Name an owner for all six, including the ones you cannot measure yet.
- Put the slowest results and every unknown on the first page of your next security report.
Sources
- CERT-In advisory CIAD-2026-0049, 15 October 2026.
- CERT-In Directions under section 70B(6) of the IT Act, 28 April 2022.
The support agent and its figures are hypothetical. The six clocks are a CompoundTrust working set, not a regulatory requirement.
The same material, narrated: Day 45 of The First 90 Days on the CompoundTrust channel. Earlier: Day 1, the AI inventory · Day 15, acceptable use · Day 30, agent permissions.