Measure an AI agent by the task it can verifiably complete, not simply by its arrival on a website. Microsoft Clarity’s 4 November announcement adds Agentic Analytics and simulation assessments, initially focused on commerce and travel. It describes supported-agent activity and simulations that report completion, time and interactions. That is a useful trigger to define what success actually means for your business.
Separate the actor from the source of a visit
A human following an AI recommendation is still a human visitor. An agent operating a site on someone’s behalf is a different actor. Keep those populations separate when examining funnels. A referral label answers where a visit came from; it does not by itself prove that software attempted an order, appointment or other task.
Create three explicit evidence groups: observed human visits, detected supported-agent activity and controlled simulations. Retain an unknown category when classification is uncertain. Microsoft lists Google, OpenAI and Anthropic operators among supported agents; that is not a claim that every form of automation is detected. Do not extrapolate an observed subset into the whole market.
Define the denominator before calculating success
Write one task as a testable instruction: for example, find an available appointment for a specified service and obtain a booking confirmation. Record scenario, actor or operator, site version, environment and time. State how a task attempt begins and ends. A pageview is not automatically an attempt, and a repeated action within one attempt should not silently become another success opportunity.
For a defined cohort, calculate confirmed completions divided by recorded task attempts. Publish the numerator and denominator alongside the percentage. Log tasks that could not start separately and include them when reporting an end-to-end assigned-task success rate. Otherwise, excluding early failures can make an unusable journey look healthy. Keep unknown outcomes visible rather than assigning them to success.
Check the outcome outside the interface
An agent reaching a thank-you message is useful evidence, but verify the business record when possible. Was a booking created for the correct service? Was an order accepted with the intended item and quantity? Was a form stored once? Use identifiers to reconcile the attempt with the confirming record and check for duplicate submissions. A simulated completion should never be reported as a real sale.
For diagnostics, use a controlled environment or an agreed test path with appropriate authorization. Do not create unintended purchases just to improve a completion metric. Fix the task conditions, repeat the scenario and document where it fails: discovery, option selection, form entry, validation or confirmation. This is an editorial measurement framework, not a list of guaranteed Clarity report fields.
Turn a failure into an owned improvement
Prioritize reproducible failures in important journeys. Assign an owner, describe the expected result and record the smallest change being tested. Re-run the same scenario after the change, then check human usability and observed activity separately. A successful simulation demonstrates that scenario under those conditions; it does not forecast live conversion uplift.
Use duration and interaction count to explain friction after checking completion, not as substitutes for it. A faster failed booking has not improved the business outcome. Keep this work separate from the announcement’s other products: Ask Clarity is described as a closed beta for retailers. Before planning implementation, verify the availability of the specific capability you need and the scope of its evidence.
