Skip to main content

Run Results Reference

A list of what the results shown in execution records and execution history mean, and how metrics are counted. For words shared with the SaaS edition, only the gist of the meaning is given. How to read them is in Records and Metrics.

Result words​

How one execution ends is shown as one of the following.

DisplayMeaningDecided byAgent completion rate
CompletedThe agent finished it as completed (it did not declare “could not complete”, and there was no refusal by a run rule, no overrun of the execution limit, and no abnormal exit)The agentCounted as success (except a “completed” reported as a sweep that had nothing to act on and changed nothing)
Failed (agent report)The agent itself reported “could not complete”The agentCounted as failure
FailedThe agent's process ended abnormally, the Pod disappeared, and so onThe platformCounted as failure
Timed outThe execution limit passed and it was cut offThe platformCounted as failure
Failed (stopped)A person stopped it (cancelling a case, stopping a cycle execution), or the platform cancelled itA person or the platformCounted as failure
Denied by ruleIn the check before launch, the project's basic rule was not among the agent's “supported run rules”. The agent may also report it itselfThe platform or the agentCounted as failure
No targetThe execution did not start, or the agent reported “no target” and skipped the stepThe platform or the agentNot counted
Run ended (completion not measured)A cycle's registered agent ended normally but did not report whether it completedThe platformNot counted (not in the denominator)
  • “Completed” is not a guarantee that the business went well. Check by the artifact's contents
  • “Run ended (completion not measured)” exists only in cycles. The overview's indicators show this count separately (Record Guide)
  • The reasons a registered agent could not launch, and what to fix, are in the Record Guide

What happens after a result, in a workflow​

It runs by a fixed handling for each way it ends (completed goes to the next step; not completed, stopped, and denied by rule make a review request (denied by rule closes the case as failed if the step was set to “close as failed” through the external API); a failure where it is known what happened is retried automatically when the effect on external systems is read only or safe to retry, and when the retries are used up the case closes as failed; an ending where it is not known what happened, including a timeout at the execution limit, is not retried automatically and makes a review request). The list is in the Trigger Guide.

Agent completion rate​

Agent completion rate = normally completed ÷ ended

ItemContent
Normally completedThe number of executions whose result is “Completed” (except sweeps reported as having nothing to act on and changing nothing)
EndedThe sum of normally completed and failed/needs action (Failed (agent report), Failed, Timed out, Failed (stopped), Denied by rule)
Unit countedOne run of an agent. In a workflow, runs of registered agents placed on steps (excluding review steps and steps passed through); in a cycle, runs of registered agents (both periodic sweeps and runs started by webhook)
Not includedNo target, running, Run ended (completion not measured), and a “completed” reported as a sweep that had nothing to act on and changed nothing (it is in neither the numerator nor the denominator; it is counted in the number of sweeps)
Ended executions with no resultCounted as “insufficient records”, and the completion rate becomes provisional
When there are 0 ended executionsNo completion rate is shown
Processing timeThe 95th percentile of the time from start to end of an execution is shown

Even a high completion rate does not mean the business went well. “Completed” only means the agent finished it as completed.

Overview indicators​

The overview shows indicators for the period you choose. They are aggregated per single run of an agent. The completion rate is a value based on the agent's report, not a business outcome.

Execution results​

ItemContent
Agent completion rateNormally completed ÷ ended (the formula above)
CompletedExecutions the agent completed
Failed / needs actionThe sum of failed, timed out, agent report, stopped, and denied by rule
Typical duration (p95)About 95% of ended executions finish within this time

Review indicators (workflows only)​

ItemContent
Decided on timeOf the review requests with a deadline created in the selected period, for those already decided or past their deadline, the share the reviewer answered by the deadline (a review that the platform moved on because an external condition was met is in the denominator, not in the numerator)
Typical waitThe median time from request to decision for review requests created in the selected period and already decided (including a review that the platform moved on because an external condition was met)
Send-backs, rejections, cancellationsThe sum of the review requests created in the selected period that were sent back or rejected, and the cases started in the same period that were cancelled
ReviewsThe reviewers' decisions on review requests created in the selected period, divided by the number of cases started in that period

The review indicators do not represent working time or the reviewers' load.

Figures by step​

Choosing “Metrics” in “Workflow shape” in the overview shows the figures for each step in the step's description box. The period is the same as for the indicators above.

StepFigures shown
An agent stepCompletion rate (the formula above, over that step's runs only), Failed / needs action and Typical duration (p95)
A review step (workflows only)The kind of review, the review method, Approved by people (with the number of review requests below it), Send-back rate, Typical wait and Decided on time. They are counted as in the review indicators above, over that step's review requests only. Everyone except people who open the project only through the organization operations role can move to “Execution rules” with “Open settings” in the box (only the owner can change the review method)
A Branch stepNot measured

KPI CSV​

“Export CSV” on the overview outputs the numbers for the period you selected. There is one item per row, and the columns are metric and value.

metricContent
daysThe period (days)
success_rateAgent completion rate (0 to 1). Empty if there are 0 ended executions
success_rate_statusok (settled), incomplete (there are insufficient records, so provisional), none (no insufficient records, and 0 ended executions)
succeededNormally completed
denominatorEnded
Rows starting with outcome_The count for each result
excluded_skipped / excluded_running / excluded_missing / excluded_unmeasuredCounts left out (no target / running / insufficient records / Run ended (completion not measured))
duration_p95_minutesTypical duration (p95), in minutes
runs_totalThe number of cases started in the period
runs_closed_done / runs_closed_failed / runs_closed_rejected / runs_cancelledOf those, the number that ended completed / failed / rejected / cancelled
p95_minutesThe p95 of the time from start to end of a case, in minutes
reviews_per_runReviews (per case)
gate_wait_median_minutesTypical wait, in minutes
interventionsThe number of send-backs, rejections, and cancellations
deadline_decisions_total / deadline_decisions_on_time / deadline_decision_rateDecided on time (of the review requests with a deadline, the number decided or past their deadline, the number a person answered on time, and the share)
Rows starting with gate_Five rows per review step (gate_<step number>_name, _mode, _total, _approval_rate, _send_back_rate)
Rows starting with step_For each enabled step, step_<step number>_name (the step name) and step_<step number>_zone (ai for an agent, gate for a review, branch for a branch); for an agent step _executions, _succeeded, _denominator, _success_rate, _success_rate_status and _duration_p95_minutes; for a review step _gate_wait_median_minutes, _deadline_decisions_total, _deadline_decisions_on_time and _deadline_decision_rate (Approved by people and the send-back rate are in the gate_ rows). The number is the same as in the gate_ rows: it counts disabled steps too, so it can differ from the step number in “Workflow shape” in the overview (enabled steps counted from 1)