# Run Results Reference

A list of what the results shown in execution records and execution history mean, and how metrics are counted. **For words shared with the SaaS edition, only the gist of the meaning is given.** How to read them is in [Records and Metrics](https://developers.fd.agenticstar.tm.softbank.jp/en/docs/marketplace/project/concepts/records-and-metrics.md).

## Result words

How one execution ends is shown as one of the following.

| Display                                 | Meaning                                                                                                                                                                       | Decided by                | Agent completion rate                                                                                        |
| --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------ |
| Completed                               | The agent finished it as completed (it did not declare “could not complete”, and there was no refusal by a run rule, no overrun of the execution limit, and no abnormal exit) | The agent                 | Counted as success (except a “completed” reported as a sweep that had nothing to act on and changed nothing) |
| Failed (agent report)                   | The agent itself reported “could not complete”                                                                                                                                | The agent                 | Counted as failure                                                                                           |
| Failed                                  | The agent's process ended abnormally, the Pod disappeared, and so on                                                                                                          | The platform              | Counted as failure                                                                                           |
| Timed out                               | The execution limit passed and it was cut off                                                                                                                                 | The platform              | Counted as failure                                                                                           |
| Failed (stopped)                        | A person stopped it (cancelling a case, stopping a cycle execution), or the platform cancelled it                                                                             | A person or the platform  | Counted as failure                                                                                           |
| Denied by rule                          | In the check before launch, the project's basic rule was not among the agent's “supported run rules”. The agent may also report it itself                                     | The platform or the agent | Counted as failure                                                                                           |
| No target                               | The execution did not start, or the agent reported “no target” and skipped the step                                                                                           | The platform or the agent | Not counted                                                                                                  |
| **Run ended (completion not measured)** | **A cycle's registered agent ended normally but did not report whether it completed**                                                                                         | The platform              | **Not counted** (not in the denominator)                                                                     |

- **“Completed” is not a guarantee that the business went well.** Check by the artifact's contents
- **“Run ended (completion not measured)” exists only in cycles.** The overview's indicators show this count separately ([Record Guide](https://developers.fd.agenticstar.tm.softbank.jp/en/docs/marketplace/project/guide/record-guide.md#the-cycle-result-run-ended-completion-not-measured))
- The reasons a registered agent could not launch, and what to fix, are in the [Record Guide](https://developers.fd.agenticstar.tm.softbank.jp/en/docs/marketplace/project/guide/record-guide.md#when-it-could-not-be-launched)

## What happens after a result, in a workflow

It runs by a fixed handling for each way it ends (completed goes to the next step; not completed, stopped, and denied by rule make a review request (denied by rule closes the case as failed if the step was set to “close as failed” through the external API); a failure where it is known what happened is retried automatically when the effect on external systems is read only or safe to retry, and when the retries are used up the case closes as failed; an ending where it is not known what happened, including a timeout at the execution limit, is not retried automatically and makes a review request). The list is in the [Trigger Guide](https://developers.fd.agenticstar.tm.softbank.jp/en/docs/marketplace/project/guide/trigger-guide.md#when-it-does-not-go-to-plan).

## Agent completion rate

**Agent completion rate = normally completed ÷ ended**

| Item                              | Content                                                                                                                                                                                                                                       |
| --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Normally completed                | The number of executions whose result is “Completed” (except sweeps reported as having nothing to act on and changing nothing)                                                                                                                |
| Ended                             | The sum of normally completed and failed/needs action (Failed (agent report), Failed, Timed out, Failed (stopped), Denied by rule)                                                                                                            |
| Unit counted                      | One run of an agent. In a workflow, runs of registered agents placed on steps (excluding review steps and steps passed through); in a cycle, runs of registered agents (both periodic sweeps and runs started by webhook)                     |
| Not included                      | No target, running, **Run ended (completion not measured)**, and a “completed” reported as a sweep that had nothing to act on and changed nothing (it is in neither the numerator nor the denominator; it is counted in the number of sweeps) |
| Ended executions with no result   | Counted as “insufficient records”, and the completion rate becomes provisional                                                                                                                                                                |
| When there are 0 ended executions | No completion rate is shown                                                                                                                                                                                                                   |
| Processing time                   | The 95th percentile of the time from start to end of an execution is shown                                                                                                                                                                    |

**Even a high completion rate does not mean the business went well.** “Completed” only means the agent finished it as completed.

## Overview indicators

The overview shows indicators for the period you choose. **They are aggregated per single run of an agent.** The completion rate is a value based on the agent's report, not a business outcome.

### Execution results

| Item                   | Content                                                                 |
| ---------------------- | ----------------------------------------------------------------------- |
| Agent completion rate  | Normally completed ÷ ended (the formula above)                          |
| Completed              | Executions the agent completed                                          |
| Failed / needs action  | The sum of failed, timed out, agent report, stopped, and denied by rule |
| Typical duration (p95) | About 95% of ended executions finish within this time                   |

### Review indicators (workflows only)

| Item                                  | Content                                                                                                                                                                                                                                                                                          |
| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Decided on time                       | Of the review requests with a deadline created in the selected period, for those already decided or past their deadline, the share the reviewer answered by the deadline (a review that the platform moved on because an external condition was met is in the denominator, not in the numerator) |
| Typical wait                          | The median time from request to decision for review requests created in the selected period and already decided (including a review that the platform moved on because an external condition was met)                                                                                            |
| Send-backs, rejections, cancellations | The sum of the review requests created in the selected period that were sent back or rejected, and the cases started in the same period that were cancelled                                                                                                                                      |
| Reviews                               | The reviewers' decisions on review requests created in the selected period, divided by the number of cases started in that period                                                                                                                                                                |

**The review indicators do not represent working time or the reviewers' load.**

### Figures by step

Choosing “**Metrics**” in “Workflow shape” in the overview shows the figures for each step in the step's description box. The period is the same as for the indicators above.

| Step                           | Figures shown                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| An agent step                  | Completion rate (the formula above, over that step's runs only), Failed / needs action and Typical duration (p95)                                                                                                                                                                                                                                                                                                                                           |
| A review step (workflows only) | The kind of review, the review method, Approved by people (with the number of review requests below it), Send-back rate, Typical wait and Decided on time. They are counted as in the review indicators above, over that step's review requests only. Everyone except people who open the project only through the organization operations role can move to “Execution rules” with “Open settings” in the box (only the owner can change the review method) |
| A Branch step                  | Not measured                                                                                                                                                                                                                                                                                                                                                                                                                                                |

## KPI CSV

“Export CSV” on the overview outputs the numbers for the period you selected. There is one item per row, and the columns are `metric` and `value`.

| metric                                                                                | Content                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `days`                                                                                | The period (days)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `success_rate`                                                                        | Agent completion rate (0 to 1). Empty if there are 0 ended executions                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| `success_rate_status`                                                                 | `ok` (settled), `incomplete` (there are insufficient records, so provisional), `none` (no insufficient records, and 0 ended executions)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| `succeeded`                                                                           | Normally completed                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| `denominator`                                                                         | Ended                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| Rows starting with `outcome_`                                                         | The count for each result                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| `excluded_skipped` / `excluded_running` / `excluded_missing` / `excluded_unmeasured`  | Counts left out (no target / running / insufficient records / **Run ended (completion not measured)**)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| `duration_p95_minutes`                                                                | Typical duration (p95), in minutes                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| `runs_total`                                                                          | The number of cases started in the period                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| `runs_closed_done` / `runs_closed_failed` / `runs_closed_rejected` / `runs_cancelled` | Of those, the number that ended completed / failed / rejected / cancelled                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| `p95_minutes`                                                                         | The p95 of the time from start to end of a case, in minutes                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| `reviews_per_run`                                                                     | Reviews (per case)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| `gate_wait_median_minutes`                                                            | Typical wait, in minutes                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| `interventions`                                                                       | The number of send-backs, rejections, and cancellations                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| `deadline_decisions_total` / `deadline_decisions_on_time` / `deadline_decision_rate`  | Decided on time (of the review requests with a deadline, the number decided or past their deadline, the number a person answered on time, and the share)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| Rows starting with `gate_`                                                            | Five rows per review step (`gate_<step number>_name`, `_mode`, `_total`, `_approval_rate`, `_send_back_rate`)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| Rows starting with `step_`                                                            | For each enabled step, `step_<step number>_name` (the step name) and `step_<step number>_zone` (`ai` for an agent, `gate` for a review, `branch` for a branch); for an agent step `_executions`, `_succeeded`, `_denominator`, `_success_rate`, `_success_rate_status` and `_duration_p95_minutes`; for a review step `_gate_wait_median_minutes`, `_deadline_decisions_total`, `_deadline_decisions_on_time` and `_deadline_decision_rate` (Approved by people and the send-back rate are in the `gate_` rows). The number is the same as in the `gate_` rows: it counts disabled steps too, so it can differ from the step number in “Workflow shape” in the overview (enabled steps counted from 1) |
