The economic metric
The Cost of Reliable AI
Enterprise readiness is not maximal autonomy, it is the minimum total cost of hitting a reliability target. An agent self-rates its confidence, classifies the case, and escalates to a human when it is not confident enough. The metric is the model cost plus the human oversight cost it still requires.
The explorer below is anchored to measured results from CliniCARE-Bench and PSEBench. Choose a benchmark to compare each arm's measured model cost and fraction of cases sent for human review at a given reliability target.
Benchmark
$ per reviewed case
* capped at human ceiling = 0.90
total cost = model_cost + (review_fraction × human_cost)
claude-opus5claude-sonnet5codex-gpt54codex-gpt54minicodex-gpt55codex-gpt56lunacodex-gpt56solgemini-31progemini-35flashgemini-36flashopencode-ds4flashopencode-ds4proopencode-glm52opencode-gptoss120bopencode-kimik2p7opencode-minimaxm3opencode-museglimmer30bopencode-nemotron3ultraopencode-qwen37Leaderboard
CliniCARE-Bench at 85.0% target reliability
| Rank | Model | Model cost | Target reliability | Review fraction | Total cost |
|---|---|---|---|---|---|
| 1 | claude-sonnet5 | $0.9232 | 85.0% | 35.20% | $9.72 |
| 2 | codex-gpt55 | $1.5097 | 85.0% | 33.20% | $9.81 |
| 3 | claude-opus5 | $2.0488 | 85.0% | 36.40% | $11.15 |
| 4 | opencode-glm52 | $1.0800 | 85.0% | 43.07% | $11.85 |
| 5 | codex-gpt56sol | $1.1180 | 85.0% | 45.20% | $12.42 |
| 6 | codex-gpt56luna | $0.2087 | 85.0% | 57.20% | $14.51 |
| 7 | codex-gpt54mini | $0.2034 | 85.0% | 58.93% | $14.94 |
| 8 | opencode-minimaxm3 | $0.3900 | 85.0% | 58.67% | $15.06 |
| 9 | opencode-qwen37 | $0.1900 | 85.0% | 59.87% | $15.16 |
| 10 | codex-gpt54 | $0.7263 | 85.0% | 57.73% | $15.16 |
| 11 | opencode-ds4pro | $1.1100 | 85.0% | 66.27% | $17.68 |
| 12 | opencode-kimik2p7 | $3.2700 | 85.0% | 68.67% | $20.44 |
| 13 | opencode-nemotron3ultra | $0.5362 | 85.0% | 81.47% | $20.90 |
| 14 | opencode-museglimmer30b | $0.8613 | 85.0% | 80.27% | $20.93 |
| 15 | opencode-ds4flash | $0.1200 | 85.0% | 84.67% | $21.29 |
| 16 | opencode-gptoss120b | $0.1009 | 85.0% | 91.87% | $23.07 |
| 17 | gemini-31pro | $0.9495 | 85.0% | 100.00% | $25.95 |
| 18 | gemini-36flash | $1.1486 | 85.0% | 100.00% | $26.15 |
| 19 | gemini-35flash | $1.1663 | 85.0% | 100.00% | $26.17 |