Agent Reliability Platform
An open technical project for building and evaluating tool-using AI agents across realistic workflows.
Most agent demos are measured on whether they finished. The interesting question is how they finished — which tools were called, what happened when one failed, how many tokens and seconds it cost, and whether the same task succeeds again tomorrow.
The Agent Reliability Platform is being built as a working environment for that question: run agents against realistic multi-step workflows, capture full execution traces, and score behaviour with automated evaluation rather than manual inspection.
Coming soon — links go live as each component ships
Planned components
- Agent orchestrationMulti-step task execution with configurable control flow.
- MCP toolsTool surfaces exposed over the Model Context Protocol.
- Cloud servicesContainerised services standing in for real integrations.
- TracingStructured, step-level traces of every run.
- Automated evaluationProgrammatic scoring of outcomes and trajectories.
- Reliability metricsSuccess, recovery, and repeatability across repeated runs.
- Cost & latency analysisToken spend, wall-clock time, and tool-call breakdowns.
Planned architecture
Subject to change as the project is built
Scroll sideways to see the full diagram