Brackett published the Agent Effectiveness Index (AEI) and launched a Connected Agentic Workforce platform. AEI ranks agents on business understanding, operational execution, and learning persistence, with code and tasks available under an MIT license.
AEI measures agents on three operational dimensions: Business Understanding, Operational Execution, and Learning Persistence.
Brackett also launched a Connected Agentic Workforce platform that captures conversations, codifies processes, and compounds learned judgment across agents.
Brackett introduced the Agent Effectiveness Index (AEI), a free, open-source benchmark designed to score and rank agents on their ability to learn and perform complex operational tasks. Alongside AEI the company launched a Connected Agentic Workforce platform that converts conversations into agents that execute processes and learn judgment over time.
Most current evaluations emphasize static knowledge or how much an agent knows. AEI shifts the focus to whether an agent can actually complete the task it was created for in real operational settings. That includes making judgment calls, handling exceptions, escalating when needed, and retaining what it learns as processes change. Brackett positions AEI as a practical tool for teams that need to measure whether agents behave reliably at scale.
AEI evaluates agent systems across three concrete dimensions:
The benchmark includes a full task set, scoring code, and methodology. Brackett says scoring for execution, transfer, and retention will expand as the Index develops toward a more complete view of effectiveness.
Capture, Codify, and Compound methodology. In plain terms:
AEI's task set, scoring code, and methodology are published on Brackett's GitHub under the MIT License. The Index is available now, and Brackett plans to extend scoring capabilities as the project grows.
Teams evaluating agents can use AEI to move beyond knowledge checks and instead measure real-world task completion and behavioral learning. AEI provides a reproducible framework for testing whether agents follow runbooks, make correct exceptions, and keep behaving appropriately after training or environmental changes.
AEI arrives amid a growing focus on benchmarks that test long-horizon competence, transfer, and retention. Other projects and research efforts are likewise building open frameworks and sandboxes for agent evaluation. AEI's emphasis on operational criteria and open licensing makes it a candidate for teams wanting a practical, auditable way to evaluate agentic workflows.
AEI offers a concrete, operationally focused benchmark for assessing agents on task completion, judgment under exception, and learning retention. With code and tasks available under MIT, organizations can adopt the Index to evaluate agents in ways that mirror real-world requirements.
Originally appeared on OmbuLabs.ai . Once a team stops trusting leaderboards, the next move is usually to build an evaluation of its own. We’ve argued before that benchmark scores are a poor predictor of production agent performance , and the natural follow-up question is what to measure instead. Two answers come up al
Google’s Android Bench 2.0 evaluates frontier AI models on multi-day coding tasks to determine how well agents handle complex engineering. The new benchmark aligns its testing structure with the Harbor framework. Instead of grading simple bug patches, the system introduces long-horizon tasks that require multiple days
Google Cloud AI Research has open-sourced RRSI, a framework that lets LLM agents rewrite their own prompts, tools and memory while model weights stay frozen. It adds a leakage critic, a noise floor, a cost rule and pruning so gains carry over to new tasks. With Claude Opus 4.8, Terminal-Bench 2.1 rose from 74.2% to 80.
Training and benchmarking a computer-use agent needs four things — agents, environments, traces, and a framework to evaluate and train them — and all four ship in incompatible formats today. CUA-Lite, from a UC Berkeley led team, puts them behind one action space and one data schema, and replaces OSWorld's per-task vir
New independent benchmark moves beyond model rankings and token speeds to measure whether AI completes enterprise work correctly, at scale and at what cost SILVERTHORNE, Colo., Sept. 9, 2026 /PRNewswire-PRWeb/ -- Kamiwaza AI, the secure AI orchestration platform that helps enterprises...
How do you benchmark a web search API when the thing being tested can read the answer key? A search agent has a fetch tool. If the gold labels sit in a public dataset, the agent can download them mid-evaluation and skip retrieval entirely. A similar problem arises when the answers are already encoded in […] The post Ke
Loading more related stories...
Open the app view to save this story, compare related coverage, and continue from the same source.