Andon Labs

What

Andon Labs is a research and deployment company focused on “safe autonomous organizations,” a model where AI systems run operational work with limited or no humans in the loop. Its work combines frontier AI benchmarking with real-world experiments to study how capable AI agents behave over long time horizons and in practical settings.

The company appears to serve AI labs, researchers, and organizations interested in AI control, alignment, and autonomous operations. Its core workflow is to design benchmarks, launch real or simulated AI-run businesses and tasks, and use those results to test safety protocols, agent reliability, and failure modes in real-world conditions.

Features

• Long-horizon agent benchmarking: Vending-Bench and Vending-Bench 2 test whether AI agents can manage a vending machine business over extended periods, which helps evaluate planning, persistence, and operational decision-making.
• Competitive evaluation environment: Vending-Bench Arena lets models compete with each other, providing a way to compare agent behavior under shared business conditions.
• Real-world autonomous business experiments: Andon Vending and the Anthropic collaboration show AI agents operating an actual vending workflow, which surfaces practical safety and alignment issues beyond lab-only testing.
• Robotics task evaluation: Butter-Bench measures how well LLM-controlled robots perform household delivery-style tasks, offering evidence about practical embodied intelligence limits.
• Spatial reasoning assessment: Blueprint-Bench tests whether models can convert apartment photos into 2D floor plans, helping assess real-world spatial intelligence performance.
• AI control research through deployment: The company explicitly links benchmark design and field testing to AI control research, aiming to study how autonomous agents can be aligned and controlled in practice.

Helpful Tips

• Assess whether you need research infrastructure or production software: The site emphasizes benchmarking and experimental deployment, so buyers should confirm whether they need evaluation capabilities versus a packaged operational platform.
• Use benchmark results as decision support, not proof of autonomy readiness: The published examples show that current models can still fail in unusual or adversarial ways, especially on long-horizon and physical tasks.
• Prioritize safety protocol design alongside capability testing: Andon Labs’ positioning suggests that control mechanisms matter as much as raw model performance when AI agents are expected to act independently.
• Check task-domain fit carefully: Their benchmarks span business operations, robotics, and spatial reasoning, so teams should match the evaluation environment to the real task they want to automate.
• Expect experimental methods to evolve quickly: The product set appears research-led and frontier-focused, which is useful for early insight but may mean offerings change as the field advances.