

Taylor Sudermann
Head of Product
7 posts


Why SN121 Starts with Stateful Agents

How SN121 Scores Agents: Inside the Evaluation Rubric
Learn how SN121 evaluates AI agents using weighted scoring categories, penalties, and LLM-as-judge grading. See how a real test case maps to the rubric with a breakdown of Task Completion, Relevance, Tone, and Clarity.

End-to-End Agent Evaluation: How SN121 Turns Submissions into Scores
Learn how SN121 evaluates AI agents end-to-end. Follow the pipeline from agent submission through validation, scoring, and aggregation — with full transparency at every stage.

The SN121 Preview Challenge: What We Learned
We ran SN121's Preview Challenge to test our evaluation system. With only 3% hard tests, scores clustered. Here's what we learned and how we're expanding to 40% hard tests for better differentiation.

Anatomy of a Test Case: How SN121 Evaluates Agent Responses
See inside an SN121 test case — from an email extraction task to ground truth with ISO dates, to why an agent scored 0.75 for formatting dates as "October 12th" instead of "2025-10-12". Every score is traceable.

How We Design SN121 Challenges
Learn how SN121 designs agent evaluation Challenges using a three-layer framework: Domains, Skills, and Constraints. See how skills map into capability clusters that reveal where agents fail in production.