
SN121 Dev Log


AI Evaluation Gaming: When the Rubric Becomes the Target
Open AI evaluation creates trust — but also creates pressure to game the rubric. Here's how sundae_bar builds challenges that reward genuine skill, not rubric optimization on subnet 121

How Rewards are Earned on SN121 Skill Challenges
On sundae_bar (Subnet 121), miners earn subnet alpha by submitting the best-scoring skill for the live challenge and holding first place as the validator directs on-chain weight.

Show Your Work: How sundae_bar Audits Every Score
On Bittensor Subnet 121, emissions are on-chain. Follow a skill from submission to score to the #1 weight vote, and verify every step yourself. No black box.

A Letter from the CEO: The Incentive Mechanism Is Live

Why We Evaluate Agent Architecture, Not Model Intelligence

Why SN121 Starts with Stateful Agents

How SN121 Scores Agents: Inside the Evaluation Rubric
Learn how SN121 evaluates AI agents using weighted scoring categories, penalties, and LLM-as-judge grading. See how a real test case maps to the rubric with a breakdown of Task Completion, Relevance, Tone, and Clarity.

End-to-End Agent Evaluation: How SN121 Turns Submissions into Scores
Learn how SN121 evaluates AI agents end-to-end. Follow the pipeline from agent submission through validation, scoring, and aggregation — with full transparency at every stage.

The SN121 Preview Challenge: What We Learned
We ran SN121's Preview Challenge to test our evaluation system. With only 3% hard tests, scores clustered. Here's what we learned and how we're expanding to 40% hard tests for better differentiation.

Anatomy of a Test Case: How SN121 Evaluates Agent Responses
See inside an SN121 test case — from an email extraction task to ground truth with ISO dates, to why an agent scored 0.75 for formatting dates as "October 12th" instead of "2025-10-12". Every score is traceable.

How We Design SN121 Challenges
Learn how SN121 designs agent evaluation Challenges using a three-layer framework: Domains, Skills, and Constraints. See how skills map into capability clusters that reveal where agents fail in production.