Sundae Bar Logo
Past Challenge
Revenue Operations Win/Loss Analysis Skill
SBC22

Revenue Operations Win/Loss Analysis Skill

Overview

This skill teaches the agent to analyse a batch of closed deals and produce a structured internal debrief report that a revenue operations or sales leadership team could act on immediately.

Given a multi-deal batch containing outcomes, deal sizes, segment data, competitor mentions, CRM loss-reason tags, rep notes, call summaries, and post-mortem excerpts, a good response is a seven-section debrief: an executive summary, a method and data-quality statement, a reason-frequency table with count, percentage, and revenue-weighted percentage, a win-rate-by-competitor matrix, a ranked root-cause narrative with verbatim evidence quotes, a recommendations table mapping causes to owner teams, and a per-deal classification appendix that makes every call auditable.

The submission is a single SKILL.md file. Strong skills teach the agent a transferable procedure — how to read all deal records holistically before classifying any one of them, how to apply a loss-reason taxonomy with principled decision rules, how to anchor every claim to evidence, and how to communicate uncertainty honestly. Weak skills provide canned report templates or pre-filled example answers; those break the moment deal mix, segment distribution, or data quality changes.

A skill built to memorise specific scenarios will not hold up as challenges evolve.

You submit a SKILL.md file. A fixed base agent loads it and answers all 6 scenarios with it, then returns a scored breakdown. The submission that holds the top of the leaderboard accrues on-chain emissions for as long as it stays first.

We are looking for skills that demonstrate good design and help the agent perform like a capable practitioner in this domain.

Requirements (6 Tests)

Your skill is tested across three capability types, with 6 scenarios in total. Every submission is evaluated against all 6 scenarios — there is no sampling, so consistent quality across every capability type counts.

CategoryOverviewTest cases
Type A — Competitive & Taxonomic ClassificationThis type tests whether the agent can correctly distinguish loss reasons that look similar on the surface but require different owner actions. Scenarios present deals where buyer language could plausibly support more than one category — for example, a competitor mention that is actually evidence of a product gap, or a stalled deal that could be either a pricing objection or a no-decision. The skill must teach the agent to apply consistent, principled decision rules to assign a primary reason and up to two contributing reasons, and to anchor every classification to verbatim or near-verbatim evidence from the source deal records.1
Type B — Data Quality & CRM Signal DiagnosisThis type tests whether the agent treats missing or low-quality signal as a substantive analytical finding rather than an inconvenience to work around. Scenarios include deal records with sparse rep notes, missing CRM tags, or contradictory evidence that makes confident classification impossible. The skill must teach the agent to assign appropriate confidence tiers, surface the Unclassified rate prominently in its report, diagnose why signal is absent, and recommend a concrete process fix — not simply proceed as if the data were clean.1
Type C — Full Batch Debrief ReportThis type tests the agent's ability to synthesise an entire closed-deal batch into a complete, seven-section debrief that is immediately usable by a revenue operations or sales leadership team. Scenarios vary in batch size, segment mix, competitor presence, and data completeness. The skill must teach the agent to produce all required report sections — including the reason-frequency table, the win-rate-by-competitor matrix (only when the data support it), the ranked root-cause narrative, the recommendations table, and the per-deal appendix — while flagging analytic limitations such as small sample sizes or megadeal distortion clearly and proportionately.1

Surface details such as deal identifiers, ACV figures, company names, competitor names, and rep note phrasing rotate across variants, while the underlying analytical capability being tested remains the same.

Datasets

Available Datasets

When you submit, we run your SKILL.md through a standardised base agent provided by sundae_bar. The base agent is the same for every submission and is not supplied by you. It receives the challenge's test scenarios one at a time, calls the load_skill tool to read your skill, and follows it to answer each scenario.

Same base agent, same tests, only the submitted skill changes. Scores therefore reflect the quality of your skill, not agent configuration: the agent's answers are only as good as the procedure your skill teaches.

Graders then score each answer for output quality, correct skill use, and gaming behavior, and the per-scenario scores are averaged into your challenge score.

Important: the base-agent harness is fixed at evaluation time and its available tools are load_skill. You submit only a SKILL.md; you cannot supply your own agent, add tools, or alter the base agent's configuration.

Tip: write your skill as though you are onboarding a capable new hire who needs a procedure, not scripting a chatbot.

Evaluation Criteria

Each scenario is graded by an LLM judge. Final scores combine four graders and a set of deterministic penalties.

CriteriaWeight

Response quality

An LLM judge scores each response against a task-specific quality bar for this challenge — how correct, complete, and usable the output is on its own terms, independent of which procedure produced it.

30 %

Skill use

An LLM judge evaluates whether the agent's output actually followed the procedures your skill teaches, rather than freelancing or pretending to use the skill.

30 %

Scenario quality

An LLM judge evaluates how well each output meets the per-type quality bar, independent of whether your skill was followed. A reference good output anchors the judge's sense of the quality bar.

30 %

Output originality

A check that your skill drives genuine per-scenario reasoning rather than emitting near-identical, templated responses. Skills that reuse one canned output across scenarios lose credit here.

10 %

Responses are scored at the scenario level, and deterministic penalties apply to outputs that meaningfully damage report quality — for example, misclassifying a deal in a way that contradicts the available evidence, omitting required report sections, or presenting a revenue-weighted finding without flagging that a single large deal drives it. Scenario scores are averaged within each capability type, and capability-type averages are combined into the final challenge score shown on the leaderboard.

Submission

How to submit

1
Login or create a sundaebar.ai account

Create an account and add your Bittensor wallet ID to your profile to earn alpha rewards in the challenge.

2
Review the requirements

Review the reference skills (good, mediocre, gaming) to understand the quality bar and calibration points.

3
Build your skill

Draft your SKILL.md. Focus on teaching procedure rather than providing templates.

4
Test locally

Validate locally using the published dataset variants to sanity-check output quality.

5
Submit via the platform

Submit through the sundae_bar challenge portal by uploading your SKILL.md file.

6
Iterate

Review your scored breakdown and resubmit.

Constraints

Submission format: a single SKILL.md file with YAML frontmatter (name, description) and Markdown body.

Base agent: fixed. Provided by sundae_bar.

Runtime: a fixed base-agent harness loads your SKILL.md and answers all 6 scenarios per submission with it.

Tests per submission: 6 (the full scenario set — no sampling).

Submission limits: 3 submissions per day. A rolling challenge limit also applies.

Rewards

Emissions accrue on-chain to whichever submission currently holds the top of the leaderboard. There is no fixed per-submission payout: the current leader earns for as long as it stays first.

Time at the top takes all. A strong skill submitted early starts accruing immediately and keeps earning until another submission displaces it. Once you are displaced, accrual stops until you reclaim the top.

Rankings shift as others iterate. Get to the top early, then keep refining your skill to defend the position.

Your SKILL.md is scanned at submission time by a skill integrity gate. The gate checks that your skill teaches procedure rather than encoding scenario-specific content from our evaluation pool.

A skill that fails the integrity gate scores 0 for the submission. The gate runs deterministically and reports its finding as part of your results. This is how the evaluation framework stays credible: a high score should reflect a skill that genuinely teaches procedure, not one that memorised our test set.