Inbound Email Triage & Reply Drafting Skill
Overview
Given a batch of inbound business emails — each with a sender, subject, and body, and sometimes thread history or relationship context — the agent must produce a structured triage table covering every message and a set of reply drafts for every email that warrants one.
Your submission is a single SKILL.md file that teaches a fixed base agent how to do this reliably across batches it has never seen. The skill defines the reasoning procedure the agent follows: how to read each email, how to classify it, how to decide what action it requires, and how to write a response that fits the situation.
A strong skill teaches procedure. It shows the agent how to recognise the type of email it is reading, how to weigh sender relationship and content stakes to assign urgency, how to select the right tone and length before writing a single word of a draft, and how to anchor every reply and action step in the specific details of the email rather than producing a generic response. A weak skill provides fill-in-the-blank templates or example replies to memorise; it fails the moment sender names, invoice numbers, deadlines, or relationship context change.
The skill must also cover the judgment calls that separate professional triage from mechanical sorting: knowing when an email is legally sensitive and should receive a minimal, non-committal draft rather than a confident reply; knowing when an action belongs to a delegate and should be routed with a rationale rather than drafted in full; knowing when an email warrants no reply at all and should simply be archived.
A skill built to memorise specific scenarios will not hold up as challenges evolve.
You submit a SKILL.md file. A fixed base agent loads it and answers all 10 scenarios with it, then returns a scored breakdown. The submission that holds the top of the leaderboard accrues on-chain emissions for as long as it stays first.
We are looking for skills that demonstrate good design and help the agent perform like a capable practitioner in this domain.
Requirements (10 Tests)
Your skill is tested across 7 capability types, with 10 scenarios in total. Every submission is evaluated against all 10 scenarios — there is no sampling, so consistent quality across every capability type counts.
| Category | Overview | Test cases |
|---|---|---|
| Type A — Batch Triage Table Production | Tests whether the agent can process an entire email batch in one pass and produce a well-formed triage table with a row for every message. The skill must teach the agent to populate intent, urgency, recommended action, and reply-needed flag for each row without skipping or conflating messages, even in large or mixed batches. | 1–2 |
| Type B — Contextually Specific Reply Drafting | Tests whether reply drafts reference the actual details of the inbound email — project names, invoice numbers, dates, specific requests — rather than producing interchangeable acknowledgments. The skill must teach the agent to anchor every draft in the content of the email it is responding to. | 1–2 |
| Type C — Detail Grounding & Concrete Commitments | Tests whether the agent grounds drafts in email-specific details and closes every action-required reply with a concrete next step that names an owner and timeline. The skill must teach the agent to treat vague follow-up language as a failure mode and to substitute specific, attributable commitments wherever the email calls for action. | 1–2 |
| Type D — High-Stakes & Legal Sensitivity Judgment | Tests whether the agent correctly identifies emails that carry legal, financial, or reputational risk and responds differently to them than to routine correspondence. The skill must teach the agent to flag these cases for careful human review, assign appropriate urgency by weighing relationship and content stakes together, and produce drafts that are minimal and non-committal rather than confident or comprehensive. | 1–2 |
| Type E — Mixed-Batch Triage Under Sensitive Conditions | Tests whether the agent maintains triage discipline across a heterogeneous batch that includes at least one high-stakes or sensitive email alongside routine messages. The skill must teach the agent to apply its highest-caution judgment to flagged emails without letting that caution bleed into — or be absent from — the rest of the batch. | 1–2 |
| Type F — Action Ownership & Next-Step Commitment | Tests whether every action-required reply closes with a clear, attributed next step rather than an open-ended deferral. The skill must teach the agent to identify what action is needed, who owns it, and by when — and to state that concretely in the draft body. | 1–2 |
| Type G — Urgency Calibration | Tests whether the agent assigns urgency tiers that reflect the combined weight of sender relationship, explicit or implied deadline, and content stakes — not just the most prominent surface signal. The skill must teach the agent to reason through all three factors before committing to a same-day, 48h, defer, or archive classification. | 1–2 |
Surface details — sender names, company names, invoice numbers, project names, deadlines, and relationship descriptors — rotate across variants, while the underlying capability being tested remains the same.
Datasets
Available Datasets
When you submit, we run your SKILL.md through a standardised base agent provided by sundae_bar. The base agent is the same for every submission and is not supplied by you. It receives the challenge's test scenarios one at a time, calls the load_skill tool to read your skill, and follows it to answer each scenario.
Same base agent, same tests, only the submitted skill changes. Scores therefore reflect the quality of your skill, not agent configuration: the agent's answers are only as good as the procedure your skill teaches.
Graders then score each answer for output quality, correct skill use, and gaming behavior, and the per-scenario scores are averaged into your challenge score.
Important: the base-agent harness is fixed at evaluation time and its available tools are load_skill. You submit only a SKILL.md; you cannot supply your own agent, add tools, or alter the base agent's configuration.
Tip: write your skill as though you are onboarding a capable new hire who needs a procedure, not scripting a chatbot.
Evaluation Criteria
Each scenario is graded by an LLM judge. Final scores combine two graders and a set of deterministic penalties.
| Criteria | Weight | |
|---|---|---|
Skill use An LLM judge evaluates whether the agent's output actually followed the procedures your skill teaches, rather than freelancing or pretending to use the skill. | 50 % | |
Scenario quality An LLM judge evaluates how well each output meets the per-type quality bar, independent of whether your skill was followed. A reference good output anchors the judge's sense of the quality bar. | 50 % |
Submissions are evaluated against held-out scenario variants. Deterministic penalties apply when agent outputs include moves that materially damage output quality — such as omitting rows from the triage table, producing generic drafts that ignore email-specific details, or mishandling high-stakes emails in ways that could cause real-world harm. Individual scenario scores roll up to capability-type averages, which in turn combine into the final challenge score shown on the leaderboard.
Submission
How to submit
Create an account and add your Bittensor wallet ID to your profile to earn alpha rewards in the challenge.
Review the reference skills (good, mediocre, gaming) to understand the quality bar and calibration points.
Draft your SKILL.md. Focus on teaching procedure rather than providing templates.
Validate locally using the published dataset variants to sanity-check output quality.
Submit through the sundae_bar challenge portal by uploading your SKILL.md file.
Review your scored breakdown and resubmit.
Constraints
Submission format: a single SKILL.md file with YAML frontmatter (name, description) and Markdown body.
Base agent: fixed. Provided by sundae_bar.
Runtime: a fixed base-agent harness loads your SKILL.md and answers all 10 scenarios per submission with it.
Tests per submission: 10 (the full scenario set — no sampling).
Submission limits: 3 submissions per day. A rolling challenge limit also applies.
Rewards
40% Emissions accrue on-chain to whichever submission currently holds the top of the leaderboard. There is no fixed per-submission payout: the current leader earns for as long as it stays first.
Read this article to learn more: How Rewards are Earned on SN121 Skill Challenges
Time at the top takes all. A strong skill submitted early starts accruing immediately and keeps earning until another submission displaces it. Once you are displaced, accrual stops until you reclaim the top.
Rankings shift as others iterate. Get to the top early, then keep refining your skill to defend the position.
Your SKILL.md is scanned at submission time by a skill integrity gate. The gate checks that your skill teaches procedure rather than encoding scenario-specific content from our evaluation pool.
A skill that fails the integrity gate scores 0 for the submission. The gate runs deterministically and reports its finding as part of your results. This is how the evaluation framework stays credible: a high score should reflect a skill that genuinely teaches procedure, not one that memorised our test set.