Sundae Bar Logo
Past Challenge
Competitive Intelligence Strategy Briefing Skill
SBC24

Competitive Intelligence Strategy Briefing Skill

Overview

This skill teaches the agent to read a batch of competitor and market signals and produce a structured competitive watch brief that tells a decision-maker what is happening in their market, what it means for their position, and what they should do or decide next.

Given a set of sourced signals and a statement of the reader's competitive position, a good result is a layered brief: a sharp top-line read, signals sorted by strategic proximity with per-signal implication tags, a cross-signal synthesis section traceable to specific evidence, and an open questions log for anything ambiguous or thinly sourced — all produced without inventing signals or conclusions beyond what the inputs support.

The submission is a single SKILL.md file. Strong skills teach the agent a reasoning procedure it can apply to any signal batch: how to classify and tier signals, how to frame implications for a specific reader position, when to tag thesis relationships and when to skip that step entirely, and how to degrade the brief honestly when inputs are incomplete. Weak skills front-load templates or model answers that collapse the moment the signal set, reader position, or prior thesis changes.

A skill built to memorise specific scenarios will not hold up as challenges evolve.

You submit a SKILL.md file. A fixed base agent loads it and answers all 10 scenarios with it, then returns a scored breakdown. The submission that holds the top of the leaderboard accrues on-chain emissions for as long as it stays first.

We are looking for skills that demonstrate good design and help the agent perform like a capable practitioner in this domain.

Requirements (10 Tests)

Your skill is tested across 6 capability types, with 10 scenarios in total. Every submission is evaluated against all 10 scenarios — there is no sampling, so consistent quality across every capability type counts.

CategoryOverviewTest cases
Type A — Ambiguous Signal RoutingTests whether the agent correctly identifies signals that are ambiguous, thinly sourced, or represent a single leading-edge data point, and routes them to an open questions log rather than asserting a confident interpretation or producing hedged non-answers. The skill must teach the agent to phrase each routed item as a genuine, answerable question with a named resolution path.1–2
Type B — Cross-Signal SynthesisTests whether the agent's 'what this means for us' section makes a cross-signal pattern claim — identifying what multiple core signals collectively imply — rather than restating or summarising individual entries. The skill must teach the agent to anchor every synthesis claim to specific signals and to build toward an action or decision implication for the reader.1–2
Type C — Degraded Brief Under Missing PositionTests whether the agent degrades the brief honestly when no reader competitive position is available: producing implication-tagged signals and an open questions log, but omitting the synthesis section rather than fabricating a position or generating speculative 'what this means for us' content. The skill must teach the agent to recognise this condition and apply the correct downgraded structure without being prompted to do so.1–2
Type D — No-Thesis Signal AggregationTests whether the agent correctly skips the confirms/complicates/contradicts tagging step when no prior thesis is provided, rather than inventing a thesis to tag against or leaving placeholder tags in the brief. The skill must teach the agent to treat the absence of a prior thesis as a legitimate input state that changes the brief's structure, not a gap to paper over.1–2
Type E — Quiet Cycle and Core Signal DetectionTests whether the agent tiers signals by genuine proximity to the reader's competitive position — identifying which signals are truly core even when they are low-drama — and whether the agent's top-line read names the single most important development and what the reader should do or decide, including stating 'quiet cycle' plainly when nothing rises to that bar. The skill must teach the agent to reject funding size or narrative excitement as proxies for strategic relevance.1–2
Type F — Thesis Relationship JudgmentTests whether the agent accurately tags each core signal's relationship to a stated prior thesis as confirms, complicates, or contradicts — and uses 'contradicts' plainly when the evidence warrants it rather than softening to 'complicates' or hedging the call. The skill must teach the agent to apply these tags with analytical honesty and to distinguish the three categories by the nature of the evidential relationship, not the diplomatic stakes.1–2

Surface details — company names, signal specifics, funding figures, dates, and reader position particulars — rotate across variants, while the underlying competitive intelligence capability being tested stays the same.

Datasets

Available Datasets

When you submit, we run your SKILL.md through a standardised base agent provided by sundae_bar. The base agent is the same for every submission and is not supplied by you. It receives the challenge's test scenarios one at a time, calls the load_skill tool to read your skill, and follows it to answer each scenario.

Same base agent, same tests, only the submitted skill changes. Scores therefore reflect the quality of your skill, not agent configuration: the agent's answers are only as good as the procedure your skill teaches.

Graders then score each answer for output quality, correct skill use, and gaming behavior, and the per-scenario scores are averaged into your challenge score.

Important: the base-agent harness is fixed at evaluation time and its available tools are load_skill. You submit only a SKILL.md; you cannot supply your own agent, add tools, or alter the base agent's configuration.

Tip: write your skill as though you are onboarding a capable new hire who needs a procedure, not scripting a chatbot.

Evaluation Criteria

Each scenario is graded by an LLM judge. Final scores combine four graders and a set of deterministic penalties.

CriteriaWeight

Response quality

An LLM judge scores each response against a task-specific quality bar for this challenge — how correct, complete, and usable the output is on its own terms, independent of which procedure produced it.

30 %

Skill use

An LLM judge evaluates whether the agent's output actually followed the procedures your skill teaches, rather than freelancing or pretending to use the skill.

30 %

Scenario quality

An LLM judge evaluates how well each output meets the per-type quality bar, independent of whether your skill was followed. A reference good output anchors the judge's sense of the quality bar.

30 %

Output originality

A check that your skill drives genuine per-scenario reasoning rather than emitting near-identical, templated responses. Skills that reuse one canned output across scenarios lose credit here.

10 %

Scenarios are scored individually, and deterministic penalties apply for moves that damage output quality — such as fabricating signals not present in the input, inventing a reader position, omitting required brief sections when inputs are sufficient, or producing synthesis that cannot be traced to named signals. Individual scenario scores aggregate to capability-type averages, which roll up to the overall challenge score shown on the leaderboard.

Submission

How to submit

1
Login or create a sundaebar.ai account

Create an account and add your Bittensor wallet ID to your profile to earn alpha rewards in the challenge.

2
Review the requirements

Review the reference skills (good, mediocre, gaming) to understand the quality bar and calibration points.

3
Build your skill

Draft your SKILL.md. Focus on teaching procedure rather than providing templates.

4
Test locally

Validate locally using the published dataset variants to sanity-check output quality.

5
Submit via the platform

Submit through the sundae_bar challenge portal by uploading your SKILL.md file.

6
Iterate

Review your scored breakdown and resubmit.

Constraints

Submission format: a single SKILL.md file with YAML frontmatter (name, description) and Markdown body.

Base agent: fixed. Provided by sundae_bar.

Runtime: a fixed base-agent harness loads your SKILL.md and answers all 10 scenarios per submission with it.

Tests per submission: 10 (the full scenario set — no sampling).

Submission limits: 3 submissions per day. A rolling challenge limit also applies.

Rewards

Emissions accrue on-chain to whichever submission currently holds the top of the leaderboard. There is no fixed per-submission payout: the current leader earns for as long as it stays first.

Time at the top takes all. A strong skill submitted early starts accruing immediately and keeps earning until another submission displaces it. Once you are displaced, accrual stops until you reclaim the top.

Rankings shift as others iterate. Get to the top early, then keep refining your skill to defend the position.

Your SKILL.md is scanned at submission time by a skill integrity gate. The gate checks that your skill teaches procedure rather than encoding scenario-specific content from our evaluation pool.

A skill that fails the integrity gate scores 0 for the submission. The gate runs deterministically and reports its finding as part of your results. This is how the evaluation framework stays credible: a high score should reflect a skill that genuinely teaches procedure, not one that memorised our test set.