Skill Creator: Build, Test and Check Your Own Claude Skill (Nov 2026)

Skill creator (skill-creator) is Anthropic's free, open-source skill for building other skills. In Claude Code, run /plugin install skill-creator@claude-plugins-official, then ask Claude to create a skill. It interviews you, writes the SKILL.md, tests it against a run without the skill, and tunes the description.

This guide walks through that loop one step at a time. It ends with a security checklist for the skill you build, because other people will run it on their machines.

What skill-creator does comes from its own SKILL.md and scripts, and from the Claude Code skills docs, all checked on 8 November 2026. New to skills? Start with Claude Code skills: what a skill is, how it loads and how to install one.

What is skill creator?

skill-creator is itself a skill. Its front matter describes it as a tool to "Create new skills, modify and improve existing skills, and measure skill performance." It's Apache 2.0 licensed and lives in two places with the same SKILL.md:

• anthropics/skills, Anthropic's public skills repo.
• The skill-creator plugin in the official Claude Code marketplace, claude-plugins-official. This is the one you install.

Its own summary of the job:

1. Decide what the skill should do and roughly how.
2. Write a draft.
3. Run Claude with the skill on a few test prompts.
4. Review the results with you, by eye and with numbers.
5. Rewrite the skill from your feedback, and repeat.
6. Grow the test set and try again at a larger scale.

It meets you where you are. If you only have an idea, it starts with questions. If you already have a draft, it goes straight to testing. And if you say you don't want evals, the skill tells Claude it can skip them: "if the user is like 'I don't need to run a bunch of evaluations, just vibe with me', you can do that instead."

What you need before you start

• Claude Code, in a terminal, the VS Code extension or the desktop app. The full loop (parallel test runs, baselines, the browser viewer, description tuning) is built for it.
• Python 3.10 or later on your machine. The helper scripts are Python; quick_validate.py also needs PyYAML.
• The claude command on your PATH if you want description tuning. Its scripts call claude -p for each test query.
• Some time and tokens. Each test case runs twice (with and without the skill), and description tuning runs every query three times per round.

It also has instructions for claude.ai, where there are no subagents: Claude runs each test itself, one at a time, skips the baseline and the benchmark, and shows results in the chat. Description tuning is Claude Code only.

How to install skill-creator

In a terminal, start Claude Code with claude, then type:

/plugin install skill-creator@claude-plugins-official

In the VS Code extension or the desktop app, install it from the official marketplace through the plugin manager.

If the install fails, the docs match each error message to a fix:

• Marketplace "claude-plugins-official" not found: run /plugin marketplace add anthropics/claude-plugins-official, then retry.
• Plugin "<name>" not found in marketplace "claude-plugins-official": your copy of the catalog is old. Run /plugin marketplace update claude-plugins-official, then retry.

If the summary says Run /reload-plugins to apply., Claude Code runs the reload for you. If that reload warns it would re-read the conversation, run /reload-plugins --force.

Build a skill with skill-creator, step by step

Say what you want in plain words, for example: "Create a skill that turns my git log into release notes in our changelog format." Then work through these steps. You can stop after any of them.

Step 1 of 8: Answer four questions

skill-creator starts by pinning down your intent:

1. What should this skill enable Claude to do?
2. When should it trigger? (What would you type?)
3. What's the expected output format?
4. Should we set up test cases to check it works?

On the last one, its rule of thumb: skills with outputs you can check (file changes, data extraction, code, fixed steps) benefit from tests; subjective ones (writing style, art) often don't. If you've just done the task in the same chat, say "turn this into a skill". It pulls the steps, tools and your corrections from the conversation first.

Step 2 of 8: Let it ask about edge cases

Next it asks about edge cases, input and output formats, example files, what "done" looks like, and dependencies. It waits until these are settled before writing test prompts. Have one or two real example inputs ready.

Step 3 of 8: Review the draft SKILL.md

It writes the skill folder in the standard layout:

release-notes/
├── SKILL.md        (required: front matter + instructions)
├── scripts/        (optional: code for repetitive, exact work)
├── references/     (optional: docs Claude reads only when needed)
└── assets/         (optional: templates, icons, fonts)

Read the draft. Three things matter most:

• The description. It's what Claude reads every turn to decide whether to use the skill. skill-creator writes it on purpose a little "pushy", because Claude tends to "undertrigger" skills. See write a description that triggers.
• Length. Its guidance and the docs agree: keep SKILL.md under 500 lines, and move long reference material into separate files.
• Reasons, not shouting. It explains why each instruction matters instead of writing rules in capitals: "If you find yourself writing ALWAYS or NEVER in all caps ... that's a yellow flag."

Step 4 of 8: Agree on two or three test prompts

It suggests two or three realistic prompts, the kind a real user would type, and asks you to confirm them. They go in evals/evals.json inside the skill folder:

{
  "skill_name": "release-notes",
  "evals": [
    {
      "id": 1,
      "prompt": "Write release notes for everything since v2.3.0",
      "expected_output": "Grouped by Added / Fixed / Changed, one line per change",
      "files": []
    }
  ]
}

Step 5 of 8: Run each test with and without the skill

For every prompt, it starts two subagents at once: one with the skill, one without (for an existing skill, the baseline is a snapshot of the old version). Each subagent starts with a clean context, which matters: what you said while writing the skill would hide gaps in it.

Results go in a release-notes-workspace/ folder next to the skill, one iteration-N/ folder per round. While the runs work, it drafts checks (assertions) for each prompt, then grades every run against them. It saves pass or fail, with evidence, in grading.json. It also adds up pass rate, time and tokens, with and without the skill, in benchmark.json.

Step 6 of 8: Review the results in the viewer

It opens an HTML viewer with two tabs:

• Outputs: one test at a time, with the prompt, what the skill produced, and a feedback box that saves as you type.
• Benchmark: pass rates, time and tokens for each setup.

Click Submit All Reviews when you're done. Your notes are saved to feedback.json, and the next round reads them. An empty note means "this one is fine". With no display, the viewer writes a static HTML file instead, and feedback downloads as a file.

Step 7 of 8: Improve, rerun, repeat

It rewrites the skill from your feedback and reruns everything into the next iteration-N/ folder, baselines included. The goal is a skill that works across many prompts, not just your three: it's told to avoid "overfitty changes" and to cut instructions that aren't pulling their weight. It stops when you're happy, when all feedback is empty, or when it stops making progress.

Step 8 of 8: Tune the description

Once the skill works, it offers to tune when the skill triggers:

1. It writes about 20 test queries: 8 to 10 that should trigger the skill and 8 to 10 near-misses that shouldn't.
2. You edit them in an HTML page and export the set.
3. A script splits them 60/40 into train and test sets, runs each query three times, proposes a better description from the misses, and repeats up to five times.
4. It picks the best_description by its score on the test set, not the training set, so the wording isn't overfitted. It shows you the before and after.

Improve a skill you already have

You don't have to start from scratch. Point it at an existing skill, for example "evaluate my summarize-changes skill with skill-creator". It copies the skill to a snapshot first, uses that as the baseline, and runs steps 4 to 8.

For a skill that ships in a plugin, Claude Code also has claude plugin eval, which runs each prompt in a clean session with and without the plugin and can fail a CI build below a threshold. The docs warn that its format and skill-creator's evals/evals.json "aren't interchangeable".

Write a description that triggers

The description decides whether your skill ever runs. Rules from skill-creator and the docs:

• Say what it does and when to use it. All the "when" goes in the description, not the body.
• Lead with the main use case. Claude Code cuts the combined description and when_to_use text at 1,536 characters in the skill listing.
• Name the phrases people actually type, including ones that don't use your skill's name.
• Skip angle brackets. Claude Code shows < and > as < and >, and skill-creator's validator rejects them. Write [branch] instead of <branch>.
• Keep the name short and kebab-case: lowercase letters, digits and hyphens, 64 characters at most.

A note from skill-creator on how triggering works: Claude "only consults skills for tasks it can't easily handle on its own". A one-step request like "read this PDF" may not trigger even a perfect description.

Front matter that breaks packaging or upload

This one catches people. Claude Code accepts many front matter fields, but the open Agent Skills spec allows only six. skill-creator's package_skill.py and quick_validate.py follow the spec, and so do claude.ai uploads and the Skills API.

Where the skill goes	Front matter you can use
Claude Code (personal, project or plugin skills)	Every Claude Code field, e.g. disable-model-invocation, argument-hint, when_to_use
.skill package, claude.ai upload, Skills API	name, description, license, compatibility, metadata, allowed-tools

Any other field stops packaging with an error that starts Unexpected key(s) in SKILL.md frontmatter. If the skill is only for Claude Code, you don't need to package it: share the folder or ship it in a plugin.

To share a skill, the docs list three routes: commit .claude/skills/ to your repo, put it in a plugin's skills/ folder, or deploy it through managed settings. For claude.ai, zip the folder (the skill folder at the root of the zip) and upload it under Customize > Skills.

Frequently asked questions

What is the skill creator skill?

skill-creator is a skill from Anthropic that builds other skills. It interviews you, drafts the SKILL.md, runs test prompts with and without the skill, shows the results in a viewer, and tunes the description so the skill triggers on the right requests.

Is skill-creator free?

Yes. It's open source under the Apache 2.0 license, in the anthropics/skills repo and the official claude-plugins-official marketplace. You pay only for the tokens its test runs use.

How do I install the skill creator skill?

In Claude Code, run /plugin install skill-creator@claude-plugins-official. If Claude Code can't find the marketplace, add it first with /plugin marketplace add anthropics/claude-plugins-official. Then ask Claude to create or evaluate a skill.

Does skill-creator work in claude.ai?

Its instructions cover claude.ai, with less: there are no subagents, so Claude runs the tests itself one at a time, skips the baseline and the benchmark, and asks for feedback in the chat. Description tuning needs claude -p, so it's Claude Code only.

Do I have to run evals?

No. skill-creator suggests tests for skills with outputs you can check and often skips them for subjective ones like writing style. You can also just say you don't want them.

Where does skill-creator save test results?

Test prompts go in evals/evals.json inside the skill. Runs, grades, benchmarks and feedback go in a <skill-name>-workspace/ folder next to the skill, one iteration-N/ folder per round.

Why does packaging fail with "Unexpected key(s) in SKILL.md frontmatter"?

The packager follows the Agent Skills spec, which allows only name, description, license, compatibility, metadata and allowed-tools. Remove Claude Code-only fields such as disable-model-invocation, or share the folder or a plugin instead of a .skill file.

Can skill-creator improve a skill I already have?

Yes. Ask it to evaluate the skill by name. It snapshots the current version as the baseline, runs your test prompts against both, and proposes changes.

How do I create a skill with skill creator?

Install the plugin, then tell Claude what you want, for example "create a skill that turns my git log into release notes". It asks four questions about intent, writes the SKILL.md, runs your test prompts with and without the skill, shows the results in a viewer, and rewrites the skill from your feedback. You can stop after any step.

Is skill-creator safe to use?

It comes from Anthropic's official marketplace, and its Python scripts are public, so you can read them before you install. It spawns subagents and runs those scripts on your machine, and test runs cost tokens. Try it in a throwaway project first if you want to watch what it does.