How we test

The rules below are the product. Everything else on this site is their output.

Who runs SkillsTested

SkillsTested is an independent testing project run by Fred, an independent builder. Fred installs each package, runs the disclosed task and environment, checks the resulting artifact, and writes the finding against the saved evidence. SkillsTested does not accept payment for placement or a favorable verdict. It is not affiliated with Anthropic or the skill authors.

Corrections are welcome through Fred's public GitHub profile. A correction changes the current page and is recorded in the changelog; it does not erase the original test boundary.

1. We publish the verification level

No page ships until the package has been installed in a clean environment. From there we state exactly how far the evidence goes: install verified, function verified, effect verified, or workflow verified. A package-structure inspection is not presented as a completed user outcome; reading a README is research, not verification.

2. The environment is disclosed, always

Every test record states the OS, Node, npm and Python versions it ran on, and the exact upstream commit tested. Our default box is a headless Linux sandbox with two cores — closer to CI and to cheap servers than to a maxed-out MacBook, which is why we catch failures that Mac-based reviews never see. When a skill needs a desktop environment we say precisely where our verification stopped, rather than pretending it went further.

3. Timings are measured, not estimated

Install times come from wall-clock measurement around the actual command. One-off costs (CLI bootstrap, dependency install) are reported separately from the skill's own install so you can predict your second install, not just your first.

4. Failures are content

Every failure is recorded verbatim — the step, the error, and the fix if we found one, as a three-part entry in the test log. A skill with zero failures gets that stated explicitly, because "nothing broke" is information too. We do not penalize a skill for our environment's constraints; we document the boundary honestly and move on.

5. Current facts have one source

Test records live as structured JSON. Tested blocks, current star counts, install times, failure counts and the front-page summary are generated from those records by the site build script. Narrative history remains prose, but is explicitly dated. Current facts are never duplicated into prose; star counts always carry the date they were read.

6. Retests are scheduled, not aspirational

Skills ecosystems move weekly, but a single calendar hides risk. Our discovery and freshness policy checks P0 metadata daily and repeats the full hands-on test every 14 days; P1 runs at 7/30 days, P2 at 14/60, and P3 at 30/90. Security, archive, license, installer, permission, content-hash, and published-Path changes bypass the calendar. Every retest appends to the changelog, and every page keeps its exact tested date.

7. A tested skill is not automatically a tested path

A multi-skill path earns workflow_verified only after one fixed input travels through every required stage, every handoff is preserved, the final artifact passes its mechanical verifier, and the complete run is published. Until then the path is labeled partial or needs_retest, even when every individual package installed successfully. The path builder selects only published path IDs; it does not let a language model invent skills, commands, evidence, or verification status.

8. Planner behavior is tested offline before release

The Planner API runs a deterministic parser and matcher against the same build-generated Path catalog used by the page. Its supported, clarification-required, and not-tested boundaries are frozen in a 60-goal English and Chinese fixture set. The endpoint makes zero model calls, does not store goal text, rejects cross-origin and oversized requests, and has an in-browser catalog fallback. An API response can select a published Path; it cannot create a new one.

9. Discovery is broad; eligibility stays narrow

We search direct GitHub sources, official collections, Claude plugin marketplaces, skills.sh, ClawHub, and multiple community indexes on independent schedules. Demand determines which eligible candidate is tested first, not which one wins. Before execution we resolve the canonical source, pin the commit, inspect license, duplication, scripts, hooks, permissions, network and secret access, then run the same fixed brief and controls. Counters from different platforms remain separate. The complete source map, queue rules, update frequencies, and known blind spots are published on the discovery radar.

What we are not

Not an unreviewed mirror of every directory result, not affiliated with Anthropic or with any skill author, and not paid for placement. The discovery radar is an intake queue, never a catalog of endorsements. If a skill author disputes a finding, we retest and publish the correction in the changelog — corrections are content here, not embarrassments.