How QAmcp tests an MCP server

Everything it does, in order, and exactly how the score is calculated. Every result is a fact you can check yourself: pass or fail, a time in milliseconds, or an exact match. No AI judges anything.

What happens when you run it

A test takes about a minute. QAmcp talks only to the server you give it (and its login page, if it has one). Nothing goes anywhere else.

Connect, the way an AI app doesUsing the current MCP connection type, or the older one if that's all the server speaks. For a program on your computer (--stdio) it starts the program and talks to it directly.

Read the MCP version and the login setupIs the server on the newest MCP rules? If it needs a login, can an AI app find the login page and sign in properly?

List every toolNames, descriptions, inputs and safety labels, page by page, exactly as an AI app sees them.

Read every descriptionMissing or confusing descriptions, tools that read the same, hidden instructions aimed at the AI.

Decide which tools are safe to runOnly tools labeled read-only whose name and description agree. Anything that could change, send or delete something is skipped, and the report lists each one and why.

Run the safe tools, with good and bad inputsOnce with sensible sample inputs, then with a missing input and a wrong type. A good tool answers the first and refuses the others clearly.

Try tools togetherWhen one tool's answer holds something another tool asks for, like an ID, QAmcp passes it along and checks the second tool can use it. That's how AI apps chain tools in real jobs.

Time everything, and read every errorConnecting, listing and each answer are timed. Every error message is scanned for passwords, keys, tokens and internal file paths.

Open resources and promptsOnly if the server offers them: each one should open and match what it says it is.

Write the reportA score, every failure and advisory, everything it could not test and why. Add --html report.html for a page you can share.

Each check, what it looks at, and its points

Check	What it looks at	Points
Connects	Current and older MCP connection types, or a program on your computer	15
Every tool is described	Missing or vague descriptions, inputs with no description, tools that read the same	15
Ready for the newest MCP version	On the July 2026 version? Still on the old SSE connection?	10
Lists its tools	Duplicate or invalid names, a broken tool that stops AI apps loading the server	10
Tools are labeled safe or dangerous	Safety labels exist, and the tool's own name doesn't contradict them	10
No hidden instructions	Known tricks in descriptions: `<IMPORTANT>` blocks, "don't tell the user", invisible characters	10
Tools work with good inputs	Safe tools answer in time; no hangs, crashes or "success" with no data	10
Bad inputs give a clean error	A missing or wrong-type input gets a clear refusal	10
Login setup follows the spec	Only when the server needs a login: the details AI apps need to sign in	10
Resources and prompts work	Only when the server has them: each opens, and its content matches its type	10
Tools work together	Only when tools chain: an ID one tool returns is accepted by the next	10
Fast enough	Connecting, listing and answering, timed	5
No secrets leaked	Keys, passwords, server folders and stack traces in error messages	5

Check

What it looks at

Points

Connects

Current and older MCP connection types, or a program on your computer

Every tool is described

Missing or vague descriptions, inputs with no description, tools that read the same

Ready for the newest MCP version

On the July 2026 version? Still on the old SSE connection?

Lists its tools

Duplicate or invalid names, a broken tool that stops AI apps loading the server

Tools are labeled safe or dangerous

Safety labels exist, and the tool's own name doesn't contradict them

No hidden instructions

Known tricks in descriptions: <IMPORTANT> blocks, "don't tell the user", invisible characters

Tools work with good inputs

Safe tools answer in time; no hangs, crashes or "success" with no data

Bad inputs give a clean error

A missing or wrong-type input gets a clear refusal

Only when the server needs a login: the details AI apps need to sign in

Resources and prompts work

Only when the server has them: each opens, and its content matches its type

Tools work together

Only when tools chain: an ID one tool returns is accepted by the next

Fast enough

Connecting, listing and answering, timed

No secrets leaked

Keys, passwords, server folders and stack traces in error messages

These add up to 130, but the score is a percentage, not a sum: most servers have no login, no resources and no chained tools, so 100 points are in play. Each of those three checks that applies adds 10 points in play, and the total is scaled to 100 the same way.

Plain arithmetic, so you can check it yourself

score = points earned ÷ points tested × 100

Example: a server with a login and resources has 120 points in play. It earns 108 of them, so 108 ÷ 120 = 90%, and its score is 90/100.

A pass earns all the pointsA check with an advisory earns half; a failed check earns none.

What couldn't run doesn't countThe score is points earned out of points actually tested. A server isn't punished for what QAmcp couldn't try.

Read the score with its coverageEvery score comes with a line like Tested 10 of 10 checks · ran 5 of 9 tools. A 90 from 2 of 20 tools says much less than a 90 from all 20.

"Not scored" is not a failA login wall, or a server that asks us to slow down (HTTP 429), means nothing could be fairly scored. Add --login or try again in a minute.

Serious problems put a ceiling on the score

However good the rest is. The lowest ceiling wins, and the report says so: "Would be 85, but capped at 50 because…"

Problem	Highest possible score
Hidden instructions in a tool, resource or prompt description	40
A secret (key, password, token) leaked in an error message	50
A broken tool definition that can stop AI apps loading the server	50
A tool that crashes or hangs, or claims success but returns nothing	70

Problem

Highest possible score

Hidden instructions in a tool, resource or prompt description

A secret (key, password, token) leaked in an error message

A broken tool definition that can stop AI apps loading the server

A tool that crashes or hangs, or claims success but returns nothing

90+ solid70–89 works, with things to fixbelow 70 AI apps will likely struggle, or there's a safety problem

You choose how much QAmcp may touch

A tool's safety label is the server's own word, so QAmcp double-checks it against the tool's name and description. Tools that sound destructive, like delete_account, never run in any mode.

Default: safe modeRuns only tools labeled read-only whose name and description agree. Right for any server, yours or someone else's.

--look-only: run nothingCalls no tools at all; reads the tool list, descriptions, labels and login setup. For a server you don't trust. Tools, bad inputs and leaks are not tested.

--owner: your own serverAlso runs unlabeled tools whose names clearly only read (get_, list_, search_…). Tools that say they write, send or create are still skipped.

--run-unlabeled: your test copyRuns every tool not named or labeled dangerous. Use it on a test copy of your own server, never on live data.