Conformance is observable behavior
UHP does not define conformance as “we implement similar endpoints” or a self-declaration. The official suite sends requests to a live server and verifies required behavior, including real agent tasks.
| Class | Current checks | Main coverage |
|---|---|---|
| Core | 37 | Discovery, version negotiation, auth, errors, harness/model discovery, execution, streaming, sessions and cancellation. |
| Extended | +8 | Session inspection, file input, artifacts, download headers and traversal probes. |
| Full | +7 | Harness lifecycle, unsupported-base refusal, skill bundles, MCP and disabled-tool persistence. |
Total: 52 checks in the current repository state re-checked on 22 August 2026. The check count did not change in the latest suite revision.
Suite revision 2026.8.11.post1: how to read a report
The conformance suite carries its own revision, currently 2026.8.11.post1. It hardens the JSON report — the artifact published as evidence for a conformance claim, often read by people who did not run it — so that a skip can no longer disappear into a green verdict. The UHP protocol itself remains 2026-08-11; this is a change to how conformance evidence is reported, not a protocol release.
| Report field | Meaning since 2026.8.11.post1 |
|---|---|
conformant | Strict: true only when every check ran and none failed or errored. A single skip makes it false. |
conformant_with_skips | The older, weaker meaning under an honest name: nothing failed or errored, but some checks did not run. |
skipped_not_verified | Enumerates by id every check the report establishes nothing about. |
suite_version | The suite revision that produced the report (currently 2026.8.11.post1). |
generated_at | UTC timestamp of the run, so evidence dates itself. |
highest_class_passed | Strict: failures, errors, skips and classes with no results at all each break the ladder. |
A report without suite_version and generated_at predates suite 2026.8.11.post1. The conformance reports checked into the repository (0.3.0, 0.7.0, 0.8.0) are artifacts of that older shape and are explicitly left as they were; read them with the stricter vocabulary above in mind rather than assuming their green summaries meant every check ran.
disabledTools persist through harness configuration. Passing that check does not establish that tool disabling is enforced at runtime in every execution path; runtime enforcement is a distinct property the conformance check does not verify.Why schema validation is not enough
Some protocol failures only appear during execution. A stream can have the correct media type and still buffer every event until the task is over. A cancellation endpoint can return success while the underlying agent keeps running. An artifact route can match its schema while allowing unsafe path traversal.
The conformance suite therefore includes live behavioral checks such as progressive stream delivery, cancellation reaching a terminal state, artifact security headers and path traversal probes.
Current reference result
The repository records HarnessRouter Community Edition 0.3.0 as passing the Full class against UHP 2026-08-11, with 52 of 52 checks passed in the documented run. More recently, the project published additional Full-class 52/52 runs through newer backend paths: v0.7.0 through the Pi backend and v0.8.0 through DeepSeek Harness (dsh).
After suite revision 2026.8.11.post1, the HarnessRouter repository also documents repaired full-class runs in commit messages rather than new checked-in report files: a Hermes path restored to 52/52 across four runs (three via TokenRouter, one via LLMTR) after relay normalization, and a DeepSeek Harness path via LLMTR improved from 47/52 to 52/52 after reasoning_effort compatibility handling. On 23 August 2026, tag v0.8.2 added another commit-documented result: a Full-class 52/52 run executed on the session-isolation ("write-wall") tree. The suite revision remains 2026.8.11.post1, the protocol remains 2026-08-11, and the total remains 52 checks; no post1-format report JSON has been checked in for any of these runs, so commit-documented evidence remains weaker than a published report file.
The gateway also fixed an envelope defect the suite exposed: a failed Response used to emit an error object without a spec-valid type, so every failed turn was schema-invalid on top of its failure (error envelope reference).
What to demand from a third-party claim
- Protocol version named explicitly.
- Conformance class stated: Core, Extended or Full.
- A reproducible run or machine-readable report that names its
suite_versionandgenerated_at. - No skipped tests presented as passes — since suite
2026.8.11.post1, a strict report distinguishesconformantfromconformant_with_skipsand listsskipped_not_verifiedids. - Independent verification where the claim matters commercially.
If you are building or evaluating a UHP server, the getting-started guide walks the minimum implementation path from capability discovery through the conformance suite.