KV REUSE / CORRECTNESS
Verifier documentation
Verify whether reused KV produces exactly the same generated token IDs as recomputation with caching disabled.
Start with the complete L4 quickstart →
How the comparison works
The verifier starts a cache-disabled control, then each candidate in separate sequential processes. Each probe has a cold answer followed by reusing answers. It compares exact generated token IDs against the disabled control and cold reference, recording the first differing index. Text similarity is not a substitute.
Unknown hit counts, zero-hit reuse, missing answers, or fallback cannot establish a passing reuse verdict. Native vLLM prefix caching is a real candidate with no KV connector; the separate control always disables caching.
Supported setup
The current evidence contract supports Linux with an NVIDIA GPU, vLLM 0.28.0, and stock LMCache 0.5.4/0.5.5rc5 or native vLLM prefix caching. The bounded acceptance fixture uses LMCache 0.5.4 and one NVIDIA L4. Other runtime contracts require explicit support.
Cache targets and connectors
A target is the cache configuration under test. The control is not a target: it always runs in its own process with kv_transfer_config=None and native prefix caching off. Anything not listed as supported fails preflight with an explicit error rather than quietly running a different path.
| Target | How to select it | Status |
|---|---|---|
| Stock LMCache connector | --kv-connector LMCacheConnectorV1 --lmcache-config <file> | Supported on LMCache 0.5.4 and 0.5.5rc5. Must resolve to vLLM's in-tree connector module; native prefix caching is forced off, and kv_load_failure_policy=fail so a failed load is an error instead of a silent recomputation. |
| Native vLLM prefix caching | --kv-connector none (native caching defaults on) | Supported, and a real reuse target: it must show nonzero hits and pass the same control comparison. |
| Control-only diagnostic | --kv-connector none --no-native-prefix-caching | Supported. Both caches are off, so the best possible result is control_only_pass with reuse_tested=false. It verifies no reuse. |
| Paired acceptance (both targets) | --suite acceptance --compare-native --fixture <file> | Supported. Both candidates share one set of disabled controls inside a fixed 24-generation budget. |
| Any other connector class, or an out-of-tree module | --kv-connector <other>, --kv-connector-module-path | Rejected. Only the stock LMCacheConnectorV1 module is accepted; the loaded class is checked again after the engine starts. |
| Connector extra configuration | --kv-connector-extra-config <file> | Rejected when non-empty: no supported evidence contract for it yet. |
| LMCache blending or PD mode | enable_blending, enable_pd in the YAML | Rejected. Both are outside this verifier's exact-reuse boundary. |
| Connector plus native caching together | --kv-connector LMCacheConnectorV1 --native-prefix-caching | Rejected. Two caches at once is not an isolated target, so a result could not be attributed. |
| LMCache GPU connector v3 | use_gpu_connector_v3: true in the YAML | Not a workaround. On the recorded L4 it aborts with a tensor-dimension mismatch rather than producing a comparable result. |
Package versions are checked before any engine starts: vLLM must be 0.28.0, and LMCache 0.5.4 or 0.5.5rc5 when a connector is selected. Model and tokenizer revisions must be immutable 40-character commits, or a local directory whose contents are hashed into the report.
Command-line options
| Option | Default | Notes |
|---|---|---|
--model, --model-revision | required | No model-specific default. Both the requested and resolved identity are recorded. |
--tokenizer, --tokenizer-revision | the model and its revision | Overridable independently and recorded separately. |
--kv-connector | required | LMCacheConnectorV1 or none. See the target table above. |
--lmcache-config | — | Required with a connector, rejected with none. Effective installed settings and environment overrides are resolved, not just the file's defaults. |
--suite | standard | standard (24 probes, 96 generations), smoke (6 probes, 24), or acceptance (the paired 24-generation fixture). |
--prompt-format | both for standard, chat otherwise | raw, chat or both. A missing chat template is a preflight error, never a silent fall back to raw. |
--reuse-counts | 1,3 | Distinct positive integers. Tests repeated retrieval, not just the first warm request. |
--max-tokens, --seed | 16, 0 | Greedy decoding at temperature 0, one candidate. Every effective sampling field is recorded. |
--dtype, --gpu-memory-utilization | float16, 0.45 | The utilization must be strictly between 0 and 1. |
--max-model-len, --block-size | model limit, 16 | Every prompt plus its output budget is checked against the limit during preflight. |
--chunk-size | — | With LMCache, asserts its resolved chunk size. With none, sets the probe span as a multiple of the block size. |
--tenant | isolated-verifier | Included in every probe's cache scope. No cross-tenant reuse is implied or tested. |
--report | required | JSON destination. An existing report or evidence directory is refused rather than overwritten. |
--dry-run | off | Resolves configuration and prints the probe manifest without starting an engine. Not a passing verification. |
Read a result
| Exit | Meaning |
|---|---|
| 0 | Completed checks passed for the requested run. This is a bounded result. |
| 1 | A correctness check failed. In the paired L4 fixture this is expected for LMCache. |
| 2 | Setup, execution or evidence failure. Inspect errors; incomplete probes are not passes. |
Evidence retained
- Full engine and connector configuration, model revision, package versions, GPU name and compute capability.
- Submitted and generated token IDs, per-answer cached-token counts and first differing token index.
- Target verdicts, errors, completion states, acceptance comparison and actual generation count.
- Sibling evidence files and engine logs, including connector warnings. Preserve the entire report directory.
Reports use the schema identifier efim.kv-reuse-verification.v1 and are written atomically as each probe completes, so an interrupted run still leaves valid JSON. Every report carries claim_eligible=false. No latency, cost or throughput claim is made.
Acceptance status
The paired L4 acceptance at revision 54a0af2ae8db9f8b303be0a9a0683ba8128e7121 was recorded on 2026-09-10 on one NVIDIA L4. All 24 generations completed: four disabled controls, ten LMCache answers and ten native answers. LMCache failed all four probes and native prefix caching passed all four; the overall exit was 1 with acceptance.matched true, and the cached-token counts matched the fixture (LMCache 0 then 1280, native 0 then 1488). On the three boundary probes of 511, 512 and 513 tokens, LMCache diverged from the cache-disabled answer at generated-token index 2 after reusing as little as one 256-token chunk. This validates the verifier on this one fixture only. It does not guarantee other models, hardware or workloads, and it makes no performance claim.
Source access
The EFIM implementation repository is currently private. The public documentation does not grant repository access. Open the repository (authorized collaborators). A GitHub 404 can mean your account lacks access. Obtain access from the project owner before following the quickstart.