Prefix KV reuse is standard in production inference. On the stack we tested, reusing a cached prefix changed the model’s answer — while every counter reported a clean, complete hit and no fallback.
No performance claims.
python -m diagnostics.verify_kv_reuse \
--model Qwen/Qwen3-0.6B \
--model-revision c1899de289a04d12100db370d81485cdf75e47ca \
--kv-connector LMCacheConnectorV1 \
--lmcache-config diagnostics/fixtures/lmcache-local-cpu.yaml \
--dtype float16 --gpu-memory-utilization 0.45 \
--max-model-len 2048 --block-size 16 --max-tokens 16 --seed 0 \
--suite acceptance --compare-native \
--fixture diagnostics/fixtures/efim-018-l4.json \
--report tmp/efim-019-l4/report.json# NVIDIA L4 · 54a0af2 · 2026-09-10 # 24 generations · acceptance.matched = true connector · LMCache 0.5.4 fixture 1502 0→1280×3 FAIL @4,15 chat-511 511 0→256 FAIL @2 chat-512 512 0→511 FAIL @2 chat-513 513 0→512 FAIL @2 native · vLLM prefix caching fixture 1502 0→1488×3 PASS chat-511 511 0→496 PASS chat-512 512 0→496 PASS chat-513 513 0→512 PASS tokens · cached · verdict · @first diff claim_eligible=false; no performance claim
Our reproduction, posted for the LMCache maintainers: vLLM 0.28.0 with LMCache 0.5.4 on T4 and L4 (and 0.5.5rc5 on T4), native prefix caching as the byte-exact control, and a runnable script. Not yet confirmed as the same bug the issue describes.
Not a crash, not a timeout, not a degraded response. Every health signal a serving stack exposes says the request succeeded.
No fallback reason, normal finish. The counters are indistinguishable from a correct run.
A miss matches cache-disabled recompute exactly. Only the reuse path diverges.
An 8-token window caught only the divergence at index 4. Widening to 16 exposed those at 10 and 14. A short check misses real mismatches.
A bare AsyncLLM and the in-tree connector. No gateway, no bridge, no HTTP.
Same 1502-token chat prompt, same model revision, float16, eager execution, greedy decoding, 16-token window.
A cold miss, then three reuses that each diverge from the miss and from one another. Every retrieve logs five double-unpin warnings; the cold store logs none.
The same input across four requests returns byte-exact output — with a larger reused prefix than the connector achieved.
That contrast is what makes the finding actionable. It narrows the fault to LMCache’s path rather than to the GPU, the model, the dtype, or prefix reuse as a concept.
We posted this reproduction, with a runnable script, to the LMCache maintainers on their existing silent-corruption issue.
It runs a configuration twice in separate processes — once with caching fully disabled, once with reuse enabled — and compares generated token IDs exactly. No normalization, no tolerance, no majority vote.
It currently supports Linux with one NVIDIA GPU, vLLM 0.28.0, and stock LMCache 0.5.4/0.5.5rc5 or native prefix caching. The source is private for now.
A reuse with a zero or missing hit counter is inconclusive. Absent telemetry is never read as zero.
Submitted and generated IDs, per-answer cached counts, first differing index, effective engine config.
Versioned JSON, atomic writes. An error is never hidden behind a green summary.
claim_eligible=false. We publish no speedup, cost, throughput or latency number.54a0af2: LMCache failed 4 of 4 probes, native prefix caching passed 4 of 4, and acceptance.matched was true across 24 generations. That shows the verifier separates these two configurations on this fixture — including LMCache diverging after reusing a single 256-token chunk. It does not show the verifier catches every defect, and says nothing about other models, hardware or workloads.The failure mode is silent by construction: correct on a miss, clean in telemetry, wrong only on reuse. We would rather hand you the test than tell you a number.