Skip to content

Writing

RSS
9 results ·
Inference Lab · · 12 min

Context should be a build artifact, not a prompt assembled at runtime.

Distill Lock freezes context inputs, refuses drift, builds byte-identical portable artifacts, and verifies their integrity offline.

Inference Lab · · 12 min

A cache hit is not proof that you skipped the work

A deterministic KV-cache truth auditor checks engine attestations against an independent token oracle, observed prompt work, output identity, evaluator correctness, and a hash-bound bundle.

Inference Lab · · 18 min

The compiled run had a lower request-latency sum. I still could not claim break-even.

A compiled vLLM run had a lower observed complete-response latency sum across 12 requests, but cold-start debt and different outputs ruled out a break-even claim.

Inference Lab · · 25 min

A benchmark result without lineage is just a screenshot

A technical guide to connecting heterogeneous AI performance evidence without erasing measurement scope, failure state, or claim boundaries.

Inference Lab · · 12 min

The fastest passing system was not the cheapest one

Eight pinned OpenRouter requests split the decision: GLM-5.3 won passing throughput, while Flash bought far more passing cases per dollar.

Inference Lab · · 11 min

Parsing 7,585 XML references to count 400 Metal dispatches

How LLMTraceFX resolves xctrace XML references, preserves process identity, handles sentinels, and checks the result against known dispatch counts.

Inference Lab · · 14 min

The weights fit. The inference workload didn't.

A 16,081,490,933-byte model hit a clean-boot Metal OOM at 256 prompt tokens on a 24 GiB M5 Pro. Stage checkpoints show why file size was never a fit proof.

Inference Lab · · 8 min

A Metal trace is not your workload until you attribute it by process

Five controlled Metal captures matched known dispatch counts by PID. Trace-wide totals still included 36.5% to 57.5% unrelated intervals.

Inference Lab · · 8 min

Your LLM's time to first token might be measuring your HTTP client

An 8 KB HTTP read hid visible output for 185 ms in a controlled test. The client boundary changed the result.