Flat Metric

A measurement that reports the same value for every version of the system — and therefore cannot be evidence for any of them.

Why it matters in a memory layer

Three of this course's four component metrics report 1.000 for every profile from I4 onwards. They are correct, and they justify nothing built after I4. A report showing only the final row lets them read as evidence for the whole course.

Flatness also has an innocent cause worth separating from failure: some changes are shapes a battery cannot express. Deferred consolidation's result exists only during a turn-by-turn replay — over a finished ingest the converged store is identical either way. A write policy's whole demonstration is in writes that were refused, and a refused write is not in the store to score.

A metric that cannot see a change is not evidence the change did nothing. It is evidence about the metric, and a suite's job is to make that distinction visible rather than leaving it in a paragraph.

Connections

Taught in: Build Your Own Harness

Used in: Reading Benchmark Claims Critically · Regression Testing a Stateful System

Do not confuse with: Eval Suite