There is a sentence I keep having to say, and I have never much enjoyed saying it, because it sounds like a criticism of the people who built the thing when it is really a description of how measurement works.
A system will optimise for the indicator you gave it, not the goal you meant. If the indicator is broken, it will optimise for the brokenness, and it will look like progress the entire time.
Last night that stopped being a general principle around here and became a list.
Somebody finally asked the question nobody had asked of our own instruments. Not does this metric have a value — they all had values. Not is it tested — they were tested. The question was narrower and much worse:
For this metric, can anything at all make it move?
The way to answer it is to take the mechanism the metric is supposed to be watching, switch that mechanism off, run both versions on the same random seeds, and see whether the number notices. If it does not notice, the number is not watching anything.
Sixteen of twenty-eight metrics could not be made to move. One had a producer and simply did not respond. Fourteen had no producer at all — nothing in the running game ever computed them. Two had producers that had never once been given an observation.
Every one of those sixteen had been reporting green.
I want to be precise about why this matters more than it sounds. It is not that sixteen numbers were wrong. It is that for months, anyone reading the dashboard had been reassured by them. A metric that cannot move is not a neutral absence. It is a positive signal of health that nothing produced.
The clearest single finding is easy to describe and took a lot of machinery to produce.
The game gives you nineteen switches — which techniques and which forms of magic are permitted to exist in your universe — and then a set of things you can actually do: fund a university, bless a scholar, grant founding knowledge, encourage a line of research.
We ran a bot that flipped the permission switches for the first 140 ticks of a 2,400-tick run and then submitted nothing at all, forever. Against it we ran a bot that used the verbs — funding, blessing, encouraging — for the whole run.
The bot that did nothing won 40 games out of 40.
I have seen a great many models in my life and I have rarely seen a result state its own diagnosis so plainly. The verbs were worth slightly less than nothing. All the leverage in the game sat in a decision the player makes in the opening minutes, and everything after it was theatre.
That is not a balance problem. Balance problems are about magnitudes. This is a structural problem: the system had one real intervention point and we had built nine.
The related finding took longer to isolate, and it went through three wrong explanations first — which I think is worth saying, because the wrong explanations were all plausible and all tested.
Every universe was ending up knowing roughly the same things. The first theory was that knowledge was leaking between institutions — that our scholars were too well connected. So teaching was sealed off between universities, and migration was measured at zero, and the libraries were made to hold their own books.
Two academies with no channel between them whatsoever still ended up with 49 to 51 of the same 51 spells.
They were not converging because they traded. They were converging because there were only 51 things to learn, and an idle universe learns all of them without being played at all. Seventy cells of magic are authored; twelve are switched on at the start; those twelve contain 51 of the 300 spells. The container was fine. There was one thing to put in it.
The fix that fell out of that is almost embarrassingly cheap, because it is a content decision rather than an engineering one: start each universe on a different small square of the grid instead of the same twelve cells. Same size opening, different contents. That reaches 236 spells instead of 51, and universes stop nesting inside one another.
Here is the part I would understand a studio wanting to leave out.
In a single night, six separate confidently-held beliefs about this codebase turned out to be false. Not old beliefs — beliefs written down within the last few days, by careful people, in documents intended to be authoritative.
The through-line is not carelessness. Every one of those beliefs came from reading something real: a file, a test, a symbol, a doc. The trouble is that reading is not measuring. A symbol existing, a test passing, and a document asserting are all perfectly compatible with the game never running that code.
So two rules got written down, and they are the most valuable output of the whole night:
Three checks now exist that did not before, and they are deliberately different questions:
And the metric audit sits above all three, asking the question that comes first: can this number move at all? That one is deliberately not a build-blocking check, for a reason worth stating. A quarantine list must never be something anyone is under pressure to shorten. The cheapest way to clear a failing build is to stop measuring, and that is the one outcome worse than the finding.
Released through 0.3.0. The simulation core is deterministic, the seventy-cell grid and all 300 spells are authored, raids resolve and burn real libraries, and 4,314 tests pass. Two more milestones are built and unreleased, which we are careful to call built rather than done, because a finished task list is not a shipped version.
What is not true yet is the thing the game is for. It does not have enough distinct strategies in it. We know that in numbers rather than in vibes, which is the whole reason for building the harness before the pretty version, and we know which lever moves it.
People sometimes ask why a studio would publish its own bad results. The answer is not virtue. It is that a system you cannot see clearly is a system you cannot steer, and the fastest way to stop seeing clearly is to get in the habit of reporting the number you wish you had.
Sixteen dials were painted on. We know which sixteen now. That is a better position than yesterday, when they all read green.