I ask AI for a failing test before I build a feature. I've got more than 5,000 now and I'm not sure that's healthy.

TL;DR: A test is a small program that checks one piece of the game does what it should, and I lean on them so I can release often without breaking things players rely on. My habit is to describe how a feature should work, ask the AI for a failing test first, watch it fail for the right reason, then let it write code until the test passes. That gives me better code. The doubt is the pile it’s produced: about 250,000 lines of game against 127,000 lines of tests, none written by hand, growing every month. A suite that size is slow to run, breaks when you reword a button, and probably tests some things twice, and I can’t yet tell how much of it is protecting me and how much is just weight.

What a test is, and why I have so many

If you don’t write software, a test is easy to picture. It’s a tiny program that pokes one part of the game and checks the result. Try to upgrade a building you can’t afford, and the test confirms the game refuses it. Send an attack with no troops, and it confirms nothing happens. Each test states one small thing the game should do, then proves it still does it.

The reason I have a lot of them is simple. Every time I release, I want a high chance that everything still works and that I haven’t quietly broken something three systems away from whatever I changed. I can’t click through the whole game by hand before each release, and I release often. The tests do that check for me, in a couple of minutes, automatically. Every test I write also stays in for good as a regression test, re-run on every release from then on, so a behaviour that works today can’t quietly break six weeks later without something going red. When one does go red, it’s usually caught a real break before a player ever would have.

The test comes before the code

There’s a wrong way to let an AI write tests, and it’s the obvious way. If you let it write the tests after the code is finished, it just reads the code and writes tests that agree with it. Say the code works out tax wrong. The AI sees the wrong number, assumes it’s correct, and writes a test that locks that wrong number in. Now the bug has a test defending it. You’ve doubled the amount of code and proved nothing, because the test only ever checks that the code does what it already does.

So I work the other way around. Most of the time I start by describing how the feature should behave, in plain terms, and I ask for the test before any game code exists. The test has to fail. I run it, watch it fail, and check it’s failing because the feature isn’t built yet and not because of a typo or a broken import. Only then does the code get written, until the test passes. The behaviour the test checks against comes from how I said it should work, not from whatever the code happens to do. That order, test first, is the part I’d keep if I threw everything else out.

It runs three times before it reaches you

A test doesn’t just run once when it’s written. It runs at three points, and each one is a wider net than the last.

The first is local, while I’m building. When I finish a feature I run the tests around it there and then. For a small change that’s just the handful of tests near what I touched. For something bigger the system runs a full check across the game. If something fails, the AI works out why, and most of the time it’s a small thing it fixes on its own. Now and then it stops and asks me a question to be sure it’s reading the intent right, and usually what it caught is a real problem and not a false alarm.

The second is just before a release. The whole suite runs again as a gate, so nothing ships on the strength of a local run I might have rushed.

The third is on GitHub. When the release branch goes up, GitHub Actions runs the full set on its own machines: type checks, a risk pass over what changed, the backend tests, the supporting service tests, a production build, and an end-to-end smoke test that drives the real game in a browser. A full run lands under ten minutes now. It used to be over twenty. The backend tests were the reason. Run in a single lane they took long enough that a release sat waiting on them, so now they’re split into six lanes that run at the same time, grouped by area, and that change alone did most of the work in getting the wait back down.

One line of test for every two lines of game

None of them were written by hand. I describe the behaviour, the AI writes the test and then the code, and over months it adds up. The game runs on about 250,000 lines of code. Sitting next to it are about 127,000 lines of tests. So for every two lines that actually run the game, there’s roughly one more whose only job is to check the first two still work.

Is that a lot for a game? I honestly don’t have anything solid to compare it against. Inselnova’s backend is really a web server that happens to run a game, and a lot of those tests check server behaviour rather than anything you’d think of as gameplay. So the count might say less about games than it does about the kind of code sitting underneath this one. What I can see is that the pile keeps growing every month, faster than it was at the start of the year, and that’s the bit I haven’t got comfortable with.

The first bar says as much as the climb does. I didn’t ease into testing once the game got big enough to scare me. It started with a couple of thousand tests already in place, in the first month, before there was much of a game to speak of. The habit was part of the foundation, not something bolted on later, and everything since has been built on top of that.

What all those tests cost

A suite this big still isn’t free, even when it runs in nine minutes.

It breaks too easily. A few hundred of the front-end tests check the exact words on the screen, that a button says one thing or a panel shows another. Change that wording, even when the game behaves exactly as it did before, and the tests go red. That’s the test being wrong, not the game. A small copy edit can light up a wall of failures and make me hesitate before tidying up a label, which is the opposite of what a test is meant to do for me.

And some of it is probably wasted. The AI writes fresh tests for each thing it builds, and across hundreds of endpoints the same checks come back again and again. A lot of that is correct, every endpoint should return a 401 without a login. But somewhere in there are tests proving the same thing twice in slightly different words, and I’ve never run a sweep to find out how much. It’s on the list I haven’t got to.

So, too many?

I don’t know yet, and that’s the honest answer. The method I trust. Writing the failing test first, from the behaviour I asked for, gives me better code and catches real breakage before it reaches a live world. The pile it produces is the part I’m still working out. A test suite is only worth having if you can run it quickly, if it doesn’t fall over when you brush against it, and if you actually know what’s inside it. Mine is fast enough, a bit too brittle, and never properly audited. That’s two out of three, and I’m working on the third.