How I ask every new player for feedback, and what comes back

TL;DR: About an hour after someone joins Inselnova, a message from me lands in their in-game chat asking what was confusing and how they found the game. I wrote it once, the game sends it, and when a player answers, I reply. 17.5% of players answer, but that number depends almost entirely on who’s asking and who’s still around: 43% of long-term players reply to their message, and players who left in their first week almost never do. The replies tell me where players really came from, which tracking misses, and they turn into cards on my task board, tagged with the player’s name.

1,142 players in world 5 have received some version of this message.

It goes to everyone, and it’s the start of most of the conversations I have with players. Some answer in a sentence and never write again. Some turn into a thread that runs for months, where they send me bugs, ideas and the parts of the game that made no sense to them.

The welcome message

I built an admin tool alongside the game, and part of it is a set of messages that go out on their own when a player hits a certain point. Each one is a short message I wrote and a rule for who gets it. It arrives in the player’s chat, from my account, like any other DM. The welcome one reads:

hey Vulpine Fowl, you’ve been in a little while now.

anything confusing or unclear so far? i’m changing things all the time based on what people tell me

also curious how you found the game

When a player answers, the conversation is mine. I’ve answered 195 of the 200 replies. Most come fast: 113 of them arrived within fifteen minutes, while the player was still in their first session.

Who actually answers

The welcome message is one of five. The others go out later, depending on what the player has done since: one for players who tried the game and left in their first week, two for players who played a while and stopped, and one for players two weeks in who are still logging in.

The “explored then left” number looks terrible, and I don’t expect it to be anything else. 257 of the 807 went to guests, players who never registered, so there was no email to send it to as well. A message in the chat of someone who has closed the tab and isn’t coming back will never be read. The registered ones got an email too, and 15 of 529 clicked it.

The players who have been around two weeks and still log in answer 43% of the time. That’s the group with enough time in the game to have opinions, still there to read the message. It shows up inside the welcome message too: players who are still guests reply 4.6% of the time, registered players 23.6%. Players who reply also stick around. 48% of the players who answered the welcome message have played on seven or more separate days, against 3.3% of the ones who didn’t. That doesn’t mean replying makes people stay. It means the people who were going to stay are the ones who reply.

Why the reply rate fell in June

More players join each week now than in May, so raw reply counts don’t mean much. The chart shows the weekly reply rate instead, with an average for the weeks before and after June.

Leaving out the Reddit week, the rate sat around 27% through April and May. Since June it’s been around 16%. The welcome message went through several versions, but the words were identical from version 8 to version 12, and that same text got 35% in the first week of April and 11% in the week the Reddit post landed. Who arrives moves the number more than what the message says. Two things changed in June. Players could start without an account, and my marketing got a lot more focused, so a different mix of people was arriving. Since then more than half the players getting the welcome message are guests, and guests rarely answer.

Analytics doesn’t know where everyone came from

The last line of the welcome message turned out to be one of the most useful things I ask. A lot of players who find the game through ChatGPT don’t click a link. They read the recommendation, then search for the name on Google or type the address in, so my tracking records them as Google or direct.

My tracking has had a ChatGPT source since late August, and it says 24% of new players since then came from ChatGPT. The replies say that’s short. It’s a small sample, but my guess is ChatGPT is worth at least another ten percent of my players on top of what the dashboard shows. The replies also show how fast that’s changed.

Some players just forget

"Its a good game i just closed the tab and forgot about it."

The later messages are mostly about bringing people back. Some players forget, some go on holiday, one had exams. Of the 102 players who got the “engaged, churning” message, 28 played again afterwards, usually a couple of days later. I can’t prove the message is why, but it costs nothing to ask.

"Are u a real person"

Two players asked if I was a bot

Aragorn wasn’t the first. Two days earlier Jittery Gecko wrote “Hi, not going to lie you sound a bit like AI, are you one? Just doing the tutorials rn.” Fair question. The message went out word-for-word to everyone:

Hey {name}. what did you think when you first landed, anything confusing or unclear? i’m tweaking things constantly based on what players say. also curious how you found the game

That went on the task board like any other report, and the new wording went out on 28 August. Has it changed anything? Not measurably. The old wording got a reply 16.6% of the time across 910 sends, the new one 15.5% across 155, and with so many of those going to guests there’s no way to separate the two yet.

Players report symptoms, I work out the fix

Once a player has a DM thread open with me, they use it. The first reply is usually small, a confusing screen or a slow start, but the thread stays open, and later it’s where they send me ideas, bugs and things that didn’t make sense. When something is worth doing I turn it into a card on my task board and tag the player on it as the reporter, so I can see what came from whom and tell them when it ships.

28 of the 32 players tagged as reporters on a card had answered one of these messages first. Of all 1,538 cards I’ve completed, 121 came from players, which is 7.9%. That only counts feedback that became a card with a player’s name on it. Most of what players tell me arrives as chat and changes what I build next without ever becoming a card.

Two players, one problem

In August, Miko Misan hit day 30 and ran out of goals, and wanted “something special that requires protection.” A few days earlier I’d floated stewards to Flowkap while chatting, and Flowkap pushed back that it sounded “a bit too much into building games.” Both of them were describing the same thing, which is that every island is the same list of buildings on the same curve.

What shipped in 0.107.0 was island stewards: an island that reaches Dominion can seat one officer for life, with a name, a face and a doctrine (Defence, Attack, Wealth or Stealth), who takes the island past level 20 and changes what it can do. That isn’t the island siege Miko asked for. It’s a way to make one island different from the next, built into the world as a character rather than a settings screen, and it came out of both of them telling me what was missing.

Chat does the asking

This works because Inselnova has chat. The message lands in the same place as alliance chat and DMs from other players, somewhere they already look, and answering is typing a sentence. A game without chat would need a popup or a form to ask the same question, which is more to build and more friction for the player, and I’d expect far fewer answers. It also means the reply is a conversation with me rather than a form submission.

That’s also why players end up close to the developer. A new player sends a sentence about something confusing, gets an answer from the person who built it, and sometimes sees it changed a week later. The Discord has 51 members without much pushing from me. It’s quiet most days, but a few players have used it to pitch ideas, and some of those are in the game now.