Ask the coach why. Ask the board what if.

On 15 October I reviewed one of my own blitz games in EloCoach. A Queen’s Gambit Declined, I had Black, I was lost after move 18, my opponent gave it back, and I won on time. The review was mostly good. Then I asked follow-up questions about one move, and over the next three replies the coach suggested three different bishop moves that cannot be played on that board.

I want to show exactly where it went right and where it went wrong, because the line between the two is the most useful thing I can tell you about how to use the coach.

What it got right

The parts of the review that came from the engines were right. Stockfish and Maia run on your phone before the coach writes a word, and the coach receives their verdicts on every move. From that data it said:

• My 12...Bd5 was a blunder, the better move was 12...Bxf1, and White’s strongest reply was 13.e4. All three are in the engine data.
• On moves 17 and 18 White’s knight on f3 had no defenders, and I ignored it twice to play my own ideas. The engine marks both of my moves as blunders, with Qxf3 as the best move each time. This was the real lesson of the game, and it found it.
• The game turned on move 41, when White took my a-pawn and put their king and rook on the same rank. 41...Rh5+ skewered them and won the rook.

If you only read the verdicts, you got an accurate review of the game.

Where it went wrong

The mistakes started where the coach stopped reporting what the engines found and began working out the board for itself. In its first answer it said 12...Bxf1 was strong because White’s bishop on f1 “had only the king defending it” and taking it would strip White of castling rights. White had castled on move 8, and the rook on e1 defended f1.

I pointed that out and asked why 12...Bd5 counted as a blunder when the bishop was protected. The coach answered with a line: after 13.e4, if the bishop retreats to b7 or c6, 14.e5 forks my knight and bishop. Neither retreat exists. My own knight was on c6, so the bishop could not land there or pass through it to b7.

I said so. It agreed, then explained that b7 was the bishop’s only retreat square. I pointed at the knight on c6 again. It agreed again, said the bishop had “zero retreat squares on the entire board”, and suggested giving it up with 13...Bxf3 14.Qxf3. That move is illegal too: the pawn White just pushed to e4 stands between d5 and f3.

The real answer is simple. After 13.e4 the bishop has four legal moves, and each loses material: c4 is covered by the bishop on f1, b3 by the a2 pawn and the queen, a2 by the rook, and taking on e4 gives up a piece for a pawn. My first instinct, that the bishop had nowhere safe to go, was right. The engine’s verdict was right. Only the explanation was wrong, three times, each time with total confidence.

Then I asked it how I could make it better at this. Its answer included a sample position for move 12 that was not this game’s position, and a list of the bishop’s “legal moves” that included a move onto its own pawn. It hallucinated while explaining how to stop it hallucinating.

Why this happens

A language model reads a chess game as text. To know where White’s rook stood on move 12, it has to replay every move before it in its head, and it has no board to look at while it does. It slips most on the things a human sees at a glance: a long diagonal blocked by a piece in the middle, a rook that left f1 two moves earlier.

The engines do not have that problem. Every evaluation, every “blunder” label and every best move in a review is calculated, not written. That is why the verdicts held up. The risk is in the sentences around them, and it grows the further a conversation moves from the game into lines nobody played. The engine data covers each move of the game and the best alternative. It does not cover what happens three moves into a variation you just made up, so there the coach has nothing to read from.

In ordinary reviews this is rare. When I ran the coach on 324 games by titled players and streamers, a script checked 2,001 of its claims about which piece stood where and found four errors (details in an earlier post). This conversation hit the other case: follow-up questions that pushed it into a variation the engines never analysed.

Can it be eliminated? Not today. The full fix would check every move the coach thinks about against a real board while it writes, and that would make replies far slower. What I can do is shrink the risk, and tell you where it lives.

What to use the coach for

Ask the coach about:

• Why the engine’s move is better. The verdict is calculated. The coach’s job is the idea behind it: what the move fights for, what your move gave away.
• Openings and theory. What line you were in, where you left it, and what the usual plans are.
• Concepts. A skewer, a loose piece, how to mate with a king and rook. The review above taught me two of those correctly.
• Your patterns. The coach remembers your games across sessions. Missing a free piece because you were busy with your own plan is a habit it can track over months.

Do not use it to go deep into tactical sequences. If your question is “what if he plays this, then I play that”, and the answer runs more than a move or two, it is guessing. Play it on the board instead. In a review, open the board and move the pieces yourself: Stockfish evaluates every position you reach, on your phone, and those numbers are calculated, not written.

Ask the coach why. Ask the board what if.

One more rule of thumb. When a reply puts a line of moves in a sentence, trust the verdict next to it more than the moves. If the line matters to you, play it out on the board.

What I am changing

The first fix is written: from the next backend update, every key moment the coach receives will include the full position, written as a list of each piece and its square, so it no longer has to replay the game to know where the rook is. Next, I plan to give it the engine’s own continuation at each moment, so the line it narrates comes from Stockfish instead of from memory.

Neither will get the risk to zero, and the advice above will still apply. If you catch a mistake like these, tap No under the answer and say what was wrong. It reaches me.