Experiments3 October 20269 min read

I gave Gemini and EloCoach the same four-move game. Both got things wrong.

Earlier this month I lost a five minute blitz game in four moves. I was Black, rated 582. White was 611. I resigned after the queen took on f7. It is not a game anyone will study, which is exactly why I used it: it is short enough that you can check every claim yourself in about a minute.

I gave the same game to two coaches and then did something I should have done long ago. I replayed every concrete chess claim they made in chess.js and wrote down which ones were false. This post has the game, both full answers, and the list.

The game

[Date "2026.10.02"]
[WhiteElo "611"]
[BlackElo "582"]
[TimeControl "300"]
[Result "1-0"]
[Termination "White won by resignation"]

1. e4 c5 2. Bc4 Nc6 3. Qf3 d6 4. Qxf7+ 1-0

Player names and the game link are removed. Everything else is as played. The mistake is 3...d6: after 3.Qf3 the queen and the bishop both point at f7, and the king is the only defender.

The two setups

Gemini got the PGN above and the line “how did i do coach”. Nothing else. No engine, no history.
EloCoach ran Stockfish and Maia on my phone first. The coach then received the annotated game, the engine’s evaluation of each move, what players at my rating tend to play, and the notes it keeps about how I play.

This is not a fair fight and I am not claiming it is one. It is a test of whether extra structure changes the quality of what comes out.

Gemini’s answer

Read Gemini’s full answer, unedited

You fell into the classic Scholar's Mate pattern on move 4, but the good news is that this is one of the easiest traps to recognize and punish once you know what to look for.

Here is a quick move-by-move breakdown of what happened:

1. e4 c5: Great start. The Sicilian Defense is a strong, ambitious response to 1. e4.
2. Bc4 Nc6: Solid development. Bringing your knight out to control central squares is standard, good chess.
3. Qf3?: The Red Flag. When White plays Bc4 followed by Qf3 (or Qh5), both pieces are battery-aimed directly at your weakest square at the start of the game: f7 (which is only protected by your king).
3... d6?: The Blunder. While d6 opens up a diagonal for your light-squared bishop, it completely ignores the immediate threat on f7.
4. Qxf7+: White strikes. Your king is forced to move to d7, stripping your castling rights and leaving your king exposed to a winning attack.

How to punish early queen attacks next time

When you see White setup Bc4 and Qf3, stop and address f7 immediately. In this exact position after 3. Qf3, you have three great defenses:

3... Nf6! (Best move): Develops a piece toward the center and completely blocks the queen's path to f7.
3... Ne5! (Aggressive counter-attack): Attacks White's queen and bishop simultaneously, forcing White to waste turns retreating.
3... e6 (Solid & safe): Constructs a wall on e6 that stops both the bishop and queen dead in their tracks.

Early queen attacks rely on Black missing the threat. Once you block the trick, White has wasted early tempo moves, leaving you with better piece development and control of the center.

It is short, readable and mostly right. The diagnosis is correct: 3...d6 ignores the threat on f7. Three claims are not:

“Your king is forced to move to d7, stripping your castling rights.” Kd7 was the only legal reply, so the claim is sensible, but it never happened. I resigned instead.
3...e6 “stops both the bishop and queen dead in their tracks.” It blocks the bishop. The queen still sees f7 straight down the f-file.
3...Nf6 is labelled “Best move”. The engine data EloCoach was given lists e6 as best at that point. Nf6 is a good move, but “best” is unsupported.

It also found something EloCoach never mentioned. 3...Ne5 is legal, attacks the queen and the bishop at the same time, and if White takes on f7 the knight simply takes back. That is a good practical suggestion and I would not have thought of it.

EloCoach’s answer

Read EloCoach’s full review, unedited

Taken from the database record. The app’s board diagrams and section tags are removed, and the link to one of my tracked habits shows as “[habit link]”. The words are exactly what the coach wrote.

The Opening: Bowdler Attack Setup

You walked into a four-move miniature because you played a routine setup move while a mating battery was aimed directly at your f7-pawn. The game was decided on move 3 the second you pushed your d-pawn instead of shielding your king.

Against your Sicilian Defense, White opted for the Bowdler Attack with 2. Bc4. It is a blunt, aggressive try that lower-rated players lean on to recreate early kingside mating patterns against unfamiliar setups. Your response with 2...Nc6 was completely sound, developing a knight toward the center and controlling key central squares like d4 and e5.

The Turning Point: Missing the Battery on f7

White tipped their hand immediately on move 3 with Qf3, lining up both the queen on f3 and the light-squared bishop on c4 against your undefended f7-square. This is the classic scholar's mate battery adapted against the Sicilian structure.

Instead of recognizing the immediate threat to deliver checkmate, you played 3...d6, treating the position like a normal opening where you could quietly open a diagonal for your light-squared bishop. That pawn push did nothing to block the f-file or challenge the queen.

White's threat required immediate intervention. Putting up a wall with 3...e6 blunts the c4-bishop's diagonal completely, while developing with 3...Nf6 brings a piece out with tempo by blocking the queen's path to f7 and attacking the queen directly. Both moves leave Black completely safe and already slightly better, because White brought the queen out prematurely.

Instead, White crashed through with 4. Qxf7+, ripping away your pawn shield and forcing your king into the open. Down a key central pawn with your king forced to d7 and White's pieces swarming, you chose to resign on the spot.

This game is an immediate echo of your tracked habit in [habit link] — playing a routine, sensible-looking move while an urgent, forcing threat is staring you in the face.

Strengths

Accurate early development: You chose 1...c5 and followed up cleanly with 2...Nc6, claiming central territory and developing properly before the tactical blind spot occurred.

Areas to work on

Check opponent threats before executing your plan: Use the method of asking what every opponent move attacks before choosing your reply. When an opponent lifts their queen out on move 3, they are not developing patiently; they are hunting for a quick strike. Spotting the queen-bishop battery on f7 makes 3...e6 or 3...Nf6 obvious candidate moves.

Coach's verdict

This was a three-move lapse in concentration. At your strength in rapid games, you calculate far better than this, but in blitz, early queen outings punish autopilot play instantly. When an opponent moves their queen into your half of the board in the first four moves, stop your routine opening moves and calculate the forcing lines first.

When White played 3. Qf3, did you see the threat against f7 and assume your position held, or did you play 3...d6 automatically without registering the queen move?

I played 3...d6 on autopilot without looking at the queen

I saw the queen but missed that f7 was undefended

The diagnosis is the same and the structure is better. It names the threat, shows two fixes, links the game to a habit it had tracked, and ends with a question and two reply buttons. But it makes four claims that do not hold up, and a fifth I could not check:

3...Nf6 “attacking the queen directly.” A knight on f6 does not attack f3.
“With your king forced to d7 … you chose to resign on the spot.” Same Kd7 problem as Gemini. The king never went there.
“A key central pawn.” The pawn lost was on f7.
“A three-move lapse in concentration.” The review itself identifies one mistake, 3...d6.
“At your strength in rapid games, you calculate far better than this.” I did not check this one against my history, so I am marking it unverified.

Then I asked it about the mistake

I asked why it said Nf6 attacks the queen. It admitted the error straight away, which I liked. Then it explained the move again:

Read the follow-up exchange

Same note as above: nothing was changed except the app markup.

Me: Why do you say Nf6 would have attacked the queen directly? Genuinely curious 🧐

Coach: Sharp eye — that was my misstatement. A knight on f6 does not attack the queen on f3. Knights jump in an L-shape; they don't attack down the file.

Here is what 3...Nf6 actually does:

It physically blocks the battery. The queen on f3 wants to reach f7 along the f-file. Dropping the knight onto f6 stands directly in that line of sight. White has no check and no capture on f7.
It attacks the unprotected e4-pawn. White's e4-pawn has no defenders, so your knight threatens a free central pawn on the very next move.
It prepares castling. Developing the kingside knight brings your king one step closer to tucking away behind a castled shield.

The other clean response was 3...e6, which shuts down the light-squared bishop's diagonal into f7 while blunting White's entire opening idea.

Both moves solve the f7 problem immediately. The key lesson remains the same: the moment White played 3. Qf3, the f-file had to be addressed before anything else.

Look at point two. “White’s e4-pawn has no defenders.” It does: the queen on f3 defends it, so Nxe4 loses the knight to Qxe4. The correction contains a new false claim about the position. I asked it afterwards what caused the original mistake, and it blamed two things: mixing up two opening patterns, and no explicit step for checking which squares a piece attacks. That is the model’s own account of itself and I treat it as a hypothesis, not a finding.

The scoreboard

Claim	Gemini	EloCoach
3...d6 missed the f7 threat	Correct	Correct
Kd7 described as happening	False	False
Nf6 blocks the queen’s path to f7	Correct	Correct
Nf6 attacks the queen	Not claimed	False
What 3...e6 stops	False for the queen	Correct (bishop)
Nf6 is “best”	Unsupported	Not claimed
3...Ne5 attacks queen and bishop	Correct	Not mentioned
Pawn, lapse length	None	Wrong
After 3...Nf6, e4 is undefended (follow-up)	n/a	False
Linked to my earlier play	No	Yes
Asked me a question	No	Yes

On whether the chess statements were true, this is roughly a draw, and EloCoach wrote more so it had more places to be wrong. I did not expect that. I checked all of it with chess.js and you can repeat the checks with the PGN above in a minute.

What I take from it

Both made up Kd7, so I cannot blame my prompt. Gemini got a bare PGN and did exactly the same thing. My first guess was that something in what we feed the coach caused it. It is more likely a habit of the model: it knows the forced continuation and tells the story as if it happened.
Giving a model engine data does not make its sentences true. The data covers evaluations and likely moves. It does not cover whether a knight on f6 attacks f3. The model still writes that sentence on its own.
Asking the model to double check is not verification. The same model, challenged, fixed one claim and invented another. Checking has to happen outside it.
Where EloCoach actually differs is not chess accuracy. It connected this game to something I do repeatedly, it asked me what I was thinking, and it gave me something to practise. Gemini gave the better single idea in this game. Those are different strengths.

One fix I am considering

None of this is shipped, and I have not decided to build it. The idea is small. For each key moment, work out in code what the played move and the alternatives actually do (what they attack, what they block, whether they give check) and give the coach that table, so it quotes facts instead of computing them. State plainly that the game ended after the last move played. And log, at first without blocking anything, whenever the coach names a move that was never played, so I can see how often this happens before I decide how strict to be. The follow-up chat is a separate gap that this does not cover, and I will say so when I write it up.

If you want the earlier story of how I ended up pasting games into Gemini in the first place, it is here.