What is fixing one chess habit worth? How EloCoach estimates it

The Coaching tab used to tell you what your habits were. “You keep missing threats.” True, and a little useless, because the obvious reply is: so what is that costing me? I wanted a number on the button, something like “Fix this habit, ~25 Elo”. The catch is that a rating promise on a button has to survive the player doing the arithmetic. If someone can check it against their own results and it falls apart, I have lost them for good. This is how the number is built, including the first version I threw away.

The first number was 80, and nobody would believe it

The raw material is something the review already produces: for each loss it reviewed, which habit decided the game at the move where it was actually lost. On the account I tested with (529 games, 260 wins, 29 draws, 240 losses) only 11 losses had been reviewed, and one habit, threat response, decided 6 of them. So take the shortest possible route: that habit decided about half the reviewed losses, so it is behind about half the losses, so if I turn each of those into a draw the score goes up, and the rating with it.

Eighty points for fixing one habit is a claim I cannot back, and a player at that level would know it. Every step above is defensible on its own. The product of them is too optimistic, because it assumes the habit is the whole story of those losses and that fixing it converts every one of them. Neither is true.

The three choices that make it honest

• A draw at best. Fixing the habit does not win the game you lost, it rescues part of it. So each rescued loss is worth half a point, never a full one.
• Two different samples. The loss rate comes from every game you have played, 529 here. The share comes only from the games that were reviewed, 11 of the 240 losses. Mixing those up would let a tiny sample set the size of the claim. The big sample sets the scale, the small one sets the proportion.
• Only a fraction is fixable. I assume fixing the habit turns one in three of those losses into a draw. That is a guess, and I put it in a single named constant so it can be changed in one place and argued about in the open.

With those three, the score gain drops from about 0.11 to about 0.04, and the rating change from +80 to roughly +25.

A game can only be lost once

Habits overlap. On the same account there are 8 habit tags across 7 lost games, because one game was decided by two habits at once. Count each tag in full and that game is credited twice, so adding up what every habit is “worth” gives you more than the player could ever gain. The fix is to split a game across the habits that decided it: a game tagged by two habits gives each of them half. The threat-response share is then 5.5 of 11, not 6 of 11, and across all habits the shares can never total more than 100% of your losses. If a habit has been reset and recurred, its window is shorter than the others, so I scale the shares down together when they would otherwise add up to more than one.

From score to rating points

The last step is the ordinary logistic curve that Elo is built on: a rating difference of 400 points means roughly a 10 to 1 expected score. The player on this account scores about 52%. Add the 0.04 and they score about 55.7%, which the curve says is around 26 points. I round to the nearest 5, so the card says “~25”, and I hide anything under 5 because it is noise.

There is one more guard, and it exists because a review found a bug I had merged into the main branch. A score of exactly 0% sends the curve to minus infinity, and my first clamp to avoid that was so tiny that a player with no wins in a short window was told a habit was worth about 1,700 Elo. The score rate is now kept between 5% and 95% before the curve is applied, and the worst case the formula can print is a few hundred, not thousands.

What the number will not say

• It says “Worth up to ~25 Elo”, never “will boost” or “+25 rating”. It is a ceiling with a reason attached, and the reason sits on the card: “Decided 6 of your 11 reviewed losses”.
• It does not appear until at least 6 losses have been reviewed.
• It is not shown for a habit that is improving or resolved, for a pattern seen only once, or for a strength. There is nothing to fix there.
• It leads the tab only for a habit with enough positions from your own games to practise.

What I am still unsure about

The honest test is to compare the estimate against what actually happens to rating after someone practises a habit, and adjust the one third accordingly. Until there is enough of that, the number is a reasoned estimate with its working shown, and I would rather show the working than a bigger number.