Engineering8 min read
By Benjamin from EloCoach
How EloCoach fetches your games from Chess.com and Lichess, and what a 422 means
Everything EloCoach does starts with your games arriving. If that step fails, there is no review, no memory and nothing for the coach to say. It is also the one part of the product that depends entirely on two companies I do not control and who owe me nothing. This is how the fetching works on each platform, what the 422 you might have seen actually means, and the limits I have to live inside.
Two platforms, two very different front doors
Chess.com and Lichess both let outside apps read player games for free. They go about it in opposite ways, and the difference shapes everything we do.
| Chess.com | Lichess | |
|---|---|---|
| How we get in | Public read-only API, no key | OAuth, one token per user |
| Shape of the data | One archive per month | One stream of recent games |
| Who is it for us | A handle you type | An identity you sign in with |
| What can go wrong | Politeness is our only lever | Slow stream, strict request limits |
Lichess: a key that belongs to you
Signing in with Lichess is real OAuth. You approve EloCoach on Lichess’s own page, and we receive a token that is unique to you. Background and onboarding syncs ask for your games as you, with your token, not as an anonymous server. That is the right shape for a good citizen: Lichess can see who is asking, and can switch off a single token without touching anyone else. The token is encrypted at rest and a daily job refreshes it. I have not yet measured how much extra headroom a token buys, and the test below deliberately did not use one.
The practical limit with Lichess is speed, not permission. Games arrive as a stream at roughly ten games a second no matter how many you ask for, so a few hundred games takes the better part of half a minute. We ask for a recent slice, not your entire history, and each sync makes a single request for it. Lichess’s own guidance is that a 429 means waiting a full minute before asking again, which is the main reason I would rather fetch less than fetch fast.
Chess.com: no key, so good manners are the whole strategy
Chess.com’s public API is generous and has no sign-in for third parties, which is the part that shapes our approach. There is no token that makes us a known, individually accountable client. All we have is behaviour. So we make as few requests as we can, as gently as we can:
- One request at a time. A player’s games live in monthly archives. We walk them newest first, one after another, never in parallel, and stop once we have enough. Parallel hammering is the quickest way to get rate-limited, and going serially costs us a second or so.
- Do not ask twice. If we fetched your games in the last quarter of an hour, opening the app again reads our own copy instead of calling Chess.com. Games are de-duplicated by their unique ID, because adjacent months can return the same game.
- Background work with a small budget. A scheduled job refreshes stale players every couple of hours, one job per user, with a cap on how many run at once per platform. The job queue retries a failed sync a few times rather than looping on a struggling API.
- Say who we are. Every request identifies the app honestly. If something we do is a problem, the people running the API can tell it is us and have a way to say so, which is better for both sides than a mystery client.
When a single month fails to load, we skip it and carry on, and the next sync picks it up. When it is a real outage, the whole sync fails and the job queue retries later.
I measured both platforms on public accounts, from a beginner to grandmasters and streamers, using GothamChess, Chessbrah, Magnus Carlsen, Alireza Firouzja and others.
| First sync (median) | Re-sync, nothing new (median) | |
|---|---|---|
| Chess.com, 270 to 516 games | 16.8 s | 3.0 s |
| Lichess, about 100 games | 13.1 s | 9.3 s |
A first sync includes storing every game, not only downloading it. The interesting row is Lichess: asking again with nothing new still costs about nine seconds, because a re-sync reads the same stream of recent games again. Chess.com’s re-sync is cheap, Lichess’s is not, and that asymmetry is a good argument for asking Lichess for only what is new.
The more important result was about speed of asking. When I synced several Lichess players at the same time from one server, Lichess answered 429 eight times. Run one at a time, every sync succeeded. Lichess allows roughly one request in flight per client, and a server’s address looks like a single client however many people are behind it.
What we throw away on purpose
Both platforms host far more than standard chess: Chess960, bughouse, atomic, custom starting positions and more. We keep standard chess only. The opening labels would be meaningless, and Maia, the model that estimates what a player at your level would play, is trained on standard chess, so its read of a variant game would be confident nonsense. Games from the main time controls come through, and so do games against bots on Chess.com.
The 422
If you have seen a “no recent games found” message and wondered what happened, behind it is an HTTP 422. In plain terms: your request was fine, we understood it, and we had nothing we could process. It is not a 404, because the route exists, and not a 500, because nothing crashed on our side.
It shows up for a few reasons, from the boring to the awkward:
- A brand-new account with no finished games yet.
- A handle with a typo, or one that exists on the other platform.
- An account that only plays variants, which we filter out by design.
- A player who has not played in the window we look at.
- The awkward one: the platform did not answer. A failure on one platform is caught and logged so that the other can still succeed. If both come back empty-handed, we say there are no games, even when the truth is that we could not reach them. I watched this happen in that Lichess test: the 429s reached the user as “No recent games found”.
That last case is a real flaw, not a feature. “You have no games” and “we could not ask” are different problems with different fixes, and your screen should not blur them.
Limits I would rather say out loud
- Chess.com handles are not verified. You type one. That was a deliberate decision: your private coaching memory is tied to your login, not to the handle, so a wrong handle cannot expose anyone’s data.
- Public data only. We see games the platforms already make public. Nothing private, nothing behind another person’s login.
- We are not instant. Fresh games arrive when we sync, not the second you finish them.
- Recent history, not everything. We cap how many games we pull so that a prolific player in one format does not crowd out the rest.
The risk, and why I am building here anyway
Either platform could decide tomorrow that independent apps are not welcome. They could tighten limits, close an endpoint or require a partnership we cannot get. That is the honest cost of building on someone else’s ecosystem, and I would be lying if I said the product has no exposure to it. I have designed around it where I can: your games and the memory built from them live with us, so an interruption pauses new games rather than erasing what the coach already knows about you.
But I am building here because of what these platforms make possible. Two organisations funded in completely different ways, one a company and one a donation-supported open-source project, both chose to let developers read the games that millions of people play. That is a gift, and a rare one in the app world. Everything interesting in chess software right now exists because that data is reachable: analysis tools, training apps, opening explorers, and coaches like this one. Players are better off with a rich marketplace of tools to choose from than with a single walled garden, and both platforms are keeping the door open for it.
So, to the people behind Chess.com and Lichess: thank you. Playing on your servers is what gives us anything to analyse, and we will keep being as gentle on them as we can. The rest of the machinery that carries those games to your phone is in the stack post.