The Peril and Promise

Can AI be a chess coach?

Welcome back to Chess In Small Doses, a Substack about adult improvement in chess. Recently I read a couple of books on AI which got me wondering if AI could function as a chess coach. So I began a little experiment that had some surprising results. Long story short, yes it can… sort of. AI is more like a genie in a lamp than an all knowing oracle. It greatly depends on what and how you ask it. Let’s get into it.

The Methods

First, let’s get some details down. For this experiment I used an AI called Lumo from Proton, a non-profit organization based in Switzerland that emphasizes privacy and security. What appealed to me about their chat bot is all searches and histories are known only to me. The company has specifically set up their services with end to end encryption, preventing anyone else (including Proton) from being able to access my data. The free version of the AI only keeps about 7 days of history before they are gone forever. There may be a better product out there, but I doubt there’s a better privacy policy.

Second, Lumo claims to be able to read .pgn files and then be able to analyze games. It should be noted that Lumo does not use an engine to do so. Instead when asked how it analyzes games, it said:

• I don’t call a live engine; I rely on the chess knowledge embedded in the model.

• The analysis proceeds through phase detection → opening theory → material, king safety, pawn structure, piece activity → forced‑move scan → endgame heuristics.

• The output is a human‑readable commentary with strategic explanations, a qualitative “advantage” estimate, and concrete improvement suggestions.

Thirdly, I copied the PGN’s from my personal game studies and simply pasted them into the chat. I have two studies where I keep the analysis from my online games. There’s one for games as White and one for games as Black. These PGN’s included my own commentary and analysis, including move evaluations as mistakes, blunders, good moves, etc..

Lastly, anyone who’s used AI knows that the questions you ask greatly influence the output. Otherwise known as “prompt engineering”, it can be summed up as “good prompt equals good result.” We’ll get into why a little later. Just understand that it took me a few days of experimentation to get what I felt was the right question.

Analyze These Games

That question was:

Analyze these chess games, focusing on the player nvasquez. Summarize your analysis in 3 paragraphs mentioning strengths, weaknesses, and top opportunities for improvement.

Lumo took a minute and then said this (I added the bold text to emphasize the points):

Strengths

Across the many games you played as White, you consistently demonstrate a solid grasp of opening ideas, especially in London‑System structures and various Caro‑Kann lines. You often reach comfortable middlegames by following thematic plans: early h‑pawn pushes to gain space, timely e4/e5 breaks, and active piece placement on the c‑ and d‑files. Your willingness to exchange pieces when ahead (e.g., the series of queen trades in the London games) shows good judgment about simplifying to a winning endgame. Moreover, you frequently spot tactical motifs such as forks, pins, and back‑rank threats, converting small advantages into decisive material gains (e.g., the queen‑side rook lift in the 2025‑07‑26 classical game and the forced mate in the 2025‑08‑24 rapid game).

Weaknesses

A recurring difficulty appears when the position becomes sharp or when you feel under time pressure. Several games contain missed tactics that cost you the win (e.g., the overlooked Nxe5! in the 2025‑07‑08 rapid game, the missed Qb6! in the 2025‑08‑22 classical game, and the fatal Qxd4? in the 2025‑09‑12 rapid game). You also tend to make “scared” defensive moves—moving the king or playing overly passive pawn pushes—when a clear forcing continuation exists. In many endgames you hesitate to activate the king or to push passed pawns, resulting in drawn or lost positions despite a material edge (see the endgame of the 2025‑12‑02 classical game and the Slav‑type ending against Chess2662). Finally, occasional mis‑order of moves (e.g., playing Nd7? instead of the more forcing Nxe5 in the Slav game) indicates a need for deeper calculation before committing to a plan.

Opportunities for Improvement

1. Pattern‑recognition drills: Reinforce typical tactical motifs that arise in your favorite openings (London, Caro‑Kann, Slav). Practice puzzles that involve queen‑side attacks, back‑rank mates, and fork‑type ideas so that they surface automatically under pressure.

2. Calculation routine: Before playing a move that feels “dangerous,” allocate a brief mental pause to enumerate the opponent’s forcing replies (checks, captures, threats). Write down candidate lines on a scrap paper during training games to build the habit of comparing alternatives rather than settling on the first plausible move.

3. Endgame fundamentals: Spend regular sessions on basic rook‑and‑pawn versus rook endings, king activity, and pawn‑majority plans. Knowing the correct method to convert a passed pawn or to defend against opposite‑side rook pawns will turn many drawn endings into wins.

By tightening your calculation discipline, sharpening tactical awareness in critical moments, and polishing fundamental endgame technique, you’ll convert the many promising positions you already create into consistent victories.

I was amazed at the natural language description of my game. There are macro trends highlighted with specific game examples. It correctly noted my “scared” move issue thought I had used those words in my own analysis. Then it went on and made some suggestions on what I could do to improve. It seemed very astute. But can I trust it?

The Promise

AI is an amazing general purpose tool. In the book, Prediction Machines, the authors highlight that the Large Language Models (LLM’s) that make up the chat bots we interact with aren’t blessed with any knowledge whatsoever. They are, however, blessed with very large data sets that somehow enable them to perceive connections to the prompt you have written. In other words, they are very good at predicting what answer will best fit your question. Because of this, LLM’s are unlike any software we’ve experienced before. They are capable of producing startling insights, especially when there is a large amount of data with clear, objective outcomes.

I had wondered if Chess would be one of those areas. All the history of chess and the massive online repositories of moves between Chess.com and Lichess would seem to qualify. Chess of course has only three outcomes: win, lose, draw. I began to think that AI could act as a chess coach.

The problem we’ve all faced since the invention of the internet is an overwhelm of information. For most of human history, access to information was the challenge. Professions like mine (I’m a doctor) were useful because we could be points of access to better information than most people would otherwise have. After the Web came online in the 90’s, that stopped being true. The problem then became relevancy.

When you’re overwhelmed with information (like most of us are in chess improvement), then knowing which piece of information is most useful to you becomes critical. When Google came around in the early 2000’s it helped the world solve that challenge. Sadly, there was no guarantee as to the quality or veracity of the information you had access to. Google would find things for you, but it was up to you to decide if you trusted it or not.

The promise of LLM’s was that perhaps, just maybe, they could help not only provide information but provide actual relevancy of that information to your specific situation or issue. LLM’s are capable of digesting an incredible range and amount of information and then spitting out a summary. A truly incredible technology…sort of.

The Peril

The issue as I said is that LLM’s don’t actually know anything. They’re just really good at predicting what answer you’re looking for. As long as there is a large amount of data and clear, objective outcomes that answer can largely be trusted to be accurate with reality. However, when either of those things are not true, then LLM’s can produce hallucinations. They will occasionally make up stuff to answer your question. Perhaps not really surprising since they’re trained using the open Internet, which itself is about 50% fake. This is a problem that came up in my little experiment.

In the book Co-Intelligence, author Ethan Mollick described 4 rules for AI:

1. Always invite AI to the table

2. Be the human in the loop

3. Treat AI like a person

4. Assume the AI is the worst you’ve ever used

His point is to use AI given its incredible promise, but to remember it has flaws and your judgement is still required. If you think of AI as a person who is sometimes wrong but never in doubt, you are more likely to catch the mistakes that it confidently makes.

Since AI has proliferated, the authors of Prediction Machines pointed out that the “price” of prediction has come down. The odd thing is that this fact makes human judgement that much more important. Humans will need to give thought to how and when to use the output from AI. It puts even more burden on people to think hard. The peril of AI is that when people use it, they often suspend their own judgment and simply treat AI as an all-knowing oracle. There are famous examples of this, but none better than a lawyer who used ChatGPT to make legal briefs only to have the judge catch that there were fictional cases in the brief. Ouch.

Which brings us back to my experiment. Can I trust what it’s telling me?

AI as a Chess Coach

Turns out, not really. At one point in my experiment, using a different prompt, Lumo hallucinated and told me that I was losing many games because I was running out of time. I asked:

OK, now over all my games please comment about my time management. Am I falling behind in time and if so how often? In games where I am low on time, is my opponent ahead or behind on time? How often do I have a lead in time and does it add to wins?

It gave me this breakdown:

Holy crap! I got low on time in 42% of my games!?! I didn’t recall that being true, but maybe I need to completely overhaul my training and work on clock management! An astounding insight if true… only it wasn’t. Not even close. I have personally looked over all of my games I submitted and can only find 5 that ended with less than 2 minutes left on the clock. Which means either Lumo can’t really read the time stamps on a PGN or it made it up. I now suspect it made it up to find the best fit for the question I asked it.

This experience was very frustrating. Talking with AI is honestly like making a wish with a genie. You’d better be sure you want what you’re asking for because that’s exactly what you’ll get. At one point I asked the AI to take on the voice of a grumpy old Soviet coach and it told me to “stop treating chess like a game and more like a job.” Yikes.

Where Lumo was accurate and helpful was giving me a narrative description of my strengths and weaknesses. It went through all my games and my self analysis and accurately summarized it. Perhaps not surprising as that’s one thing LLM’s tend to be very good at. Where it wasn’t helpful was chess specific suggestions for improvements. It suggested I read “Think Like a Grandmaster (Gary Kasparov)” which obviously was written by Kotov. It also suggested I use Lichess Puzzle Storm or Chess.com Puzzle Rush daily to drill “one-move-only” patterns. Not my favorite idea. It did recommend I make a habit of scanning for checks, captures, and threats, but oddly I didn't find this advice all that helpful. Yes it’s evergreen advice, but it felt like AI was simply repeating the most common advice instead of actually tailoring its advice to my games.

Some of the advice it gave was actually surprising and potentially very helpful. It suggested that if I get low on time to play deliberate simplifications to reduce the amount of calculation I need. Honestly, not a bad idea. Secondly it suggested that I do puzzles based upon my common openings. I was surprised that I hadn’t thought of this previously. Lichess offers these kind of puzzles, which made me wonder if I should use them for basic, simple tactic training. Lastly, it recommended that I write down at least 2 thematic pawn breaks in my openings, something I had never intentionally done. It recommended that I set up common middle game positions where the pawn breaks would occur and play them out against weak engines. These suggestions, and several others, seemed actually valuable. Probably not a bad idea to implement any of these suggests.

There’s The Catch

But there is the real rub. I now have to decide on my own whether this information is actually relevant and useful to me. I’ve spent days, dear reader, pondering and pouring over what Lumo has told me. I’ve experimented with different prompts and changed the data I submitted to Lumo. It took significant thought to arrive on what I felt was the right prompt, the one I shared at the top. I’ve also spent a lot of time double checking Lumo’s work. I honestly went an entire 24+hours thinking my number one problem was that I played too slowly! (The reality is I am Speed Racer, the complete opposite.)

In the end, I believe AI is just another tool, albeit one that has the potential to fundamentally change many lives. It could have some use for chess improvers like me, but only by testing it out will we learn where it is most helpful. (It is honestly terrible at interpreting FEN’s). I believe that for improvers AI is just an interesting distraction. The net result of my little experiment is that I have spent the better part of a week playing around with this technology, with not much actual chess improvement to show for it. Still, I have to admit that I have given serious thought about adding some of the more intriguing suggestions to my training routine.

To me this is a mixed bag. The results of my experiment has left me with both keen interest in Lumo and a deep distrust of its output. Having to double check every line of its output for accuracy is a lot of work. Also, it’s not just Lumo. I repeated the same experiments with Perplexity AI and had similar results. It seems that LLM’s have similar strengths and weaknesses when it comes to chess. While no one knows the future, you can be assured AI will very likely continue to get much better and quickly. Still, I do not believe that AI is ready to be a coach…yet.

Currently if you have a large PGN file full of your own analysis, then an AI can be most helpful by giving you a summary of it. Hopefully that shows you something surprising and useful, but it greatly depends on how you ask the question. Beyond that, AI remains strangely bad at chess. So for now, I’ll stick with the human coaches.

Thanks for reading and Happy New Year! Here’s hoping 2026 is a good one for your chess! I’d love to hear your experience with AI and chess. Until next time!

P.S. I wrote this entire piece (as I have all my posts) by hand, typing it out myself. However I did ask Lumo to write why AI may or may not be a good chess coach and it said this:

AI possesses remarkable capabilities that make it an attractive chess coaching aid: precise analysis, constant availability, and data‑rich personalization. However, its inability to teach concepts empathetically, to nurture creativity, and to manage the learner’s emotional state limits its suitability as a standalone mentor. The optimal path forward is not a binary choice but a collaborative partnership—using AI for rigorous, objective evaluation while relying on human coaches for pedagogical nuance, motivation, and the cultivation of a unique playing style. When combined thoughtfully, these resources can accelerate improvement far beyond what either could achieve alone, turning the ancient art of chess into a modern learning experience powered by both silicon and humanity.

As always, it’s up to you to judge. Thanks again!