Hacker News new | past | comments | ask | show | jobs | submit login

> but LLMs don’t even "see" the board

This is a very vague claim, but they can reconstruct the board from the list of moves, which I would say proves this wrong.

> LLMs have limited memory

For the recent models this is not a problem for the chess example. You can feed whole books into them if you want to.

> so they struggle to remember previous moves

Chess is stateless with perfect information. Unless you're going for mind games, you don't need to remember previous moves.

> They’re great at explaining chess concepts or moves but not actually competing in a match.

What's the difference between a great explanation of a move and explaining every possible move then selecting the best one?




Chess is not stateless. En Passant requires last move and castling rights requires nearly all previous moves.

https://adamkarvonen.github.io/machine_learning/2024/01/03/c...


Ok, I did go too far. But castling doesn't require all previous moves - only one bit of information carried over. So in practice that's board + 2 bits per player. (or 1 bit and 2 moves if you want to include a draw)


Castling requires no prior moves by either piece (King or Rook). Move the King once and back early on, and later, although the board looks set for castling, the King may not.


Yes, which means you carry one bit of extra information - "is castling still allowed". The specific moves that resulted in this bit being unset don't matter.


Ok, then for this you need minimum of two bits - one for kingside Rook and one for the queenside Rook, both would be set if you move the King. You also need to count moves since the last exchange or pawn move for the 50 move rule.


Ah, that one's cool - I've got to admit I've never heard of the 50 move rule.


Also the 3x repetition rule.


And 5x repetition rule


Chess is not stateless. Three repetitions of same position is a draw.


Yes, there’s state there that’s not in the board position, but technically, threefold repetition is not a draw. Play can go on. https://en.wikipedia.org/wiki/Threefold_repetition:

“The game is not automatically drawn if a position occurs for the third time – one of the players, on their turn, must claim the draw with the arbiter. The claim must be made either before making the move which will produce the third repetition, or after the opponent has made a move producing a third repetition. By contrast, the fivefold repetition rule requires the arbiter to intervene and declare the game drawn if the same position occurs five times, needing no claim by the players.”


> Chess is stateless with perfect information. Unless you're going for mind games, you don't need to remember previous moves.

while it can be played as stateless, remembering previous moves gives you insight into potential strategy that is being build.


> Chess is stateless with perfect information.

It is not stateless, because good chess isn't played as a series of independent moves -- it's played as a series of moves connected to a player's strategy.

> What's the difference between a great explanation of a move and explaining every possible move then selecting the best one?

Continuing from the above, "best" in the latter sense involves understanding possible future moves after the next move.

Ergo, if I looked at all games with the current board state and chose the next move that won the most games, it'd be tactically sound but strategically ignorant.

Because many of those next moves were making that next move in support of some broader strategy.


> it's played as a series of moves connected to a player's strategy.

That state belongs to the player, not to the game. You can carry your own state in any game you want - for example remember who starts with what move in rock paper scissors, but that doesn't make that game stateful. It's the player's decision (or bot's implementation) to use any extra state or not.

I wrote "previous moves" specifically (and the extra bits already addressed elsewhere), but the LLM can carry/rebuild its internal state between the steps.


If we're talking about LLMs, then the state belongs to it.

So even if the rules of chess are (mostly) stateless, the resulting game itself is not.

Thus, you can't dismiss concerns about LLMs having difficulty tracking state by saying that chess is stateless. It's not, in that sense.


> good chess isn't played as a series of independent moves -- it's played as a series of moves connected to a player's strategy.

Maybe good chess, but not perfect chess. That would by definition be game-theoretically optimal, which in turn implies having to maintain no state other than your position in a large but precomputable game tree.


Right, but your position also includes whether or not you still have the right to castle on either side, whether each pawn has the right to capture en passant or not, the number of moves since the last pawn move or capture (for tracking the 50 move rule), and whether or not the current position has ever appeared on the board once or twice prior (so you can claim a draw by threefold repetition).

So in practice, your position actually includes the log of all moves to that point. That’s a lot more state than just what you can see on the board.


You can feed them whole books, but they have trouble with recall for specific information in the middle of the context window.


>Chess is stateless with perfect information. Unless you're going for mind games, you don't need to remember previous moves.

In what sense is chess stateless? Question: is Rxa6 a legal move? You need board state to refer to in order to decide.


They mean that you only need board position, you don't need the previous moves that led to that board position.

There are at least a couple of exceptions to that as far as I know.


Yes, 4 exceptions: castling rights, legal en passant captures, threefold repetition, and the 50 move rule. You actually need quite a lot of state to track all of those.


It shouldn't be too much extra state. I assume that 2 bits should be enough to cover castling rights (one for each player), whatever is necessary to store the last 3 moves should cover legal en passant captures and threefold repetition, and 12 bits to store two non-overflowing 6 bit counters (time since last capture, and time since last pawn move) should cover the 50 move rule.

So... unless I'm understanding something incorrectly, something like "the three last moves plus 17 bits of state" (plus the current board state) should be enough to treat chess as a memoryless process. Doesn't seem like too much to track.


Threefold repetition does not require the three positions to occur consecutively. So you could conceivably have a position repeat itself for first on the 1st move, second time on the 25th move, and the third time on the 50th move of a sequence and then players could claim a draw by threefold repetition or 50 move rule at the same time!

This means you do need to store the last 50 board positions in the worst case. Normally you need to store less because many moves are irreversible (pawns cannot go backwards, pieces cannot be un-captured).


Ah... gotcha. Thanks for the clarification.


The correct phrasing would be is it a Markov process?




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: