As long as LLMs have no true memory, this is expected. Think about the movie Memento. That is the experience for an LLM.
What could any human do with a context window of 10 minutes and no other memory? You could write yourself notes… but you might not see them because soon you won’t know they are there. So maybe tattoo them on your body…
You could likely do a lot of things. Just follow a recipe and cook. Drive to work. But could you drive to the hardware store and get some stuff you need to build that ikea furniture? Might be too much context.
What could any human do with a context window of 10 minutes and no other memory? You could write yourself notes… but you might not see them because soon you won’t know they are there. So maybe tattoo them on your body…
You could likely do a lot of things. Just follow a recipe and cook. Drive to work. But could you drive to the hardware store and get some stuff you need to build that ikea furniture? Might be too much context.
I think solving memory is solving agi.