I wish they would increase the context window or better handle it when the prompt gets too long. Currently users get "prompt is too long" warnings suddenly which makes it a frustrating model to work with for long conversations, writing etc.

Other tools may drop some prior context, or use RAG to help but they don't force you to start a new chat without warning.