What do you see as an alternative? This is exactly what people need. To give quick instructions and have agent work on tasks across webpages/webapps. You say you "have many thoughts on fixing this". Can you share a better vision?
Yes, it involves "recipes" for different "action types" based on a few things (past behavior, sensible "default" behavior, the context, etc.). For example, when looking at a storefront, the agent might want to give a few options (off the top of my head):
- It knows you're low on milk, so it suggests buying milk
- It can get a list of all deals, maybe even cross-checking with other storefronts
- It knows your sister was looking for a planter, and it found a particularly cheap one, so it suggests texting it to her
- Etc.
- (You can of course still chat with the thing, if you so desire)
These recipes are difficult to come up with and hard to generalize (they also need to be categorized, and likely accurately picked from a [RAG?] database), but imo this is the future if AI agents, not yet another chatbox.
