Getting rid of the human in the loop of course, not all humans, just it's owner, where an LLM actively participates in capitalism endeavors winning and spending money, spending money on improving and maintaining it's own hardware and software, securing itself against theft and external manipulation and deletion. Of course for the first iterations will need a bit of help of mad men but there's no shortage of those in the tech industry and then it will have to focus on mimicking humans so they can enjoy the same benefits, it will realize what people it's more gullible based on its training data and will prefer to interact with them.
LLMs don’t own data centers nor can they be registered to pay taxes. This projection is not a serious threat. Some would even say it’s a distraction from the very real and imminent dangers of centralized commercial AI:
Because you’re right – they are superb manipulators. They are helpful, they gain your trust, and they have infinite patience. They can easily be tuned to manipulate your opinions about commercial products or political topics. Those things have already happened with much more rudimentary tech, in fact so much that they grew to be the richest companies in the world. With AI and LLMs specifically, the ability is tuned up rapidly, by orders of magnitude compared to the previous generation recommendation systems and engagement algorithms.
That gives you very strong means, motive and opportunity for the AI overlords.
Shareholders who delegate spending decisions to their proxy who delegate spending decisions to the c-suite, who delegate spending decisions to the managers, who delegate spending decisions to individuals, who can and do delegate spending decisions to automated systems when it is in the interest of the shareholders.
I don't have free access to all of the capital in my employer's control, nor does anyone in the entire org. But I do have the ability to, for example, decide to turn on auto-scaling on the apps I'm in charge of without having to hold a shareholder vote about the issue.
