1. Protein folding is not AGI, and its a technology (much like, say, telecommunications), which no human on their own would be able to perform or even expected to.
3. How can an AI (in a general sense) create a model of underlying reality if the humans who create them do not have access to underlying reality but only the forms of its appearance?
And Transformers are able to learn to use tools, so even mediocre ones can write and then invoke specialist AI.
3. To reach the quality level I specified, that of "You need a PhD to compete with this", it is only necessary to program the AI to be as capable of learning as a PhD student.
This is the standard AI researchers are aiming for, and what I was writing of.
Can they do that? Dunno, and I hope not. But I've bought some tech shares just in case they actually can, because I doubt I'll be able to keep up if that's the near future.
3. How can an AI (in a general sense) create a model of underlying reality if the humans who create them do not have access to underlying reality but only the forms of its appearance?