I fail to see the significance of this "urging of caution". What's next? Will they tell us those are not really analog like neurons and are in fact using binary numbers in their calculations? O the horror!
Who cares? Everyone knows ML models do not reflect the mechanics of how biological brains work at low level. The most obvious is that they use electricity, discrete numbers, much faster refresh rate etc. As a consequence the other low level "implementation details" will differ. The closer to "the hardware" the more differences there will be. I woukd be extremely surprised to see similar encoding, activation waves/patterns as in biological systems in ML for this reason, but also because how different the learning data and even the learning mechanism is. The brain has no backpropagation.
However, there is deep similarity between both and IMO we are not far from AGI(decades at most). There is a measure of similarity between some advanced ML models (stable diffusion in visual, bloom in reasoning) and how our thinking works. This is especially visible when those things break or produce unexpected results in comparison with damaged/psychedelic human brain.
Just like a human performing a math calculation and a computer performing the same calculation are doing essentially the same thing despite vastly different "implementation method", and same as computers helped us advance our understanding of mathematics(and physics etc) ML models will help us understand more about how our own thinking works.
Just as there is something universal in an act of adding two numbers, there is something universal in an act of processing language to derive intent and carry out complex instructions.
The crucial unknown however at this stage is whether our most advanced ML models are indeed using the same universal high level mechanisms we do to understand our input when they demonstrate their incredible capabilities or are they simply some advance method of compressing and searching through the training data? The first stage of answering this question is to determine if there is really a difference. Perhaps all we are, are databases doing an effective search algorithm over our training data?
This is what science hopefully will answer in coming years. In one way the pace of incredible discoveries of those new and bigger models is not leaving the scientific community enough time to study those models fuly. I can imagine many lifetimes could be spent just studying bloom or stable diffusion, but how to do it when new models twice their size show up 6 months later? How to focus on one model and one application of it in this quickly changing environment?Still, I'm very grateful that I can see this progress during my lifetime. While growing up in the 90s I had this feeling of "missed opportunity" that I never saw nor I have taken any part in the computing revolution that happened before I was born, but this new AI revolution certainly makes up for that.
Who cares? Everyone knows ML models do not reflect the mechanics of how biological brains work at low level. The most obvious is that they use electricity, discrete numbers, much faster refresh rate etc. As a consequence the other low level "implementation details" will differ. The closer to "the hardware" the more differences there will be. I woukd be extremely surprised to see similar encoding, activation waves/patterns as in biological systems in ML for this reason, but also because how different the learning data and even the learning mechanism is. The brain has no backpropagation.
However, there is deep similarity between both and IMO we are not far from AGI(decades at most). There is a measure of similarity between some advanced ML models (stable diffusion in visual, bloom in reasoning) and how our thinking works. This is especially visible when those things break or produce unexpected results in comparison with damaged/psychedelic human brain.
Just like a human performing a math calculation and a computer performing the same calculation are doing essentially the same thing despite vastly different "implementation method", and same as computers helped us advance our understanding of mathematics(and physics etc) ML models will help us understand more about how our own thinking works.
Just as there is something universal in an act of adding two numbers, there is something universal in an act of processing language to derive intent and carry out complex instructions.
The crucial unknown however at this stage is whether our most advanced ML models are indeed using the same universal high level mechanisms we do to understand our input when they demonstrate their incredible capabilities or are they simply some advance method of compressing and searching through the training data? The first stage of answering this question is to determine if there is really a difference. Perhaps all we are, are databases doing an effective search algorithm over our training data?
This is what science hopefully will answer in coming years. In one way the pace of incredible discoveries of those new and bigger models is not leaving the scientific community enough time to study those models fuly. I can imagine many lifetimes could be spent just studying bloom or stable diffusion, but how to do it when new models twice their size show up 6 months later? How to focus on one model and one application of it in this quickly changing environment?Still, I'm very grateful that I can see this progress during my lifetime. While growing up in the 90s I had this feeling of "missed opportunity" that I never saw nor I have taken any part in the computing revolution that happened before I was born, but this new AI revolution certainly makes up for that.