> Who needs an actual model of cognition when you have billions of parameters and brute force?
Well, that is a hypothetical model of cognition. It's not a biological model, but transformers certainly mimic observed properties of brain structures, e.g., grid cells.
They mimic by estimating the patterns of knowledge & restructuring it into something that looks right.
They don't understand anything + calling generative "intelligence" is disingenuous. It's pattern replication. It's like fractal artwork with a happy interface.
Not many of us touch this stuff bare metal to final product ; it's complicated, too complicated for most individuals to work on solo. I wish I cared enough to learn the math to prove out my logical understanding.
Big tech cares about profit too much to actually document what these models are, how they draw their conclusions. Until independent researchers who care to document mathematically:
Probability of next segment being part of generated pattern, logical connections across known patterns that attach to current /previous pattern
We will continue to have this generative farce propped up on a pedestal, gilded with pyrite, and profits poured into a half-baked pattern bruteforcer.
I beat on the tech and badmouth it, but I'm still using it to make money. :) I'll ride the bandwagon until something better comes along. It's a means to an end.
Well, that is a hypothetical model of cognition. It's not a biological model, but transformers certainly mimic observed properties of brain structures, e.g., grid cells.