Deep nets are deservedly big because they've managed to improve upon most of the decades-old state-of-the-art methods in the world of signal processing (DSP): voice, image, video, game play, and a significant amount of natural language. No other single computational/algorithmic method has achieved so much in so many domains, ever. That's revolutionary.
The rate of advance using deep nets in signal processing will likely slow down now, but they aren't going away, not in the foreseeable future.
The hype around DNNs arose when we took our unbridled enthusiasm for what's they've achieved in DSP and extended it to other domains where data is less 'dense' and thus aren't as amenable to fast de/convolution in N-D space or time.
Will DNNs revolutionize or introduce all the techniques needed to achieve AGI/Strong AI? I very much doubt it. As yet, there's little sign that DNNs can perform relational operations on interdependent symbols, like the transforms available via type theory, bayesian nets, or predicate logic.
The multitude of disparate facts and semantics in a rich knowledgebase can't be organized into dense matrices the way that continuous signals can, so the SIMD operations that are so effective in DSP won't implement the rich transformations needed in a relational fact-based knowledge space equally as well, if at all. Thus DNNs almost surely aren't going to take us to the heights of logical or compositional thinking that human level intelligence requires.
But how far up relational mountain will DNNs take us? I suspect that won't be known for a decade or longer. But even if we don't reach the summit, we'll be considerably closer than we were before.
This is a really fantastic and interesting look at ML, for someone who’s just beginning first steps. Any recommendations on where else I can read about the areas DNNs and associated stuff (GANs, RL, etc) in terms of what they’ll likely not be capable of in the near mid term?
just commenting to thank you for a very accessible assessment of a simple question "is the hype warranted". please do more of these on HN (or elsewhere!)
The rate of advance using deep nets in signal processing will likely slow down now, but they aren't going away, not in the foreseeable future.
The hype around DNNs arose when we took our unbridled enthusiasm for what's they've achieved in DSP and extended it to other domains where data is less 'dense' and thus aren't as amenable to fast de/convolution in N-D space or time.
Will DNNs revolutionize or introduce all the techniques needed to achieve AGI/Strong AI? I very much doubt it. As yet, there's little sign that DNNs can perform relational operations on interdependent symbols, like the transforms available via type theory, bayesian nets, or predicate logic.
The multitude of disparate facts and semantics in a rich knowledgebase can't be organized into dense matrices the way that continuous signals can, so the SIMD operations that are so effective in DSP won't implement the rich transformations needed in a relational fact-based knowledge space equally as well, if at all. Thus DNNs almost surely aren't going to take us to the heights of logical or compositional thinking that human level intelligence requires.
But how far up relational mountain will DNNs take us? I suspect that won't be known for a decade or longer. But even if we don't reach the summit, we'll be considerably closer than we were before.