There's nothing to fix. People work on what they want to work on. Things that seem important to you are not important to me, and the opposite. I'm OK with that.
"People work on what they want to work" ideally yes, but ultimately they work on something that please them AND that give them a decent salary.
Funding should not go to fun (but useless in the real world) Nlp tasks.
"Things that seem important to you are not important to me, and the opposite." and here's go relativism or the abandon of thought...
It's indeed difficult to quantify cardinally the utility of an NLP task against an other, but we can agree on an ordinality (order of magnitude)
E.g do you understand that POS tagging or dependency/constictuency parsing are angular tasks needed by much of the others. Thus making them the most important NLP tasks as they enable other Nlp tasks and are the most used in practice?
You think that what exactly is more important?
Are you talking about text generation? Why is that important?
Something important enable to solve important problems in the real world.
How text generation solve any real world problem is beyond my knowledge.
But if you rationally think that it's more important that angular Nlp tasks, you can probably explain why and give an example or two?
Yes, an AGI will need to emit text just as humans do, indeed.
But before that she needs to understund the natural language before emitting it.
GPT-2 maybe capture an aesthetic of the initial input pretty well but it does not generate meaningful sentences or only by accident, so no GPT-2 does not advance the quest to create an intelligent agent mastering natural language.
do you understand that POS tagging or dependency/constictuency parsing are angular tasks needed by much of the others.
I'm not sure. I rarely have to do that explicitly in my head. Perhaps a model should learn to infer/guess them implicitly, from context, just like I do.
what exactly is more important?
In my opinion, having a world model (for common sense) and situational awareness (e.g. through sensor fusion, or from prior conversational history, or using some externally supplied conditioning) would be far more important.
GPT-2 does not generate meaningful sentences or only by accident
You think adding POS tags would help it generate meaningful sentences?
I'm not sure. I rarely have to do that explicitly in my head.
Well I can't prove it but I strongly believe that our brains use part of speech too, unconsciously.
Perhaps a model should learn to infer/guess them implicitly, from data.
That's exactly what deep learning POS tagger do, they are far better than hard coded algorithms. SOTA has 97.96% of accuracy.
In my opinion, having a world model (for common sense) and situational awareness (e.g. through sensor fusion, or from prior conversational history, or using some externally supplied conditioning) would be far more important.
Haha you basically want a general intelligence (AGI), I want it too! And not enough persons works on "architecting" such a thing. Opencog may interest you a lot then.
But the reality is many other "simpler" tasks are needed to make this happen.
You think adding POS tags would help it generate meaningful sentences?
I would be clearly insufficient yet necessary.
I believe they already use internally a POS tagger and a dependency parser.
I was imagining using a separate model just for providing tags as they are very accurate. It would theoretically give gpt-2 useful data.
GPT-2 has not (yet) been trained to predict POS tags to my knowledge, nor BERT, or ernie 2 or xlnet has, but I think they have great potential to improve POS accuracy.
There's nothing to fix. People work on what they want to work on. Things that seem important to you are not important to me, and the opposite. I'm OK with that.