Inference, question-answering, NER detection/disambiguation are pretty important NLP tasks (at least from a practitioner's perspective). While GPT-2 has gained mindshare for its generative capabilities, BERT and other pre-trained Transformer Encoder models are used for production workloads (we use BERT for text classification and explored using it for clustering).
It's useful to view the GitHub projects for these models as reference implementations. They're intended to provide a roadmap for reproducing the research and to aid in implementing production libraries.
Regarding the latter, take a look at the work by HuggingFace, the Flair project, Spark-NLP and others.
"Inference, question-answering, NER detection/disambiguation are pretty important NLP tasks"
Yes indeed.
"While GPT-2 has gained mindshare for its generative capabilities, BERT and other pre-trained Transformer Encoder models are used for production workloads"
You rephrased my point pretty well, while openAI search for "fun" tasks, deepmind and others allow progress on real world tasks.
You use BERT which is nice but do you consider using it's successor: XLnet?
"take a look at the work by HuggingFace, the Flair project, Spark-NLP and others."
I was aware of Flair (from Zalando) but thank you for Huggingface and Spark-NLP, I will take a look!
It's useful to view the GitHub projects for these models as reference implementations. They're intended to provide a roadmap for reproducing the research and to aid in implementing production libraries.
Regarding the latter, take a look at the work by HuggingFace, the Flair project, Spark-NLP and others.