As a non-programmer who likes to work on one-off projects or personal projects, I completely agree with you. If someone is still doing some of these things after 10 years of experience in the industry, then yes, they probably are a bad programmer. But if they're new, this can just be chalked up to a lack of experience. I think this relates to the industry's general tendency to shut out anyone who "doesn't know what they're doing". The language used to answer some of the more basic questions on Stack Overflow is a perfect illustration.
I think that the hype and buzzwords around Big Data and data science cause more than just bad business decisions. I believe they are also damaging the industry and creating a larger sense of disillusionment (I'm mostly thinking of "deep learning"). Not sure what this means for data science in the long term though, just thinking out loud.
I'll also add that I frequently see sledge hammers being used to hang a picture frame. By that I mean using huge clusters to run algos that would actually run in Tableau, Excel etc.
I had a conversation with a 'big data consultant' some time back. He mentioned one of his clients needed to set up a Hadoop cluster and wanted me to work with him. I said, 'why do they need a cluster, they probably don't have that much data'. His response was 'if a client wants to jump off a building, you don't say don't do it, you ask them what floor'.
