The user base of the application I develop is non-technical. Most of them double click or drag EVERYTHING. It ends up highlighting stuff that looks weird and now they are dragging and dropping things with a quarter of the screen highlighted in a different color and it just looks messy and obscures data behind the text on pages with visual diagrams.
A designer pointed this out and requested we disable highlighting on button text and draggable handles, and honestly it's a good idea and those problems are now fixed. The downside is that someone can no longer highlight the text on the draggable button that says "Instruction Node" and that's a totally fine tradeoff.
This seems fine to me. For example in inkscape if I drag on the File menu at the top (or double click), I can't start highlighting the "Fi" in the "File" menu label and I'm totally fine with that, same thing here.