The the enormous benefit of being able to claim GPT doesn't just regurgitate its training data verbatim, which puts it clearly within the crosshairs of copyright lawsuits. That's a potentially business-ending catastrophe, so this has massive benefit. Only to OpenAI, not to you or me.
Hiding the fact that they did it by intentionally giving false answers is in no way clever or even smart. You aren't following the rules just because you make it so you can't give the correct answer. You just convince me that your product has nothing of value to add because every response is now clouded by doubt.
Edit:
I just ran my same lyrics test with Bard and it passed 100%. It gave me the full lyrics and chorus using part of the lyrics as input, not telling it specifically what song to use. So in my experience Bard is now better than ChatGPT even though nothing about ChatGPT has changed other than OpenAI has lobotomized it.
But that'd fly in the face of seeking forgiveness not permission. While they'd be wasting time "following the rules" (which incidentally don't exist yet since they haven't been tested/ established), a less moral company would eat their lunch. This is the reality of innovating at the edge.
strangely i haven't seen any lawsuits claiming they didn't have a right to ingest pirated/unlicensed material, it's all front end claims of reproducing substantially similar output
The the enormous benefit of being able to claim GPT doesn't just regurgitate its training data verbatim, which puts it clearly within the crosshairs of copyright lawsuits. That's a potentially business-ending catastrophe, so this has massive benefit. Only to OpenAI, not to you or me.