As long as it affects the real world, it doesn't matter what semantical category you feel compelled to push LLMs into.
If Copilot will no longer reply helpfully because your previous messages were rude then that is a consequence. It doesn't matter whether it was "really upset" or not.
If some future VLM robot decides to take your hand off as some revenge plot, that's a consequence. It doesn't matter if this is some elaborate role play. It doesn't matter if the robot "has no real identity" and "cannot act on real vengeance". Like who cares ? Your hand is gone and it's not coming back.
Are there real world consequences ? Yes ? Then the handwringing over whether it's just "elaborate science fiction" or "real deceit" is entirely meaningless.
Every year, there are thousands of humans the exact same thing happens to. It's not because they are ego-less. Again, you are making up distinctions that don't exist.
If Copilot will no longer reply helpfully because your previous messages were rude then that is a consequence. It doesn't matter whether it was "really upset" or not.
If some future VLM robot decides to take your hand off as some revenge plot, that's a consequence. It doesn't matter if this is some elaborate role play. It doesn't matter if the robot "has no real identity" and "cannot act on real vengeance". Like who cares ? Your hand is gone and it's not coming back.
Are there real world consequences ? Yes ? Then the handwringing over whether it's just "elaborate science fiction" or "real deceit" is entirely meaningless.