Apparently this is typical to LLMs: https://chatgpt.com/s/t_687a285d9f708191a884d2ad39ddcb53

I mean, it's probably the most linked YouTube video by a factor of 100x so it makes sense for it to be hardcoded in the model.