There must far more to it than that. As soon as sleep is a thing, it can be optimised for different goals. Since animals have widely varying sleep requirements, there's clearly some evolutionary factor that influences sleep length.

That is, though sleep might have physiological requirements, it doesn't mean that the amount of sleep is not influenced by non physiological effects.

I'm constantly amazed by the ability of biologists to be amazed by the reach and ingenuity of evolution.