In the most recent game I automated builds for the initial build took 1-3 hours, but subsequent builds with the Library/ folder cached took 5-10 minutes.

Are you switching platforms between builds locally? (like building one for web / server / pc / mac etc). Platform switching essentially blows away the cache, making it take the full build time each time. There was an asset years back that would create folders for each platform's Library/ cache and switch between them when you switched build platforms.

In CI (eg using https://game.ci) you can prefix build cache keys with the platform to manage that automatically.

Unity Cloud Build is in general slower/more flakey/sometimes has cache issues compared to DIY CI, but it should speed up after a successful first build as long as it caches the Library/ folder, and they do separate that out by build platform properly. (Worth checking each build config setting is set to cache that)

Nope not switching between platforms. I'm actually using ParrelSync that allows me to have 2 instances of Unity open working off same codebase. One instance is set to Server and other to WebGL platform. The purpose of it is mostly for faster multiplayer development & testing.

Server Builds are super quick, it's the Unity WebGL Client builds that take forever even if I have cache. Although I have not migrated to Unity 6 yet, that's actually next on my to-dos which I believe there are lots of improvements for WebGL builds there. So I'm hopeful for better dev experience there.

I never knew about game.ci seems like it could be very helpful! Going to check that out... thank you!!