> you may be able to speed up the hardware architecture to compensate for these losses

It reminds me of the concept of Scratchpad memory: https://en.wikipedia.org/wiki/Scratchpad_memory

However, the main issue is with branches. As all code is executed all the time, the execution time is exponential in the number of basic blocks in the control-flow graph C program.