You're absolutely right; Most advances do eventually trickle down. But, most advances in FOSS have benefited users who are less well off, making computing, both libre and in general, more available to them. It's frustrating that in order to get a truly free computer one has to pay about the equivalent of 3 months wage, which is a no go for most people I know.