It does matter but just ask stalker. He got way better and more consistent performance out of his CA M24(?) when it was firing like 350-380 (can't remember the exact velocity) than 450+. He claimed, IIRC, that he got better range and truer flight path.
Not all hop ups are the same though. Good hop up rubbers don't wear down easily. Good nubs are important too. And then the hop up unit itself has to be able to keep completely static and not shift the amount of hop from the force of the BB hitting the nub. TM's, from my experience, have a good hop up design and quality (some aftermarket ones are better, obviously). They keep their setting quite well. That is what matters. If your hop-up unit changes its setting after 100 shots, then all of a sudden your path is different.
I love freedom and consequently America