You're experiencing what's called "Parallax."
Basically, you're comparing two different trajectories... at some point, they will meet in the middle, but when you're trying to zero at such a close range, if you try to hit a target out at 150 feet, you'll be so far off your RDS picture, that you won't even know where the BB is. It'll be right out of your sight picture.
There's no point in zeroing at 3-4 metres, because you'll almost never shoot a target that close, and it will give a very extreme parallax difference between your sight trajectory and your actual shot trajectory. The *minimum* distance you should be zeroing in at is 30 feet, ideally 50 or even 70 feet.