I used to have this problem when I started, just with pistols.
with rifle I was right eye, right hand shooting.
pistol was left eye, right hand shoot.
I'm pretty sure I was able to train it out of me since my left eye is actually weaker than my right, it just wants to pick up the sights on the pistol for some reason.
I futz with V2s, V3s and V6s. I could be wrong... but probably, most likely not, as far as I know.