99% of the time, yes. Unscrew the old one, screw on the new one.
Or...unscrew the old one, scratch the paint up with some sand paper, drop it into a glass filled with paint stripper and leave it over night. Remove paint with a 3M scrubbing pad (or similar). Prime and paint.