There is no greater form of flattery than imitation. (I clearly remember Brian M saying this)
However, I could see that if you wore unit patches/designations it would definitely piss some people off. You didn't bleed for it, you should not wear it. The exception could be made if you are doing a re-enactment though, but that touches on my first comment.
Everyone is going to have different and passionate opinions on the matter, as they should. If your going to wear that patch you must realize that symbol is very important to many, and if you present it in anything less than a satisfactory manner, you can hardly blame them for jumping down your throat.
Also, do not make claim of something that your are not. Let me say that again, do not claim to be something you are not
Otherwise I think if you treat it with respect as those who wear them professionally do, you should be fine.
EXAMPLE: This is an actual member on ASC, not longer active I think. But it can give you some perspective on how people might react to you doing dress up.