Metal and metal coated BBs are going to wreck your gun for sure, and they're more likely to break anything SOLID that they hit (gun parts, teeth, eye glasses), but they're not going to hurt more when they hit skin.
When a plastic BB hits a solid object, it deforms, flattens, spreads out it's impact force slightly, absorbs some impact, and then bounces away from elastic force.
Metal and especially silica BB's don't deform, so they make what ever they hit deform, so they're REALLY good at shooting through glass and breaking shit.
When BB's hit skin, it's your skin that deforms, not the BB, so getting shot by silica BBs doesn't hurt more than plastic.
We use plastic to do less damage to the playing field, our expensive guns, eyeglasses, etc