Fixed it with a new button! Zing.... and locally too. I guess the old radio shack is still handy.
The Connectons are a bit smaller than the OEM button but if you have some needle nose you can press them down nice and snug. Once that is done the screw on the button tighten it up to the top of the button.. When putting the cover back on leave the button to hang outside. Screw it all together and add some glue to the edge of the button. Press it firmly back into the hole and your done. It does fit snug without the glue but I put some elmers clear glue on there just to keep it in a touch more.
Button product info :
Price : $5.99
SPST Push-button switch. 1.2cm (1/2") mounting hole.
Catalog Number: 2750644
In all honesty It looks great, fits alot better than a piece of dowel and looks to be built alot better.