Well I think I have found a happy medium, The police said that it was fine as long as it was transported in a proper rifle case and the ammunition was stored separately, the TTC seem to basically be worried about people freaking out...
So: I'll be transporting the gun in a locked rifle case inside a snowboard bag. The bag is big enough to easily take a full-length oversize single rifle case. A snowboard case won't attract any attention and I am still complying with the law.
Ammo and batteries will be stored in whatever other bag I take with me.
I think that should do! No-one freaked out and if the police do get involved somehow there won't be any issues because I'm in compliance with the law.