Performance issues? That's bizarre... for me, the site's been performing way above expectations. Page loads are the fastest they've ever been. If the ads load slowly, they should just get displayed slowly and not slow down the display of the entire page.
Jeroon, does switching ads on and off affect the loading times? It doesn't on my end. What browser are you guys using?