using regedit, go to:

HKEY_CURRENT_USER\Control Panel\Mouse

double-click the REG_SZ MouseHoverTime and change it from 400 ms to whatever you like, I'm trying 10.

via:
http://www.mydigitallife.info/2009/02/09/how-to-speed-up-taskbar-thumbnails-preview-in-windows-7/
http://streaminggates.com/content/how-to-speed-up-taskbar-thumbnails-preview-in-windows-7

if you want to change the actual thumbnails, have a look at this:
http://www.mydigitallife.info/2010/04/17/download-windows-7-taskbar-thumbnail-customizer-to-resize-taskbar-preview-thumbnails/

So after a couple weeks I am switching this back to 400ms, having the effects (the fade slide down thing when the mouse was no longer on a taskbar button) would get in the way of me clicking something and so I would have to click again, which was more annoying than the 400ms delay.