I have discovered the issue and a fix for NVIDIA based cards.
The issue is with Driver Extension Strings. Newer versions of NVIDIA cards allow unlimited use of extension strings, and this causes issues with games using older versions of OPENGL.
To fix the issue, go into your nvidia control panel, to manage 3D settings, and set the Extension Limit option to On.
This has solved the issue for my steam copy of Aquaria. (through I also turned off v-sync as well) But based on the description of that option, I'm sure it's what's causing the issue.