Disagree with it being the bigger problem, it's certainly the processor not being able to handle 128 tick on a Windows machine. Regardless if it was virtualized or not, you simply can't expect good or even adequate performance out of a machine with that processor with 128 tick on a Windows machine trying to run 12 slots.
At your current clock speed you won't be able to get any 128 tick competitive going even if you dropped virtualization and ran on Linux. Your base clock speed is simply too low and depending on how many servers you were running would never be able to utilize the turbo boost benefit. The server currently barely even achieves a PassMark score of 1000, which means running at 64 tick at 12 slots is the best case scenario, and even then it's not great.
If you have control over the server your upgrade options on the machine itself is to upgrade to either of the following three:
Intel® Xeon® Processor E5680
Intel® Xeon® Processor E5687
Intel® Xeon® Processor E5690
These are all on the same socket type as the E5620 but have a much higher base frequency and PassMark score.