What rate settings though?
I've never had a server where I had to worry about bandwidth, but everything I'm reading seems to imply that you need more than 2Mbps upload to sustain a server. For example,
this says 15Mbps for one 15 player server.
Even these numbers, from before Valve upped the rate cap to 786432, shows numbers higher than your connection can reasonably sustain. I imagine the upload speed is what is bottlenecking and causing the ping issues, at least according to what I've just looked up.
EDIT: Of course, since you didn't provide any of your rate settings, this information may not be accurate.
__________________