I believe the cvar net_maxfilesize default value is 16 and the max is 64 so you should be able to download the map with no problems aside from the pretty slow 40kB/s download speed you are getting when download directly from the game server.
Anyways to get your fastdownload working you need to point to the directory of the game/mod rather than the map folder.
Code:
sv_downloadurl "http://enderandrew.com/tf/"