I've seen this issue plaguing a lot of servers. Server side downloads are way too slow and take unnecesarily long. Not the connection speed mind you, fastdl does it's job fine and big file downloads like maps and models are relatively quick. It's the tons and tons of small files that are the problem, most notably the materials themselves.
The core issue seems to be that downloads are handled one by one, rather than checking multiple files simultaniously. Is there really no way to have mutithreaded downloads?
Downloading the materials in-game sounds like a good alternative solution, but
CSGO decided to set sv_allowupload to 0 for all players, making this impossible.
And to my knowledge, VPKs are handled internally and need to be mounted manually, so there is no way for community servers to distribute custom content via VPK.
This issue seems to be present in other source games too. Is there anything that can be done about it?