[snelvuur]

Ok,

My TF2 server now and then "hangs" it takes 100% cpu, i tried to strace the process or use gdb to attach to it but my skills are not up to speed to actually get something usefull from hits.

When its 100% cpu, no users can connect, and no response from the server at all.

The only thing i managed to get a copy of is a full memory dump with gdb. I aquired this by doing a kill -3 on the process and having gdb startup srcds to begin with.

I've uploaded the core dump to http://78.46.73.57/ (its 302mb extracted)

Is there somebody that can take a look or explain me how to further investigate this problem?

sm version
SourceMod Version Information:
SourceMod Version: 1.3.0-dev
SourcePawn Engine: SourcePawn 1.1, jit-x86 (build 1.3.0-dev)
SourcePawn API: v1 = 4, v2 = 3
Compiled on: Dec 15 2009 20:51:48
Build ID: 2872:e1ebaed5e1bb-dev
http://www.sourcemod.net/

meta version
Metamod:Source version 1.8.0-dev
Build ID: 689:4065ba5e9155-dev
Loaded As: Valve Server Plugin
Compiled on: Dec 9 2009
Plugin interface version: 15:14
SourceHook version: 5:5
http://www.metamodsource.net/

[SM] Listing 39 plugins:
01 "Web Shortcuts" (1.0.1) by James "sslice" Gray
02 "Fun Votes" (1.3.0-dev) by AlliedModders LLC
03 "Themes" (0. by J-Factor
04 "SourceMod Radio" (1.0.0.12) by dubbeh
05 "Basic Chat" (1.3.0-dev) by AlliedModders LLC
06 "Advertisements from Database" (1.2.100) by Tsunami and <eVa>Dog
07 "TF2 Class Restrictions" (0.5) by Tsunami
08 "Basic Comm Control" (1.3.0-dev) by AlliedModders LLC
09 "SuperLogs: TF2" (1.4.4) by psychonic
10 "Advertisements" (0.5.5) by Tsunami
11 "SourceBans" (1.4.2) by InterWave Studios Development Team
12 "gScramble + Balance" (2.4.02) by Goerge
13 "Fun Commands" (1.3.0-dev) by AlliedModders LLC
14 "lethalpoints.smx"
15 "Destroy Engineer Buildings" (1.0.2) by bl4nk
16 "Reserved Slots" (1.3.0-dev) by AlliedModders LLC
17 "Suppress Messages" (1.0.6) by FlyingMongoose
18 "AFK Manager" (2.6) by Liam
19 "Basic Info Triggers" (1.3.0-dev) by AlliedModders LLC
20 "Admin Menu" (1.3.0-dev) by AlliedModders LLC
21 "FistPie" (0.4) by Darkimmortal
22 "Admin File Reader" (1.3.0-dev) by AlliedModders LLC
23 "HLSW Info" (1.1) by Tsunami
24 "FragFM Request/Shoutout" (1.0) by Lethal (uknemesis-server.net)
25 "Basic Votes" (1.3.0-dev) by AlliedModders LLC
26 "TF2 Fast Respawn" (1.2) by Tsunami
27 "Anti-Flood" (1.3.0-dev) by AlliedModders LLC
28 "Admin Help" (1.3.0-dev) by AlliedModders LLC
29 "Maplister" (1.2) by theY4Kman
30 "DeadChat (tf)" (1.3) by Greyscale
31 "HLstatsX CE Ingame Plugin" (1.6.2) by psychonic
32 "RateChecker" (0.2) by pRED*
33 "Sound Commands" (1.3.0-dev) by AlliedModders LLC
34 "YeGods High Ping Remover" (1.0.1.10) by dubbeh
35 "Player Commands" (1.3.0-dev) by AlliedModders LLC
36 "TF2 Critvote" (1.0.0) by R-Hehl
37 "Client Preferences" (1.3.0-dev) by AlliedModders LLC
38 "Basic Commands" (1.3.0-dev) by AlliedModders LLC
39 "TeamSwitch" (1.3) by MistaGee

Other servers with similar setup never had the 100% cpu usage though. Although if it actually "crashed" the core dump might have been easier to debug?

Regards,
Eric

p.s. almost forgot, i noticed that it hangs "after" a map restart. Sometimes i see plugins load, sometimes i can see it dropping users, then hust hangs there.. (but from the last 2 times, it was pretty quick "after" the map change) also with as good as 0 users on it.

[pheadxdll]

100% CPU followed by a server hang can be caused by a plugin stuck in an infinite loop, that might be good place to look.

But yes, the core dumps caused by crashes are easier to debug, I'm not even sure you can debug a memory dump.

I'd suggest removing plugins, sourcemod, then metamod until it stops happening. If it doesn't, send valve an email. Good luck!

[snelvuur]

Well yes, one could just turn of the plugin one after another, but that could takes ages since it does not happen all the time. I made a dump, i want to be able to analyse it or if somebody else can would be great too. I prefer some tips on how to do this though. Then i can just run gdb whenever there is a crash in the future.

To be honoust, you cant expect me to disable each plugin and start them up one by one everytime a new update came out? or when it crashes? That would take massive ammount of time. I am hoping this approach is possible (since i've seen people asking for proper core dumps on some posts)

I've also run sm_dump_handles several times now and then, just to see if there are more occurences of a certain smx file. If it was the case, i removed it to be certain.

[pheadxdll]

It's not that difficult to remove all the third-party plugins on your server and see if the hangup persists.

[snelvuur]

True, your advise is noted if i want to go down that path but for the moment i want to know more about the whole debugging aspect.

I am aware that a sollution can also be found by disabling plugins and so on.

[DontWannaName]

I would look at the 2 plugins that I have never seen,

"lethalpoints.smx"
"FragFM Request/Shoutout" (1.0) by Lethal (uknemesis-server.net)

[voogru]

Hey, that's interesting.

I'm experiencing a similar problem and trying to determine if it's something naughty I'm doing or a bug with TF2/engine.

I'm going to run some experiments. The symptoms are identicle, I don't run sourcemod just custom stuff.

[blik]

I also get this occasionally too, and also seems to happen after map change. The last logged events are all the players joining the game.

100% CPU lockup - sometimes I'm not able to even connect to the box to kill the process and end up doing remote power cycle...

I'm on Windows btw...

[snelvuur]

does anybody know if this problem was addressed/fixed? i do notice less crashes, but have not tested fully.

UDPATE: have not seen any crashes since..