[noodleboy347]

So for that past 2 months or so my server has been crashing randomly many times a day. The intervals are random, it could be 5 minutes, 30 minutes, or 4 hours. It only happens when players are playing. I have no idea how to read the generated .dmp files, and I've tried using Asherkin's Accelerator to no avail. At this point I'm completely stuck, I have no way of knowing what is causing crashes and I have no way of fixing it. Help would be deeply appreciated.

I'm running the server on Ubuntu. Here is what appears in the console when the server crashes:

Spoiler

Segmentation fault (core dumped) warning: exec file is newer than core file. warning: Can't read pathname for load map: Input/output error. warning: .dynamic section for "/root/tf_gmg/orangebox/bin/materialsystem_srv.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/tf/bin/server_srv.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/tf/addons/sourcemod/extensions/dbi.sqlite.ext.so" is not at the expected address (wrong library or version mismatch?) debug.cmds:5: Error in sourced command file: No function contains program counter for selected frame. email debug.log to [email protected]

Here is the plugin list, which might not even be helpful since most of them are custom:

Spoiler

[SM] Listing 75 plugins: 01 "[TF2Items] Manager" (1.4.1) by Damizean & Asherkin 02 "Server Crontab Module" (1.0.1.0) by dubbeh 03 "Pinion Adverts" (1.8.1) by Multiple contributors 04 "Telescreen" (1.0) by noodleboy347 05 "Server Crontab" (1.0.1.0) by dubbeh 06 "TF2 Full Infinite Ammo" (1.2.2) by Tylerst 07 "Stunmod" (1.3.3.7) by DarthNinja 08 <Error> "Historical" (2.0) by noodleboy347 (purposely disabled) 09 "Generic Admin Commands" (1.3.2) by noodleboy347 10 "HUD" (2.0.1) by noodleboy347 11 "Evolution" (1.1.1) by noodleboy347 12 "Admin List" (2.0.2) by noodleboy347 13 "Map Reset" (1.0.1) by noodleboy347 14 "Construction" (1.0.4) by noodleboy347 15 "Configs" (1.0) by noodleboy347 16 "Boss Battles" (1.5) by noodleboy347 17 "Gold" (1.2.11) by noodleboy347 18 "Idle Kicker" (1.1) by noodleboy347 19 "Arcade" (3.0.2) by noodleboy347 20 "Items" (2.4.2) by noodleboy347 21 <Error> "Tunnels" (1.1.2) by noodleboy347 (purposely disabled) 22 "Maintenance" (1.0) by noodleboy347 23 "Fireball" (1.0) by noodleboy347 24 "Music" (1.1) by noodleboy347 25 "PvP Duels" (1.0.4) by noodleboy347 26 "Item Tools" (1.0.1) by noodlboy347 27 "Autoupdate" (1.0) by noodleboy347 28 "Download Manager" (1.0.5) by noodleboy347 29 <Error> "Shop" (2.1.4) by noodleboy347 (purposely disabled) 30 <Error> "Bot Control" (1.0) by noodleboy347 (purposely disabled) 31 "Welcome Music" (1.1.2) by noodleboy347 32 "Voting Commands" (1.1) by noodleboy347 33 "Trading" (1.0.1) by noodleboy347 34 "Damage Management" (1.4.2) by noodleboy347 35 <Error> "Coin Drops" (1.0.3) by noodleboy347 (purposely disabled) 36 "Award Achievement" (1.2.1) by noodle & calypso 37 "Noodle" (1.0) by noodleboy347 38 "Airblast Control" (2.0.3) by noodleboy347 39 "Platinum" (1.2.7) by noodleboy347 40 "Chambers Marketplace" (1.0.7) by noodleboy347 41 "Badges" (1.1) by noodleboy347 42 "Noclip Wind" (1.1) by noodleboy347 43 "Pets" (2.2) by noodleboy347 44 "Basic Commands" (1.3.9-dev) by AlliedModders LLC 45 "Forgiveness" (1.2.5) by noodleboy347 46 "Fixes" (1.2) by noodleboy347 47 "Allowance" (2.0.1) by noodleboy347 48 "Tips" (1.2) by noodleboy347 49 "Chambers" (2.0.6) by noodleboy347 50 "Users" (2.5.2) by noodleboy347 51 "Bomb" (1.2.3) by noodleboy347 52 "Drugs" (1.0) by noodleboy347 53 "Bank" (1.0.2) by noodleboy347 54 "Chat" (2.3.1) by noodleboy347 55 "Fireworks" (1.0) by noodleboy347 56 "Setspawn" (1.0) by noodleboy347 57 "Game Description Override" (1.2) by psychonic 58 "Fun Commands" (1.4.0-dev) by AlliedModders LLC 59 "Portal Gun" (1.0) by noodleboy347 60 "Server Clean Up" (1.1.7) by Jamster 61 "Possess" (1.1) by noodleboy347 62 "No Announcer" (1.0) by noodleboy347 63 "Spray Tracer No Menu" (v5.8b) by Nican132, CptMoore, Lebson506th 64 "AdminCheats" (0.2) by devicenull 65 "Basic Chat" (1.3.7-dev) by AlliedModders LLC 66 "Fake Execute" (1.0) by noodleboy347 67 "[TF2Items] Give Weapon" (3.10) by asherkin (Updated by FlaminSarge) 68 "Improved Sounds" (1.0) by noodleboy347 69 "MOTD Backpack Enhanced" (1.0.5) by Bottiger, Munra 70 "Fun Votes" (1.4.0-dev) by AlliedModders LLC 71 "Hat Removal" (1.0) by noodleboy347 72 "Commands" (1.0) by noodleboy347 73 "Basic Votes" (1.4.0-dev) by AlliedModders LLC 74 "Player Commands" (1.4.0-dev) by AlliedModders LLC 75 "Tidy Chat" (0.4) by linux_lover

And extensions:

Spoiler

[SM] Displaying 16 extensions: [01] Automatic Updater (1.5.0-dev+3742): Updates SourceMod gamedata files [02] Webternet (1.5.0-dev+3742): Extension for interacting with URLs [03] Equinox Console Cleaner (): Bite me alien boi [04] TF2Items (1.5.3): TF2 Item Modifier [05] TF2 Tools (1.5.0-dev+3742): TF2 extended functionality [06] BinTools (1.5.0-dev+3742): Low-level C/C++ Calling API [07] SDK Tools (1.5.0-dev+3742): Source SDK Tools [08] SDK Hooks (2.2.0b4): Source SDK Hooks [09] Sound Info Library (0.9): Access information of sound files [10] Regex (1.5.0-dev+3742): Provides regex natives for plugins [11] Socket (3.0.1): Socket extension for SourceMod [12] SteamTools (0.8.3): SteamWorks for SourceMod. [13] Top Menus (1.5.0-dev+3742): Creates sorted nested menus [14] Client Preferences (1.5.0-dev+3742): Saves client preference settings [15] SQLite (1.5.0-dev+3742): SQLite Driver [16] MySQL-DBI (1.5.0-dev+3742): MySQL driver implementation for DBI

And attached are all the (non-empty) crash dumps from the past 3 days. Big thanks in advance to anyone who attempts to help.

[TnTSCS]

If any of those plugins use gamedata files, have you double checked to ensure they're up to date and have the correct values?

[asherkin]

Most of your crashes appear to be in SourceMod, but don't match the version number you provided (something patching SourceMod's memory can also cause this).

[noodleboy347]

I'm pretty sure all gamedata is up to date, or at least they're from the latest snapshots and get updated on launch. What would cause something to patch Sourcemod's memory?

And as quick little test, I tried downgrading to 1.4.7; but it just started crashing every time someone connected.

[_Misfit_]

same problem , random crashes . The problem is inside soursemod for sure

[noodleboy347]

New stuff:

warning: exec file is newer than core file. warning: Can't read pathname for load map: Input/output error. warning: .dynamic section for "bin/libvstdlib_srv.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "bin/libsteam_api.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/bin/materialsystem_srv.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/bin/shaderapiempty_srv.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/tf/bin/server_srv.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/tf/addons/sourcemod/bin/sourcemod_mm_i486.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/tf/addons/sourcemod/bin/sourcemod.logic.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/tf/addons/sourcemod/bin/sourcepawn.jit.x86.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/tf/addons/sourcemod/extensions/updater.ext.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/tf/addons/sourcemod/extensions/webternet.ext.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/tf/addons/sourcemod/extensions/game.tf2.ext.2.ep2v.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/tf/addons/sourcemod/extensions/bintools.ext.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/tf/addons/sourcemod/extensions/regex.ext.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/tf/addons/sourcemod/extensions/topmenus.ext.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/tf/addons/sourcemod/extensions/clientprefs.ext.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/tf/addons/sourcemod/extensions/dbi.sqlite.ext.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "bin/libsteam.so" is not at the expected address (wrong library or version mismatch?) warning: .dynamic section for "/root/tf_gmg/orangebox/tf/addons/sourcemod/extensions/dbi.mysql.ext.so" is not at the expected address (wrong library or version mismatch?) debug.cmds:5: Error in sourced command file: No function contains program counter for selected frame. email debug.log to [email protected]

[noodleboy347]

I'm just going to go ahead and guess that this is unfixable?