Raised This Month: $9 Target: $400
 2% 

[CRASH] SM 1.11.0.6634 Linux Profiler VPROF segfaults


Post New Thread Reply   
 
Thread Tools Display Modes
Author Message
canadianjeff
AlliedModders Donor
Join Date: Sep 2016
Old 10-17-2020 , 06:10   [CRASH] SM 1.11.0.6634 Linux Profiler VPROF segfaults
Reply With Quote #1

Code:
sm exts list
[SM] Displaying 8 extensions:
[01] Automatic Updater (1.11.0.6634): Updates SourceMod gamedata files
[02] Webternet (1.11.0.6634): Extension for interacting with URLs
[03] Accelerator (2.5.0): SRCDS Crash Handler
[04] Client Preferences (1.11.0.6634): Saves client preference settings
[05] SQLite (1.11.0.6634): SQLite Driver
[06] SDK Tools (1.11.0.6634): Source SDK Tools
[07] BinTools (1.11.0.6634): Low-level C/C++ Calling API
[08] Top Menus (1.11.0.6634): Creates sorted nested menus

meta list
Listing 2 plugins:
  [01] SourceMod (1.11.0.6634) by AlliedModders LLC
  [02] SDK Tools (1.11.0.6634) by AlliedModders LLC

sm plugins list
[SM] Listing 16 plugins:
  01 "Admin File Reader" (1.11.0.6634) by AlliedModders LLC
  02 "Admin Help" (1.11.0.6634) by AlliedModders LLC
  03 "Admin Menu" (1.11.0.6634) by AlliedModders LLC
  04 "Anti-Flood" (1.11.0.6634) by AlliedModders LLC
  05 "Basic Ban Commands" (1.11.0.6634) by AlliedModders LLC
  06 "Basic Chat" (1.11.0.6634) by AlliedModders LLC
  07 "Basic Comm Control" (1.11.0.6634) by AlliedModders LLC
  08 "Basic Commands" (1.11.0.6634) by AlliedModders LLC
  09 "Basic Info Triggers" (1.11.0.6634) by AlliedModders LLC
  10 "Basic Votes" (1.11.0.6634) by AlliedModders LLC
  11 "Client Preferences" (1.11.0.6634) by AlliedModders LLC
  12 "Fun Commands" (1.11.0.6634) by AlliedModders LLC
  13 "Fun Votes" (1.11.0.6634) by AlliedModders LLC
  14 "Player Commands" (1.11.0.6634) by AlliedModders LLC
  15 "Reserved Slots" (1.11.0.6634) by AlliedModders LLC
  16 "Sound Commands" (1.11.0.6634) by AlliedModders LLC

sm prof
Profiling commands:
    list             - List all available profiling tools.
    start            - Start a profile with a given tool.
    stop             - Stop the current profile session.
    dump             - Dumps output from the current profile session.
    help             - Display help text for a profiler.

sm prof list
Profiling tools:
    vprof            - Valve built-in profiler

sm prof start vprof
Started profiling with vprof.
Wrote minidump to: /opt/steam/L4D2_LAST_STAND/left4dead2/addons/sourcemod/data/dumps/ac5034e7-a3f7-43b9-86526fb1-329268c2.dmp
Segmentation fault (core dumped)
BFD: warning: /opt/steam/L4D2_LAST_STAND/core is truncated: expected core file size >= 429481984, found: 1069056
Cannot access memory at address 0xf7f26934
Cannot access memory at address 0xf7f26930
Failed to read a valid object file image from memory.
Cannot access memory at address 0xf7f26934
Cannot access memory at address 0xf7f26930
debug.cmds:5: Error in sourced command file:
No function contains program counter for selected frame.
email debug.log to [email protected]
Sat 17 Oct 2020 02:03:45 AM PDT: Server restart in 10 seconds
#
#Console initialized.
Code:
[email protected]:/opt/steam/L4D2_LAST_STAND# gdb -p 1756
GNU gdb (Debian 8.2.1-2+b3) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 1756
[New LWP 1757]
[New LWP 1759]
[New LWP 1762]
[New LWP 1764]
[New LWP 1768]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0xf7fc5069 in __kernel_vsyscall ()
(gdb) continue
Continuing.

Thread 1 "srcds_linux" received signal SIGSEGV, Segmentation fault.
0xf7ba37ba in CVProfNode::ExitScope() () from bin/libtier0_srv.so
(gdb) bt
#0  0xf7ba37ba in CVProfNode::ExitScope() () from bin/libtier0_srv.so
#1  0xf7303f6c in _Host_RunFrame(float) () from /opt/steam/L4D2_LAST_STAND/bin/engine_srv.so
#2  0xf730e010 in CHostState::State_Run(float) () from /opt/steam/L4D2_LAST_STAND/bin/engine_srv.so
#3  0xf730e21e in CHostState::FrameUpdate(float) () from /opt/steam/L4D2_LAST_STAND/bin/engine_srv.so
#4  0xf730e329 in HostState_Frame(float) () from /opt/steam/L4D2_LAST_STAND/bin/engine_srv.so
#5  0xf73a0e53 in CEngine::Frame() () from /opt/steam/L4D2_LAST_STAND/bin/engine_srv.so
#6  0xf739df26 in CDedicatedServerAPI::RunFrame() () from /opt/steam/L4D2_LAST_STAND/bin/engine_srv.so
#7  0xf761cd0c in RunServerIteration(bool) () from bin/dedicated_srv.so
#8  0xf761cda8 in RunServer(bool) () from bin/dedicated_srv.so
#9  0xf739e00d in CModAppSystemGroup::Main() () from /opt/steam/L4D2_LAST_STAND/bin/engine_srv.so
#10 0xf73ef7f0 in CAppSystemGroup::Run() () from /opt/steam/L4D2_LAST_STAND/bin/engine_srv.so
#11 0xf739ee1f in CDedicatedServerAPI::ModInit(ModInfo_t&) () from /opt/steam/L4D2_LAST_STAND/bin/engine_srv.so
#12 0xf761ce80 in CDedicatedAppSystemGroup::Main() () from bin/dedicated_srv.so
#13 0xf763c030 in CAppSystemGroup::Run() () from bin/dedicated_srv.so
#14 0xf763c030 in CAppSystemGroup::Run() () from bin/dedicated_srv.so
#15 0xf75e84e5 in main () from bin/dedicated_srv.so
#16 0x080486f1 in main ()
(gdb)

Last edited by canadianjeff; 10-17-2020 at 06:54. Reason: linux prof tool used
canadianjeff is offline
asherkin
SourceMod Developer
Join Date: Aug 2009
Location: OnGameFrame()
Old 10-17-2020 , 08:33   Re: [CRASH] SM 1.11.0.6634 Linux Profiler VPROF segfaults
Reply With Quote #2

Does it work if you start vprof recording using the game's built-in commands?

Does it work if you do that without SourceMod loaded at all?
__________________
asherkin is offline
Dragokas
Veteran Member
Join Date: Nov 2017
Location: Ukraine
Old 10-17-2020 , 11:23   Re: [CRASH] SM 1.11.0.6634 Linux Profiler VPROF segfaults
Reply With Quote #3

This is a known issue for L4D1/2 games.

Looks like I managed to find a solution:

Write in RCON:
Code:
vprof_on
Then start the profiler:
Code:
sm prof start vprof
It will not crash anymore.

To stop the profiler:
Code:
sm prof stop vprof
Dump it to console.log (useful together with -debug -condebug srcds startup options):
Code:
sm prof dump vprof
Finally, disable the profiler via RCON:
Code:
vprof_off
__________________
Expert of CMD/VBS/VB6. Malware analyst. L4D fun (Bloody Witch)
[My plugins] [My tools] [GitHub] [Articles] [Donate]
Dragokas is offline
Dragokas
Veteran Member
Join Date: Nov 2017
Location: Ukraine
Old 10-17-2020 , 11:30   Re: [CRASH] SM 1.11.0.6634 Linux Profiler VPROF segfaults
Reply With Quote #4

Does anybody know, can valve profiler generate a log in real-time (to be able to see the call stack before the crash) ?

Here is a complete list of commands:
Spoiler


That potentially could be extremely useful, because we can see the detailing over the plugin and method names. Sample:
Spoiler
__________________
Expert of CMD/VBS/VB6. Malware analyst. L4D fun (Bloody Witch)
[My plugins] [My tools] [GitHub] [Articles] [Donate]

Last edited by Dragokas; 10-17-2020 at 11:34.
Dragokas is offline
canadianjeff
AlliedModders Donor
Join Date: Sep 2016
Old 10-18-2020 , 01:14   Re: [CRASH] SM 1.11.0.6634 Linux Profiler VPROF segfaults
Reply With Quote #5

TY Dragokas this seems to have solved the issue

I think SM PROF should check to see if vprof is on or it will segfault/crash
canadianjeff is offline
canadianjeff
AlliedModders Donor
Join Date: Sep 2016
Old 06-22-2021 , 05:34   Re: [CRASH] SM 1.11.0.6634 Linux Profiler VPROF segfaults
Reply With Quote #6

I need to head out back into the barn and grab my shovel to start digging to attempt to find someone who can assist with Valves Profiler...... VPROF

I have some money I am willing to share for just a couple minutes of someones time to give a quick crash course on how to actually use VPROF to determine whats lagging the server the most to remove any offending plugins or assist in making L4D(2) not be so damn laggy even with meta/source mod unloaded and on a virgin server
canadianjeff is offline
Silvers
SourceMod Plugin Approver
Join Date: Aug 2010
Location: SpaceX
Old 06-22-2021 , 07:38   Re: [CRASH] SM 1.11.0.6634 Linux Profiler VPROF segfaults
Reply With Quote #7

Quote:
Originally Posted by canadianjeff View Post
assist in making L4D(2) not be so damn laggy even with meta/source mod unloaded and on a virgin server
It's laggy without SM? Then it's not any plugins fault but the server. Maybe the games network settings are wrong not allowing the right amount of packets/sizes to be sent or the server itself is bad, either bandwidth or CPU. There are posts about network settings for L4D2, one was bumped recently somewhere, Lux suggested some details.
__________________

Last edited by Silvers; 06-22-2021 at 07:38.
Silvers is offline
canadianjeff
AlliedModders Donor
Join Date: Sep 2016
Old 06-22-2021 , 08:42   Re: [CRASH] SM 1.11.0.6634 Linux Profiler VPROF segfaults
Reply With Quote #8

well let me be 100% open and honest here when I am playing with myself the 100 tickrate enabler everything runs fine a few random people join everythings fine but then out of the blue you get that 1 @SSHOLE who injects his hacks into the server and things really start to lag I want to use VPROF right when they inject hacks and catch them right as they do it..... im trying to track down the hit reg issue I notice hits start to miss and developer console goes nuts even on official dedicated servers

I for one want this crap detected and stopped

I dont mind dumping a VPROF file right when things start to lag and handing it to someone to take a look

the most common time this happens is in versus mode with 8 people on the server rarely happens in coop with just 4

Last edited by canadianjeff; 06-22-2021 at 08:44.
canadianjeff is offline
asherkin
SourceMod Developer
Join Date: Aug 2009
Location: OnGameFrame()
Old 06-22-2021 , 09:21   Re: [CRASH] SM 1.11.0.6634 Linux Profiler VPROF segfaults
Reply With Quote #9

Console printing in SRCDS is extremely expensive, so if there is an issue that is allowing people to spam the server console that could just be the cause of the performance issues.
__________________
asherkin is offline
Dragokas
Veteran Member
Join Date: Nov 2017
Location: Ukraine
Old 06-22-2021 , 16:13   Re: [CRASH] SM 1.11.0.6634 Linux Profiler VPROF segfaults
Reply With Quote #10

canadianjeff, well, you have lags randomly, but you still unsure who or what caused it?

If that's a hardware issue:

1) you must have some control panel to be able to check:
- CPU loading
- memory loading
- network channel (not always)
If on VPS, you have much more options, at least try ping some public service

2) enter net_graph 4 in console, you can estimate CPU / network loading per entity types.

3) try to change the map to exclude entities overflow issue.

4) Check handles leaking. Instruction is in Section 11.

5) Try command sm plugins unload_all - (may crash your server) - it will forcibly unload all your plugins.

6) Check addons/sourcemod/logs for error*.log

If that's human-related:

1) There is a function:

Quote:
float GetGameFrameTime()
Returns the time the Game took processing the last frame.
You could check it by some repeatable timer against some pre-defined constant and catch the moment, when the server begins to lag to start logging who is on the server at the moment for giving a ban.

2) As asherkin said, that could be commands spamming.

I'm sure somewhere should be a solution for this. But, if not, I think you could intercept messages and force a minimum delay between commands, something like:
PHP Code:
#include <sourcemod>

float g_fPressTime[MAXPLAYERS+1];

public 
Action OnClientCommand(int clientint args)
{
    if( 
GetEngineTime() - g_fPressTime[client] > 0.5 )
    {
        
g_fPressTime[client] = GetEngineTime();
        return 
Plugin_Continue;
    }
    return 
Plugin_Handled;

This may cause some problems, so better to prepare the whitelist of commands, and block only those unrecognized.

Also, check console.log for some unusual (too repeatable) commands from players.

3) Change rcon password if that's too short/simple.

4) Install all know fixes against bad performance / exploits known which published in this topic: [TUT] SourcePawn Scripting - Tips, Basics to Advanced (Section 6. Server Fixes).

Good luck!
__________________
Expert of CMD/VBS/VB6. Malware analyst. L4D fun (Bloody Witch)
[My plugins] [My tools] [GitHub] [Articles] [Donate]

Last edited by Dragokas; 06-22-2021 at 16:23.
Dragokas is offline
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -4. The time now is 17:11.


Powered by vBulletin®
Copyright ©2000 - 2021, vBulletin Solutions, Inc.
Theme made by Freecode