Raised This Month: $84 Target: $400
 21% 

Accelerator - Crash Reporting That Doesn't Suck


Post New Thread Reply   
 
Thread Tools Display Modes
Luckylock
Junior Member
Join Date: Jan 2019
Old 04-27-2019 , 13:05   Re: Accelerator - Crash Reporting That Doesn't Suck
Reply With Quote #321

Hey asher, I use the crash command to restart my server after every game. Would it be possible to ignore intentional crashes in the logs?
Luckylock is offline
Lux
Veteran Member
Join Date: Jan 2015
Location: Brexit
Old 04-27-2019 , 16:05   Re: Accelerator - Crash Reporting That Doesn't Suck
Reply With Quote #322

Quote:
Originally Posted by spumer View Post
Hi. I have problem with dumping symbols.

I got crash with `ladder_rambos.ext.so + 0xc09e`
And try to dump this binary manually: nm -nC ladder_rambos.ext.so

But in dump output I can't find function with given offset (0xc09e).

Accelerator: 2.3.3 (i use old sourcemod)

Why offsets can be different?
I checked the offset in the new binary according to the IDA offset bar the offset has not changed unless my brain is not functioning.

Function dump from IDA
Code:
.text:0000C060
.text:0000C060
.text:0000C060                               ; _DWORD LadderSafeDrop::Patch(LadderSafeDrop *__hidden this)
.text:0000C060                               _ZN14LadderSafeDrop5PatchEv proc near   ; CODE XREF: LadderSafeDrop::OnExtensionStateChanged(IConVar *,char const*,float):loc_C2B0↓p
.text:0000C060
.text:0000C060                               name            = dword ptr -1Ch
.text:0000C060                               len             = dword ptr -18h
.text:0000C060                               prot            = dword ptr -14h
.text:0000C060                               this            = dword ptr  4
.text:0000C060
.text:0000C060                               ; __unwind {
.text:0000C060 56                                            push    esi
.text:0000C061 53                                            push    ebx
.text:0000C062 83 EC 14                                      sub     esp, 14h
.text:0000C065 8B 1D E0 25 02 00                             mov     ebx, ds:_ZL23pCTerrorPlayer_PreThink ; pCTerrorPlayer_PreThink
.text:0000C06B 85 DB                                         test    ebx, ebx
.text:0000C06D 74 39                                         jz      short loc_C0A8
.text:0000C06F C7 04 24 1E 00 00 00                          mov     [esp+1Ch+name], 1Eh ; name
.text:0000C076 8B 35 D0 25 02 00                             mov     esi, ds:_ZL6offset ; offset
.text:0000C07C E8 4B 7D 01 00                                call    sysconf
.text:0000C081 C7 44 24 08 07 00 00 00                       mov     [esp+1Ch+prot], 7 ; prot
.text:0000C089 89 44 24 04                                   mov     [esp+1Ch+len], eax ; len
.text:0000C08D 89 D8                                         mov     eax, ebx
.text:0000C08F 01 F3                                         add     ebx, esi
.text:0000C091 25 00 F0 FF FF                                and     eax, 0FFFFF000h
.text:0000C096 89 04 24                                      mov     [esp+1Ch+name], eax ; addr
.text:0000C099 E8 FE 7C 01 00                                call    mprotect
.text:0000C09E
.text:0000C09E                               loc_C09E:                               ; CODE XREF: LadderSafeDrop::Patch(void)+133↓j
.text:0000C09E C6 03 14                                      mov     byte ptr [ebx], 14h
.text:0000C0A1 83 C4 14                                      add     esp, 14h
.text:0000C0A4 5B                                            pop     ebx
.text:0000C0A5 5E                                            pop     esi
.text:0000C0A6 C3                                            retn
.text:0000C0A6                               ; ---------------------------------------------------------------------------
.text:0000C0A7 90                                            align 4
.text:0000C0A8
.text:0000C0A8                               loc_C0A8:                               ; CODE XREF: LadderSafeDrop::Patch(void)+D↑j
.text:0000C0A8 8B 0D E0 24 02 00                             mov     ecx, ds:g_pGameConf
.text:0000C0AE 8B 11                                         mov     edx, [ecx]
.text:0000C0B0 C7 44 24 08 E0 25 02 00                       mov     [esp+1Ch+prot], offset _ZL23pCTerrorPlayer_PreThink ; pCTerrorPlayer_PreThink
.text:0000C0B8 C7 44 24 04 30 C3 01 00                       mov     [esp+1Ch+len], offset aCterrorplayerP ; "CTerrorPlayer::PreThink"
.text:0000C0C0 89 0C 24                                      mov     [esp+1Ch+name], ecx
.text:0000C0C3 FF 52 0C                                      call    dword ptr [edx+0Ch]
.text:0000C0C6 84 C0                                         test    al, al
.text:0000C0C8 74 5E                                         jz      short loc_C128
.text:0000C0CA 8B 1D E0 25 02 00                             mov     ebx, ds:_ZL23pCTerrorPlayer_PreThink ; pCTerrorPlayer_PreThink
.text:0000C0D0 85 DB                                         test    ebx, ebx
.text:0000C0D2 74 54                                         jz      short loc_C128
.text:0000C0D4 8B 35 E0 24 02 00                             mov     esi, ds:g_pGameConf
.text:0000C0DA 8B 06                                         mov     eax, [esi]
.text:0000C0DC C7 44 24 08 D0 25 02 00                       mov     [esp+1Ch+prot], offset _ZL6offset ; offset
.text:0000C0E4 C7 44 24 04 94 C3 01 00                       mov     [esp+1Ch+len], offset aCterrorplayerP_0 ; "CTerrorPlayer::PreThink__SafeDropLogic"
.text:0000C0EC 89 34 24                                      mov     [esp+1Ch+name], esi
.text:0000C0EF FF 10                                         call    dword ptr [eax]
.text:0000C0F1 84 C0                                         test    al, al
.text:0000C0F3 74 0A                                         jz      short loc_C0FF
.text:0000C0F5 8B 1D D0 25 02 00                             mov     ebx, ds:_ZL6offset ; offset
.text:0000C0FB 85 DB                                         test    ebx, ebx
.text:0000C0FD 75 51                                         jnz     short loc_C150
.text:0000C0FF
.text:0000C0FF                               loc_C0FF:                               ; CODE XREF: LadderSafeDrop::Patch(void)+93↑j
.text:0000C0FF A1 24 21 02 00                                mov     eax, ds:g_pSM
.text:0000C104 8B 35 50 21 02 00                             mov     esi, ds:myself
.text:0000C10A 8B 18                                         mov     ebx, [eax]
.text:0000C10C 89 74 24 04                                   mov     [esp+1Ch+len], esi
.text:0000C110 C7 44 24 08 BC C3 01 00                       mov     [esp+1Ch+prot], offset aLadderRambosCo_9 ; "Ladder Rambos -- Could not obtain offse"...
.text:0000C118 89 04 24                                      mov     [esp+1Ch+name], eax
.text:0000C11B FF 53 1C                                      call    dword ptr [ebx+1Ch]
.text:0000C11E 83 C4 14                                      add     esp, 14h
.text:0000C121 5B                                            pop     ebx
.text:0000C122 5E                                            pop     esi
.text:0000C123 C3                                            retn
.text:0000C123                               ; ---------------------------------------------------------------------------
.text:0000C124 8D 74 26 00                                   align 8
.text:0000C128
.text:0000C128                               loc_C128:                               ; CODE XREF: LadderSafeDrop::Patch(void)+68↑j
.text:0000C128                                                                       ; LadderSafeDrop::Patch(void)+72↑j
.text:0000C128 A1 24 21 02 00                                mov     eax, ds:g_pSM
.text:0000C12D 8B 0D 50 21 02 00                             mov     ecx, ds:myself
.text:0000C133 8B 10                                         mov     edx, [eax]
.text:0000C135 C7 44 24 08 48 C3 01 00                       mov     [esp+1Ch+prot], offset aLadderRambosCo_10 ; "Ladder Rambos -- Could not obtain signa"...
.text:0000C13D 89 4C 24 04                                   mov     [esp+1Ch+len], ecx
.text:0000C141 89 04 24                                      mov     [esp+1Ch+name], eax
.text:0000C144 FF 52 1C                                      call    dword ptr [edx+1Ch]
.text:0000C147 83 C4 14                                      add     esp, 14h
.text:0000C14A 5B                                            pop     ebx
.text:0000C14B 5E                                            pop     esi
.text:0000C14C C3                                            retn
.text:0000C14C                               ; ---------------------------------------------------------------------------
.text:0000C14D 8D 76 00                                      align 10h
.text:0000C150
.text:0000C150                               loc_C150:                               ; CODE XREF: LadderSafeDrop::Patch(void)+9D↑j
.text:0000C150 C7 04 24 1E 00 00 00                          mov     [esp+1Ch+name], 1Eh ; name
.text:0000C157 8B 35 E0 25 02 00                             mov     esi, ds:_ZL23pCTerrorPlayer_PreThink ; pCTerrorPlayer_PreThink
.text:0000C15D E8 6A 7C 01 00                                call    sysconf
.text:0000C162 C7 44 24 08 07 00 00 00                       mov     [esp+1Ch+prot], 7 ; prot
.text:0000C16A 89 F1                                         mov     ecx, esi
.text:0000C16C 01 F3                                         add     ebx, esi
.text:0000C16E 81 E1 00 F0 FF FF                             and     ecx, 0FFFFF000h
.text:0000C174 89 0C 24                                      mov     [esp+1Ch+name], ecx ; addr
.text:0000C177 89 44 24 04                                   mov     [esp+1Ch+len], eax ; len
.text:0000C17B E8 1C 7C 01 00                                call    mprotect
.text:0000C180 0F B6 13                                      movzx   edx, byte ptr [ebx]
.text:0000C183 C7 05 C4 25 02 00 01 00 00 00                 mov     ds:dword_225C4, 1
.text:0000C18D 88 15 B0 25 02 00                             mov     ds:_ZL30pCTerrorPlayer_PreThinkRestore, dl ; pCTerrorPlayer_PreThinkRestore
.text:0000C193 E9 06 FF FF FF                                jmp     loc_C09E
.text:0000C193                               ; } // starts at C060
.text:0000C193                               _ZN14LadderSafeDrop5PatchEv endp
.text:0000C193
.text:0000C193                               ; ---------------------------------------------------------------------------
Point of crash in IDA by the look of it.
Code:
.text:0000C150 C7 04 24 1E 00 00 00                          mov     [esp+1Ch+name], 1Eh ; name
.text:0000C157 8B 35 E0 25 02 00                             mov     esi, ds:_ZL23pCTerrorPlayer_PreThink ; pCTerrorPlayer_PreThink
.text:0000C15D E8 6A 7C 01 00                                call    sysconf
.text:0000C162 C7 44 24 08 07 00 00 00                       mov     [esp+1Ch+prot], 7 ; prot
.text:0000C16A 89 F1                                         mov     ecx, esi
.text:0000C16C 01 F3                                         add     ebx, esi
.text:0000C16E 81 E1 00 F0 FF FF                             and     ecx, 0FFFFF000h
.text:0000C174 89 0C 24                                      mov     [esp+1Ch+name], ecx ; addr
.text:0000C177 89 44 24 04                                   mov     [esp+1Ch+len], eax ; len
.text:0000C17B E8 1C 7C 01 00                                call    mprotect
.text:0000C180 0F B6 13                                      movzx   edx, byte ptr [ebx]
.text:0000C183 C7 05 C4 25 02 00 01 00 00 00                 mov     ds:dword_225C4, 1
.text:0000C18D 88 15 B0 25 02 00                             mov     ds:_ZL30pCTerrorPlayer_PreThinkRestore, dl ; pCTerrorPlayer_PreThinkRestore
.text:0000C193 E9 06 FF FF FF                                jmp     loc_C09E
Throttle crash dump.
Code:
Thread 0 (crashed):
  0: ladder_rambos.ext.so!LadderSafeDrop::Patch() + 0x3e
     eax: 0x00000000  ebp: 0xffddacd8  ebx: 0xee596117
     ecx: 0x00001000  edi: 0x00000002  edx: 0x00000089
     efl: 0x00210217  eip: 0xe7f4f09e  esi: 0xee595fc0
     esp: 0xffddac50  

     e7f4f08d  89 d8           mov eax, ebx
     e7f4f08f  01 f3           add ebx, esi
     e7f4f091  25 00 f0 ff ff  and eax, 0xfffff000
     e7f4f096  89 04 24        mov [esp], eax
     e7f4f099  e8 c2 f5 d2 0f  call 0xf7c7e660
  >  e7f4f09e  c6 03 14        mov byte [ebx], 0x14
     e7f4f0a1  83 c4 14        add esp, 0x14
     e7f4f0a4  5b              pop ebx
     e7f4f0a5  5e              pop esi
     e7f4f0a6  c3              ret
     e7f4f0a7  90              nop

     ffddac50  00 50 59 ee 00 10 00 00  07 00 00 00 00 00 00 00  |.PY.............|
     ffddac60  31 00 00 00 c0 52 f6 e7  a0 ac dd ff b5 f2 f4 e7  |1....R..........|

     Found via instruction pointer in context
I guess it's crashing when applying the patch?
https://github.com/Attano/LadderRamb..._patch.cpp#L80

I'm still rather new to this forgive me for anything wrong.
__________________
Lux is offline
asherkin
SourceMod Developer
Join Date: Aug 2009
Location: OnGameFrame()
Old 04-27-2019 , 16:24   Re: Accelerator - Crash Reporting That Doesn't Suck
Reply With Quote #323

It appears to be that the mprotect call hasn't taken affect somehow, and the memory is still write-protected.

It's worth noting that PROT_WRITE|PROT_EXEC can be problematic with certain security modules, you might have more luck making it write-only, patching, then switching back to read-and-exec.

The fact the problem came about in a game update makes problems here unlikely unless they've accidentally compiled the server binaries with CEG or something.
__________________
asherkin is offline
Naydef
Senior Member
Join Date: Dec 2015
Location: Doom Town, Nevada
Old 05-03-2019 , 14:39   Re: Accelerator - Crash Reporting That Doesn't Suck
Reply With Quote #324

Hello everyone
I have small question. How I can see which SM function was executing when the crash happened? I checked the stack and i see no readable string and I was under the impression that this pull request will add such functionality. If i'm wrong, what's the point of the pull request, then(I see that the function calling alloca() is ScriptedInvoker::Invoke, while in my case it is PluginContext::Invoke)?
If you ask me for the accelerator crash, this one
I'm pretty sure the server has a version of Sourcemod with the pull request.
__________________
My plugins:
Admin Stealth REDUX(v1.6.1)

Non-Sourcemod related projects:
8086eD x86 Emulator

Steam:
naydef

Last edited by Naydef; 05-03-2019 at 14:42.
Naydef is offline
asherkin
SourceMod Developer
Join Date: Aug 2009
Location: OnGameFrame()
Old 05-03-2019 , 14:51   Re: Accelerator - Crash Reporting That Doesn't Suck
Reply With Quote #325

Quote:
Originally Posted by Naydef View Post
I'm pretty sure the server has a version of Sourcemod with the pull request.
It does not, the SourcePawn version in your SM version is over a year older than that PR.
__________________
asherkin is offline
Naydef
Senior Member
Join Date: Dec 2015
Location: Doom Town, Nevada
Old 05-03-2019 , 15:01   Re: Accelerator - Crash Reporting That Doesn't Suck
Reply With Quote #326

Quote:
Originally Posted by asherkin View Post
It does not, the SourcePawn version in your SM version is over a year older than that PR.
Yea, sadly I noticed after too late, but what about the different names of the functions? PluginContext::Invoke doesn't have alloca(). Does this mean even with updated Sourcemod there won't be a string on the stack?

Edit:
Also can you print Sourcemod version information so we can easily identify whether SM is too old? Because right now server is running SourceMod 1.9.0.6275 which is not so old. Now I suspect incorrect installation.
__________________
My plugins:
Admin Stealth REDUX(v1.6.1)

Non-Sourcemod related projects:
8086eD x86 Emulator

Steam:
naydef

Last edited by Naydef; 05-03-2019 at 15:11.
Naydef is offline
asherkin
SourceMod Developer
Join Date: Aug 2009
Location: OnGameFrame()
Old 05-03-2019 , 17:25   Re: Accelerator - Crash Reporting That Doesn't Suck
Reply With Quote #327

Quote:
Originally Posted by Naydef View Post
Yea, sadly I noticed after too late, but what about the different names of the functions? PluginContext::Invoke doesn't have alloca(). Does this mean even with updated Sourcemod there won't be a string on the stack?
The stack hasn't been walked correctly as it has gone off on a detour as it doesn't understand the JIT frames, here's the correctly processed stack trace for your dump:

Code:
Thread 0 (crashed):
   0: server.dll + 0x1e0395
   1: server.dll + 0x1de625
   2: server.dll + 0x433127
   3: server.dll + 0x433544
   4: server.dll + 0x1956b6
   5: server.dll + 0x210129
   6: server.dll + 0x1a579d
   7: 0x1e13ac28
   8: bintools.ext.dll!CallWrapper::Execute(void *,void *) [CallWrapper.cpp:128 + 0xb]
   9: sdktools.ext.2.tf2.dll!AcceptEntityInput [inputnatives.cpp:113 + 0x3d]
  10: 0x4182ac4b
  11: 0x4182ac4b
  12: 0x1a48006f
  13: sourcepawn.jit.x86.dll!sp::Environment::Invoke(sp::PluginContext *,ke::RefPtr<sp::MethodInfo> const &,int *) [environment.cpp:282 + 0x6]
  14: sourcepawn.jit.x86.dll!sp::PluginContext::Invoke(unsigned int,int const *,unsigned int,int *) [plugin-context.cpp:465 + 0x10]
  15: sourcepawn.jit.x86.dll!sp::ScriptedInvoker::Invoke(int *) [scripted-invoker.cpp:296 + 0x1c]
  16: sourcepawn.jit.x86.dll!sp::ScriptedInvoker::Execute(int *) [scripted-invoker.cpp:190 + 0x13]
  17: sourcemod.logic.dll!TimerNatives::OnTimer(SourceMod::ITimer *,void *) [smn_timers.cpp:144 + 0xb]
  18: sourcemod.2.tf2.dll!TimerSystem::RunFrame() [TimerSys.cpp:275 + 0xe]
  19: sourcemod.2.tf2.dll!TimerSystem::GameFrame(bool) [TimerSys.cpp:232 + 0x7]
  20: sourcemod.2.tf2.dll!__SourceHook_FHCls_IServerGameDLLGameFramefalse::Func(bool) [sourcemod.cpp:54 + 0x6f]
  21: engine.dll + 0x12a9d7
  22: engine.dll + 0x129f5f
  23: engine.dll + 0x17f447
  24: engine.dll + 0x17e4cb
  25: engine.dll + 0x17cb2f
  26: engine.dll + 0x18b03e
  27: engine.dll + 0x18a520
  28: engine.dll + 0x18a664
  29: engine.dll + 0x1d30d6
  30: engine.dll + 0x1cffed
  31: engine.dll + 0x1f54dc
  32: dedicated.dll + 0x7765
  33: dedicated.dll + 0x231ec
  34: dedicated.dll + 0x82c8
  35: exec_necgaming2.exe + 0x158e
  36: exec_necgaming2.exe + 0x17d1
  37: kernel32.dll!BaseThreadInitThunk + 0x12
  38: ntdll.dll!__RtlUserThreadStart + 0x27
  39: ntdll.dll!_RtlUserThreadStart + 0x1b
Quote:
Originally Posted by Naydef View Post
Also can you print Sourcemod version information so we can easily identify whether SM is too old? Because right now server is running SourceMod 1.9.0.6275 which is not so old. Now I suspect incorrect installation.
The PR was only merged into SourcePawn master, so is only in SourceMod master (1.10) - I did mention that a few posts back as well. Accelerator unfortunately doesn't have any way to get a version number from the information is has or is able to gather. Where there are stack frames in SM or MM:S code, the links beside them take you straight to the exact commit however.

EDIT: Sorry, the mention of the SM version was in a different thread for someone's crash report, not in here.
__________________

Last edited by asherkin; 05-03-2019 at 17:27.
asherkin is offline
asherkin
SourceMod Developer
Join Date: Aug 2009
Location: OnGameFrame()
Old 05-22-2019 , 19:09   Re: Accelerator - Crash Reporting That Doesn't Suck
Reply With Quote #328

Joining the rest of us in 2019, the source code for Throttle (the Accelerator backend) now lives on GitHub: https://github.com/asherkin/throttle
__________________
asherkin is offline
ImACow
AlliedModders Donor
Join Date: Feb 2015
Old 05-23-2019 , 10:28   Re: Accelerator - Crash Reporting That Doesn't Suck
Reply With Quote #329

Quote:
Originally Posted by asherkin View Post
Joining the rest of us in 2019, the source code for Throttle (the Accelerator backend) now lives on GitHub: https://github.com/asherkin/throttle
Thank you so much!

Kudos!
__________________
ImACow is offline
Fearts
ferts of daeth
Join Date: Oct 2008
Old 06-14-2019 , 13:05   Re: Accelerator - Crash Reporting That Doesn't Suck
Reply With Quote #330

Is it possible, or would it be possible, to have a way to sort crashes by server in the dash board? Like a way I can see all crashes but only for one or more servers at a time? Even having it for one server at a time would be nice.
__________________
Fearts is offline
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -4. The time now is 04:12.


Powered by vBulletin®
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Theme made by Freecode