[Mystik Spiral]

Posted another video showcasing an all bot team (using this plugin) completing Tank Challenge v1.5, and all four bots escape!

https://www.youtube.com/watch?v=mdOp8TufaBc

[Zunzunzun]

Hello im having this error being spammed on the logs by the plugin using kerouha fork.

PHP Code:

L 10/28/2024 - 23:21:46: [SM] Exception reported: Property "m_isDualWielding" not found (entity 250/weapon_chainsaw)
L 10/28/2024 - 23:21:46: [SM] Blaming: l4d2_sb_ai_improver.smx
L 10/28/2024 - 23:21:46: [SM] Call stack trace:
L 10/28/2024 - 23:21:46: [SM]   [0] GetEntProp
L 10/28/2024 - 23:21:46: [SM]   [1] Line 5820, D:\l4d2\left4dead2\addons\sourcemod\scripting\l4d2_sb_ai_improver.sp::SurvivorHasPistol
L 10/28/2024 - 23:21:46: [SM]   [2] Line 3865, D:\l4d2\left4dead2\addons\sourcemod\scripting\l4d2_sb_ai_improver.sp::L4D2_OnFindScavengeItem
L 10/28/2024 - 23:21:46: [SM]   [4] Call_Finish
L 10/28/2024 - 23:21:46: [SM]   [5] Line 3609, l4dd\l4dd_forwards.sp::DTR_SurvivorBot_FindScavengeItem_Post
L 10/28/2024 - 23:32:53: Error log file session closed. 


Doesn't seem to cause any issues while playing but its kinda annoying having the error logs being spammed like that.

[Mika Misori]

Originally Posted by Kerouha Wondering if it's gonna resolve random crashes

Here comes the big bug report

Since I installed this plugin on servers and switched to SourceMod v1.12 with latest version of VScript, the servers started crashing quite often.

Most of the crashes are with the following error:

server_srv.so!Studio_BoneIndexByName(CStudioHdr const*, char const*) + 0x50

Here is an example of a stack trace of one of the many same crashes:
https://crash.limetech.org/6rczjeszhliq

Spoiler

4: jit_code_3592421376_1048576 + 0x5b970 [ l4d2_sb_ai_improver.smx::.118404.GetClosestTo EyePosEntityBonePos ]
eip: 0xd625b970 esp: 0xffb64330 ebp: 0xffb64358 ebx: 0xdaa3d890
esi: 0xda6e8010 edi: 0x00355850

ffb64330 60 f5 21 0c 60 d8 a3 da e4 16 35 00 7f 00 00 00 00 `.!.`.....5.....
ffb64340 90 04 00 00 00 03 00 00 00 00 58 43 b6 ff 70 b9 25 d6 ........XC..p.%.
ffb64350 84 ce 01 00 00 02 00 00 00 00 68 43 b6 ff 4f fb 21 d6 ........hC..O.!!!!

Found via call frame info


5: jit_code_3592421376_1048576 + 0x1fb4f [ l4d2_sb_ai_improver.smx::.56628.SurvivorBotTh ink ]
eip: 0xd621fb4f esp: 0xffb64360 ebp: 0xffb64368

ffb64360 34 dd 00 00 02 00 00 00 00 78 43 b6 ff a2 8f 9b d9 4.......xC......

Found via previous frame's frame pointer


6: jit_code_3649884160_1048576 + 0xebfa2 [ l4d2_sb_ai_improver.smx::OnPlayerRunCmd ]
eip: 0xd99b8fa2 esp: 0xffb64370 ebp: 0xffb64378

ffb64370 ec cf 00 00 02 00 00 00 00 00 98 43 b6 ff 6f 00 30 de .........C..o.0.

Found via previous frame's frame pointer

A player reported that this crash in-game occurred on the official Crash Course campaign when a bot tried to use the defibrillator on another bot. The crash occurred immediately after the bot used the defibrillator.

Recently there was a crash with a new error, but also referring to your plugin:

server_srv.so!CBaseAnimating::LookupBone(char const*) + 0x18.

Here is the stacktrace of this crash with the new error:
https://crash.limetech.org/fmkqkzcupe57

Spoiler

4: jit_code_3705090048_1048576 + 0x90480 [ l4d2_sb_ai_improver.smx::.118404.GetClosestTo EyePosEntityBonePos ]
eip: 0xdce03480 esp: 0xffa97b50 ebp: 0xffa97b78 ebx: 0xd8481890
esi: 0xd812c010 edi: 0x00355850

ffa97b50 e0 f6 aa 0b 60 18 48 d8 e4 16 35 00 e2 00 00 00 00 ....`.H...5.....
ffa97b60 90 04 00 00 00 03 00 00 00 00 00 78 7b a9 ff 80 34 e0 dc ........x{...4...
ffa97b70 84 ce 01 00 00 02 00 00 00 00 88 7b a9 ff 6f 70 d9 dc .........{..op....

Found via call frame info


5: jit_code_3705090048_1048576 + 0x2406f [ l4d2_sb_ai_improver.smx::.56628.SurvivorBotTh ink ].
eip: 0xdcd9706f esp: 0xffa97b80 ebp: 0xffa97b88

ffa97b80 34 dd 00 00 00 02 00 00 00 00 00 98 7b a9 ff d2 1d 44 d7 4........{....D.

Found via previous frame's frame pointer


6: jit_code_3610607616_1048576 + 0xe9dd2 [ l4d2_sb_ai_improver.smx::OnPlayerRunCmd ]
eip: 0xd7441dd2 esp: 0xffa97b90 ebp: 0xffa97b98

ffa97b90 ec cf 00 00 00 02 00 00 00 00 00 b8 7b a9 ff 6f 00 d0 db .........{..o...

Found via previous frame's frame pointer

On top of all this, there are frequent crashes with an error:

actions.ext.2.l4d2.so!ProcessHandlerImpl<ActionResult<CBaseEntity>, (lambda at /home/elysian/actions.ext/source/actions_processor_shared.h:185:12) &, CBaseEntity *&, float &> [invoke.h:73 + 0x1a]

Here is one of many stacktraces of such crashes:
https://crash.limetech.org/pk3iweadx34k

Spoiler

3: server_srv.so!Action<SurvivorBot>::InvokeUpda te(SurvivorBot*, Behavior<SurvivorBot>*, float) + 0xed
eip: 0xe77c664d esp: 0xff89d790 ebp: 0xff89d7e8 ebx: 0x125ea3b0
esi: 0xff89d838 edi: 0x12c8a3a0

ff89d790 b8 d7 89 ff a0 a3 c8 12 b0 a3 5e 12 00 d0 cc 3d ..........^....=
ff89d7a0 06 00 00 00 00 00 00 00 06 00 00 00 00 00 00 00 ................
ff89d7b0 90 a2 90 0d 00 00 00 00 08 00 00 00 90 a2 90 0d ................
ff89d7c0 00 00 00 00 00 00 00 00 c1 76 a8 e7 00 61 94 3f .........v...a.?
ff89d7d0 20 a8 ac e7 00 00 00 00 06 00 00 00 40 2b 43 10 ...........@+C.
ff89d7e0 30 c7 ea 0d b0 a3 5e 12 68 d8 89 ff 5a 56 7c e7 0.....^.h...ZV|.

Found via call frame info


4: server_srv.so!SurvivorIntention::Update() + 0x10a
eip: 0xe77c565a esp: 0xff89d7f0 ebp: 0xff89d868 ebx: 0x10432b40
esi: 0x0deac730 edi: 0x125ea3b0

ff89d7f0 38 d8 89 ff a0 a3 c8 12 b0 a3 5e 12 30 c7 ea 0d 8.........^.0...
ff89d800 00 d0 cc 3d 14 d8 89 ff 14 d8 89 ff e0 7d bc f1 ...=.........}..
ff89d810 06 00 00 00 f0 86 a8 e7 b4 72 ab e7 96 73 ba f1 .........r...s..
ff89d820 00 00 00 00 00 00 00 00 ce cc 63 43 00 00 00 00 ..........cC....
ff89d830 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
ff89d840 00 00 00 00 d8 e3 5e 12 9a 99 63 43 00 00 00 00 ......^...cC....
ff89d850 00 e4 5e 12 d8 e3 5e 12 00 00 00 00 40 2b 43 10 ..^...^.....@+C.
ff89d860 13 00 00 00 1c 2f 5f 12 98 d9 89 ff 04 c8 5c e7 ...../_.......\.

Found via call frame info

I didn't find any mention of your plugin in this stack-trace, but there are a lot of bot mentions here I don't know if these crashes are related to your plugin or to the actions extension and I should report them to the actions extension author?

Anyway, all these crashes happen very often on my 5 servers (screen attached). All servers use the same configuration:
Latest version of your plugin from GitHub (experimental branch). SourceMod (1.12.0.7157). Metamod:Source 1.11.0-dev+1155. Actions (3.8.8 ) by BHaType. "VScript" (1.8.5.manual) by 42. Linux (Ubuntu server).

In addition to your plugin, I use a plugin to limit the use of bots to the first aid kits: [L4D & L4D2] Bot Healing Values" (2.3) by SilverShot.

[Kerouha]

Originally Posted by Zunzunzun SurvivorHasPistol error

I will look into that for the next update.

Originally Posted by Mika Misori GetClosestToEyePosEntityBonePos crash

I've encountered crashes in similar circumstances, though since I use Windows (and don't host multiple servers), crashes are far between, and dumps don't display function names. Website refusing to load lately does not help either.
Might have a hacky solution for crashes though.

Originally Posted by Mika Misori actions.ext crash

This one is beyond my competence sadly. It appears that, while the extension allows you to manipulate actions, it does that in a raw manner; leaving the game's normal AI "nextbot" system with unexpected data sometimes.
It really needs to be more documented eg. what kind of actions exist, their structure, what values game will expect etc.

Originally Posted by Emana202 spaghetti-coded Frankenstein

I mean, it does the job as advertised. And some small (yet jarring) issues were too conveniently scattered around for someone not to jump in with a hammer

[Mystik Spiral]

@Kerouha
FWIW, I have not had a single error or server crash in a very long time. I also never get any "ran long" messages in the console. I have "ib_process_time" set to "0.1" to help prevent stale entity ids. The higher that value the more likely a stored entity id will no longer exist.

Regarding the SurvivorHasPistol problem... I suspect that can be resolved by adding a HasEntProp somewhere before GetEntProp since the error is that the property (for dual pistols) does not exist. If you are going to do another release, I would suggest also checking for any lines where an entity is killed and ensure that not only is the entity valid (probably already does this) but also make sure the entity is not 0 (world) since that is a valid entity, but attempting to remove it would crash the server.

[Mika Misori]

Originally Posted by Kerouha I will look into that for the next update.

Thank you, I really hope there is some way to fix these crashes. As soon as a new version of the plugin is released, I will install it immediately and continue reporting.

Originally Posted by Mystik Spiral I have not had a single error or server crash in a very long time.

Then I don't understand what's going on. Maybe some plugins are conflicting, but it would be unbearably hard to find out. Maybe I need to upgrade to the latest dev versions of SM + MM, I don't know what to do

[Mystik Spiral]

@Mika Misori
What do you have "ib_process_time" set to?
In the server console log, do you see any "ran long" Script Perf Warning messages?

You may not being doing anything "wrong", but your system might not have enough power to run everything you have loaded. As a test, you might want to try disabling all plugins except for this one and see if the problem goes away. If you have multiple L4D(2) servers on the same system, you may want to try to shutdown as many of them as possible. You might even see a difference if you switch the game server from a magnetic spinning disk to running on an NVME SSD. If you are running your server in a VM, see if allocating more CPU or RAM helps. If you are running a high tick rate and/or high min/max_updaterate, you could try lowering the values.

[Mika Misori]

Originally Posted by Mystik Spiral What do you have "ib_process_time" set to?

I use pretty much the default bot settings. “ib_process_time” = ‘0.2’ (default).

Originally Posted by Mystik Spiral do you see any "ran long" Script Perf Warning messages?

I didn't see any such warnings in the server console, but I didn't monitor the console that often.

Originally Posted by Mystik Spiral your system might not have enough power to run everything you have loaded.

Five L4D2 servers running on a dedicated machine with Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz, 8 gigs RAM, clean Ubuntu server minimized distribution. All server dependencies updated to the latest version. Peak load on the hardware, when all L4D2 servers are filled, does not rise above 60% (past L4D2 servers there are spinning my other projects, but they are isolated and do not interfere with each other). There are no performance problems, the server hardware monitoring has never shown any overload or overheating warnings. Fast network channel, no lags/spikes, players do not complain about anything.

Originally Posted by Mystik Spiral If you have multiple L4D(2) servers on the same system, you may want to try to shutdown as many of them as possible.

Each server has its own separate installation directory with files, the servers use only one shared folder with custom maps (symlink). I've already gone down from 10 running L4D2 servers to 5, and also made a different install directory for each (previously all server instances ran from the same install directory).

Originally Posted by Mystik Spiral switch the game server from a magnetic spinning disk to running on an NVME SSD.

Fast SAMSUNG 512 gigs SSD.

Originally Posted by Mystik Spiral If you are running your server in a VM, see if allocating more CPU or RAM helps.

I don't use virtualization for L4D2 servers, everything is on bare metal.

Originally Posted by Mystik Spiral If you are running a high tick rate and/or high min/max_updaterate, you could try lowering the values.

All L4D2 servers are running with the same 4 slot hardcore coop configuration, on 30 tickrate with default network settings (I didn't change any cvar like *_updaterate, etc).


The only 2 problems I am experiencing are:
1. The crashes I described above, which started after installing this plugin and switching from SM 1.11 to SM 1.12 with VScript.
2. And black screen with micro timeouts during the transition to the next map/restart of the round, which starts to happen after a long game on the server and disappears after a reboot. But this is a different story and I think not related to this plugin (probably : )

[Mystik Spiral]

@Mika Misori

Your system sounds fine and I do not see any issues. I do not know if it will help, but here is what I have installed (not including Workshop mods):

sm plugins list
[SM] Listing 31 plugins:
01 "Source Scramble Manager" (1.2.0) by nosoop
02 "[L4D2] Healing Alternative" (1.1a) by Mystik Spiral
03 "[L4D & L4D2] Left 4 DHooks Direct" (1.151) by SilverShot
04 "[L4D2] Survivor Bot AI Improver" (1.5k) by Emana202, Kerouha
05 "L4D2 Health Glows" (1.0.3.1) by Buster "Mr. Zero" Nielsen
06 "Basic Info Triggers" (1.12.0.7157) by AlliedModders LLC
07 "Player Commands" (1.12.0.7157) by AlliedModders LLC
08 "[L4D & L4D2] Spray Block" (1.0a) by Mystik Spiral
09 "Basic Chat" (1.12.0.7157) by AlliedModders LLC
10 "Basic Votes" (1.12.0.7157) by AlliedModders LLC
11 "[L4D & L4D2] Fire Glow" (1. by SilverShot
12 "Sound Commands" (1.12.0.7157) by AlliedModders LLC
13 "[L4D2] Spitter Acid Glow" (1.3) by SilverShot
14 "AFK Manager" (4.3.0) by Rothgar
15 "[L4D & L4D2] Detonation Control" (1.0.1) by Mystik Spiral
16 "[L4D & L4D2] Survivor Shove" (1.5) by SilverShot
17 "VScript" (1.8.6.manual) by 42
18 "[L4D & L4D2] Enforce Gametypes" (1.4) by Mystik Spiral
19 "[L4D2] Vomitjar Glow" (1.3) by SilverShot
20 "Map Change" (1.6) by Alex Dragokas, fdxx, sorallll, little_froy
21 "Basic Ban Commands" (1.12.0.7157) by AlliedModders LLC
22 "Anti-Flood" (1.12.0.7157) by AlliedModders LLC
23 "[L4D/L4D2] "go_away_from_keyboard" Unlock" (0.3) by cravenge
24 "HDR.smx"
25 "Admin File Reader" (1.12.0.7157) by AlliedModders LLC
26 "Basic Commands" (1.12.0.7157) by AlliedModders LLC
27 "Basic Comm Control" (1.12.0.7157) by AlliedModders LLC
28 "Admin Help" (1.12.0.7157) by AlliedModders LLC
29 "Admin Menu" (1.12.0.7157) by AlliedModders LLC
30 "[L4D & L4D2] Reverse Friendly-Fire" (2.9) by Mystic Spiral
31 "Client Preferences" (1.12.0.7157) by AlliedModders LLC

sm exts list
[SM] Displaying 12 extensions:
[01] Automatic Updater (1.12.0.7157): Updates SourceMod gamedata files
[02] Webternet (1.12.0.7157): Extension for interacting with URLs
[03] Source Scramble (0.7.1): Tools for working with memory.
[04] Actions (3.8.: Nextbot action tree manager
[05] SDK Hooks (1.12.0.7157): Source SDK Hooks
[06] BinTools (1.12.0.7157): Low-level C/C++ Calling API
[07] SDK Tools (1.12.0.7157): Source SDK Tools
[08] DHooks (1.12.0.7157): Dynamic Hooks
[09] Top Menus (1.12.0.7157): Creates sorted nested menus
[10] Regex (1.12.0.7157): Provides regex natives for plugins
[11] Client Preferences (1.12.0.7157): Saves client preference settings
[12] SQLite (1.12.0.7157): SQLite Driver

meta list
[01] SourceMod (1.12.0.7157) by AlliedModders LLC
[02] Actions (3.8. by BHaType
[03] SDK Hooks (1.12.0.7157) by AlliedModders LLC
[04] SDK Tools (1.12.0.7157) by AlliedModders LLC
[05] DHooks (1.12.0.7157) by AlliedModders LLC

meta version
Metamod:Source Version Information
Metamod:Source version 1.11.0-dev+1155
Plugin interface version: 16:14
SourceHook version: 5:5
Loaded As: Valve Server Plugin
Compiled on: May 15 2024 063:56
Built from: https://github.com/alliedmodders/met...commit/2009298
Build ID: 1155:2009298
http://www.metamodsource.net/

sm version
SourceMod Version Information:
SourceMod Version: 1.12.0.7157
SourcePawn Engine: 1.12.0.7157, jit-x86 (build 1.12.0.7157)
SourcePawn API: v1 = 5, v2 = 16
Compiled on: Aug 31 2024 04:43:41
Built from: https://github.com/alliedmodders/sou...ommit/4e6d66b7
Build ID: 7157:4e6d66b7
http://www.sourcemod.net/

Distro: Ubuntu 22.04.5 LTS
Arch: x86-64
Kernel: 6.8.0-48-generic

I had to install VSLib (from Workshop) a while back and I am including it since I do not know what was requiring it and the server was crashing without it.

VSLib by Rayman1103
https://steamcommunity.com/sharedfil...?id=2402581049

I seriously doubt it makes any difference but I am running L4D2 inside VirtualBox 7.0.20 (will upgrade to 7.0.22 fairly soon).

My only suggestion is to try setting "ib_process_time" to "0.1" and see if it makes any difference.

[Mika Misori]

Originally Posted by Mystik Spiral My only suggestion is to try setting "ib_process_time" to "0.1" and see if it makes any difference.

Thanks for your attention to my problem and for sharing your config. I followed your advice and changed ib_process_time from 0.2 to 0.1. Also updated all possible dependencies to the latest version: sourcemod-1.12.0-git7167-linux, VScript-1.9.1.85, SMExt-SourceScramble 0.8.0-pre2.

Hope it helps, I will monitor and report further.