[Byte]

Context:
On 18st Jan added 1 new model to server.
On 23rd Jan added 10+ new modelss to server.
Today (27 Jan) server is crashing on startup (crash loop). I've spend 7+ hours trying to fix it since.

Issue:
My server is crashing on restart (during the time period SM plugins are loading).
To clarify, it crashes before any players join (although GOTV client will be in) as each plugin loads.
I noticed this only today after the server crashed (for another known reason).
It then started to crash loop.
There are no accelerator dumps or any other error/issue in logs anywhere.
I have 90 plugins on the server.
Starting the server with 1 of them disabled fixes the issue.

This 1 plugin easydownloader (which I wrote myself):
https://forums.alliedmods.net/showthread.php?p=2482408

Okay so looks like some sort of precache/downloads issue?
Server could have been up since the 23rd Jan (without restarting) so its likely related to changes made then.
I find out if I restore the old downloads config, the server starts and works. (the one prior to 23 Jan changes).
I find out if I remove the models/download only lines in the downloads config bit by bit it works. (this is decreasing the entries in stringtables downloadsonly and modelprecache)
It doesn't seem to be an issue with any particular model/material/file. Just the total number of files in the stringtables.

So either the plugin is broken or I'm reaching string table limits right?
I tried another download plugins and it resulted in the same behaviour.

Different maps result in a borderline working config to work (because each map has different number of models etc). So one config that didn't work on a custom map worked on de_dust2.

If I start the server without easydownloader.smx and then late load the plugin. I can map change, players can leave join etc until the next server restart/crash in which case it goes back to crash looping. So lateloading the plugin works but if it was a strings table limit issue, shouldn't lateloading also cause a crash?

There have been no recent CSGO updates that seem relevant.
I run GOTV 24/7 and have for years. Could recent GOTV updates be related?
I wasn't affected by the precaching in the wrong spot issue that occurred a while ago because I ran GOTV. However, I did go back and check most plugins to ensure they precached in OnMapStart. I will double check this shortly.

When looking at dumpstringtables with a config that is working (but about to crash).
In other words if I add say 10 files to the downloadstable/modelprecache I crash:

Table downloadables (changed on ticks 67, 65)
1555/8192 items

Table modelprecache (changed on ticks 65, 0)
1164/4096 items

Table genericprecache (changed on ticks 65, 0)
2/512 items

Table soundprecache (changed on ticks 67, 65)
2209/8192 items

Table decalprecache (changed on ticks 65, 0)
118/512 items

Table instancebaseline (changed on ticks 745, 714)
34/1024 items

Table lightstyles (changed on ticks 65, 0)
64/64 items

Table userinfo (changed on ticks 65, 0)
255/256 items

Table dynamicmodel (changed on ticks 0, 0)
0/4096 items

Table server_query_info (changed on ticks 65, 0)
1/4 items

Table ExtraParticleFilesTable (changed on ticks 65, 0)
1/1024 items

Table ParticleEffectNames (changed on ticks 65, 0)
580/1024 items

Table EffectDispatch (changed on ticks 65, 0)
41/1024 items

Table VguiScreen (changed on ticks 65, 0)
2/256 items

Table Materials (changed on ticks 65, 0)
2/1024 items

Table InfoPanel (changed on ticks 65, 0)
1/128 items

Table Scenes (changed on ticks 0, 0)
0/8192 items

Table Movies (changed on ticks 0, 0)
0/16 items

Table GameRulesCreation (changed on ticks 65, 0)
1/1 items

Things I've tried:
I've spend so much time trying so many things:

• Reuploaded custom content (models/materials)
• Recompiled easydownloaderplugin
• Disable a whole lot of plugins (doesn't seem to matter, only if easydownloader is present the crash occurs)
• Verify steam installation
• Right now my server is up and running with players using the lateload thing I mentioned. However, I need to actually fix the issue so restarts work.

Any help/ideas tips are massively appreciated as my brain is hurting.

Server Info:

SourceMod Version Information:
 SourceMod Version: 1.9.0.6223
 SourcePawn Engine: 1.9.0.6223, jit-x86 (build 1.9.0.6223)
 SourcePawn API: v1 = 4, v2 = 12
 Compiled on: Dec 19 2017 21:42:42
 Built from: https://github.com/alliedmodders/sourcemod/commit/309e6ae
 Build ID: 6223:309e6ae
 http://www.sourcemod.net/

Metamod:Source version 1.10.7-dev
Built from: https://github.com/alliedmodders/metamod-source/commit/89c6fb3
Build ID: 956:89c6fb3
Loaded As: Valve Server Plugin
Compiled on: Apr 26 2017
Plugin interface version: 15:14
SourceHook version: 5:5
http://www.metamodsource.net/

[Byte]

Alright did more testing.

I found out the server also works if instead of easydownloader I remove this plugin which I also run as part of my 24/7 GOTV demo system:
https://forums.alliedmods.net/showthread.php?p=2070519

So in this case I'm running a server with SM base plugins + easydownloader + csgo_demo_crash_fix.
It crashes.

If I run just SM base plugins, its fine/no crashes.
If I run SM base plugins + easydownloader, its fine/no crashes.
If I run SM base plugins + csgo_demo_crash_fix, its fine/no crashes.
If I run SM base plugins + easydownloader + csgo_demo_crash_fix (crash after GOTV client in server for 36-37 seconds).



Instead of the csgo_demo_crash_fix plugin I could also just put tv_record in console.
When I do this the demo starts recording but there is no instant crash.
The crash happens ~35 seconds later (only varying by a small amount each trial (~1 second max?)).

Recent CSGO updates mentioned GOTV:
http://blog.counter-strike.net/index.php/2018/01/19916/
http://blog.counter-strike.net/index.php/2018/01/19931/

Pretty likely to be related.

Very tired now but will continue looking into it later.

[Byte]

Okay more info.
I setup a local stock CSGO server.
I added the same SM/Metamod versions in the first post.

I then added the 2 plugins (easydownloader + csgo_demo_crash_fix) and the issue occurs.
Again, if I remove either plugin, the issue doesn't occur.
If I do the precaching after `tv_record` is run for the server bootup, it works fine and I can then precache OnMapStart for all future map changes. Map changes also work fine with no issues indicating to me this isn't a precache limit/precache issue.

This is the exact crash message:

Host_Error: Overflow error writing string table baselines Scenes

I discovered something interesting!
As the crash happens X amounts of seconds after OnMapStart on server startup and it happens more or less at the same exact time each time, I tried to dump string tables right before the crash (1 second before hand).
This is what I noticed:

Table soundprecache (changed on ticks 65, 0)
  7422/8192 items

Table Scenes (changed on ticks 65, 0)
  5252/8192 items

Seems to me like starting GOTV demo recording is adding a huge number of scenes/sounds/other junk momentarily. Then it jumps back down to 2000 sounds/0 scenes.

This explains why: starting tv_record -> then lateloading precache plugin isn't causing overflow but doing it the other way might. It also explains why lessening the total number of entries in the string table fixes it.

[Byte]

Little bump.

Using the "workaround" but that leads to a whole lot of other crashes daily.

Here are the dumps for those (spans ~1 day):
https://crash.limetech.org/mx33muuchhe7
https://crash.limetech.org/jh5lea6djqt6
https://crash.limetech.org/josawxxvaqj7
https://crash.limetech.org/v7vlg3hfbsvc
https://crash.limetech.org/dsku5xxow6kr

Can anyone help me permanently fix this?
Would appreciate it a lot!

[Masterofks]

Bump, same issue.

[Byte]

Bump willing to reward anyone who solves this with $$.
I'm not sure what to try etc, need someone smart to look into this

[Masterofks]

Bump again

[Byte]

Little bump, still experiencing this!

[StealthGus]

What's the actual signature of your crash? Because I think I'm having the same issue. Your accelerator logs have expired.