[FlaminSarge]

Hi guys, I've put up a preliminary version of TF2IDB with attribute support (and nosoop's class/slot changes), as well as a plugin that pretends that it's TF2ItemsInfo and makes calls to TF2IDB, essentially completely replacing TF2ItemsInfo.
http://github.com/flaminsarge/tf2idb

This is NOT READY for formal release. I need testers. Testers that know what they're doing. As such, if you don't know how to compile either plugin properly or install either on the server, including updating the SQLite database using the repository's Python script, then this plugin isn't for you yet.

If you plan to test them, put the new tf2idb.inc in your includes and compile both .sp files; put both up on the server and delete tf2itemsinfo. Run the included Python script (change the file paths at the top to point to the right items_game.txt), and place the resulting SQLite DB (sq3 file) into sourcemod/data/sqlite. Reboot the server and see if plugins that use TF2ItemsInfo now run without it. Ignore the repository's 'test' folder unless you want to test attribute KeyValues generation for some reason.

Let me know how it goes, thanks, and feel free to test out other things in tf2idb such as the new attribute functions or TF2IDB_CustomQuery. All the new stuff can be found in the include file.

[nosoop]

Excellent. I was hoping I could stop using my dirty hack of a build.

Early morning findings. I have no idea which versions of what plugins people are using these days, so I may be outdated on certain things.

General:

• On server startup, compatibility shim's GetMaxItemID() call may attempt to query TF2IDB before the database is available. Plugin needs a delayed load. Might've already been an existing issue in TF2IDB; personally never had to query anything in OnPluginStart() so I would've never seen it either.
  

  [SM] Exception reported: Invalid database Handle 0 (error: 4)
  [SM] Blaming: tf2idb.smx()
  [SM] Call stack trace:
  [SM]   [0] SQL_Query
  [SM]   [1] Line 504, tf2idb.sp::Native_FindItemCustom()
  [SM]   [3] TF2IDB_FindItemCustom
  [SM]   [4] Line 2032, tf2idb_tf2ii_compat.sp::GetMaxItemID()
  [SM]   [5] Line 1031, tf2idb_tf2ii_compat.sp::ReloadConfigs()
  [SM]   [6] Line 247, tf2idb_tf2ii_compat.sp::OnPluginStart()

• The compatibility shim returns the item's slot and does not account for player class (e.g., multi-class shotguns will return TF2ItemSlot_Primary even if Pyro is passed in). I think TF2ItemsInfo has the same quirk, so whether or not this is proper behavior is debatable.

Tested plugins:

• Bot Weapon Randomizer: Mostly works. Of course, significant performance hit when loadouts are regenerated (server hangs for seconds) because of all the queries it ends up having to run.
• Give Weapon: Works fine. There is a quirk when it comes to equipping one of the new shotguns (removes the primary weapon and equips it as secondary), but that's probably a known issue with TF2II as well.

Known plugins that need testing (bring your friends?):

• GunGame, TF2Ware v2

Generally, if it depends on calling TF2II functions a large number of times in one go, you're going to have a pretty bad time.

[bottiger]

Originally Posted by FlaminSarge Hi guys, I've put up a preliminary version of TF2IDB with attribute support (and nosoop's class/slot changes), as well as a plugin that pretends that it's TF2ItemsInfo and makes calls to TF2IDB, essentially completely replacing TF2ItemsInfo. http://github.com/flaminsarge/tf2idb This is NOT READY for formal release. I need testers. Testers that know what they're doing. As such, if you don't know how to compile either plugin properly or install either on the server, including updating the SQLite database using the repository's Python script, then this plugin isn't for you yet. If you plan to test them, put the new tf2idb.inc in your includes and compile both .sp files; put both up on the server and delete tf2itemsinfo. Run the included Python script (change the file paths at the top to point to the right items_game.txt), and place the resulting SQLite DB (sq3 file) into sourcemod/data/sqlite. Reboot the server and see if plugins that use TF2ItemsInfo now run without it. Ignore the repository's 'test' folder unless you want to test attribute KeyValues generation for some reason. Let me know how it goes, thanks, and feel free to test out other things in tf2idb such as the new attribute functions or TF2IDB_CustomQuery. All the new stuff can be found in the include file.

Very nice, I will put a link to it in the first post. Hopefully we can get some people to test it out and make sure there are no issues.

Everything is backwards compatible right? To be honest I am a bit scared and lazy to try out new code when everything still works for me (other than region conflicts possibly, which I don't really understand).

I don't remember which part of the plugin I wrote, and which ones you added anymore. But from a quick glance everything looks okay.

However I did notice that you did not close the handle here https://github.com/FlaminSarge/tf2id...tf2idb.sp#L235

And also I am not sure I like that you are making the quality mappings in the plugin instead of the database. Is it not possible to do it once in the python file and save it to the database?

Originally Posted by nosoop [*]On server startup, compatibility shim's GetMaxItemID() call may attempt to query TF2IDB before the database is available. Plugin needs a delayed load. Might've already been an existing issue in TF2IDB; personally never had to query anything in OnPluginStart() so I would've never seen it either. Generally, if it depends on calling TF2II functions a large number of times in one go, you're going to have a pretty bad time.

You should never be calling another plugin inside OnPluginLoad. You should be doing it in OnAllPluginsLoaded.

Calling tf2ii functions a large number of times is as fast as possible for cached functions. You can try to figure out how to optimize it by using the sourcemod profiler.

[nosoop]

Originally Posted by bottiger Everything is backwards compatible right? To be honest I am a bit scared and lazy to try out new code when everything still works for me (other than region conflicts possibly, which I don't really understand). You should never be calling another plugin inside OnPluginLoad. You should be doing it in OnAllPluginsLoaded. Calling tf2ii functions a large number of times is as fast as possible for cached functions. You can try to figure out how to optimize it by using the sourcemod profiler.

TF2 Item DB functionality is backwards-compatible other than the patches for per-class item slots (existing behavior should remain unaffected) and other added features. Could always write test cases to be safe...

Calling TF2IDB in OnPluginLoad is known (and has been fixed as of the latest commit, which was made after I posted).
It's just kind of awkward because TF2II was never exactly built with the schema being unavailable at any point (e.g., unloaded at runtime), so the compatibility shim has to deal with that (what if a different plugin calls OnAllPluginsLoaded before TF2II-compat does?).

Also, Bot Weapon Randomizer's just poorly optimized. While not optimal, other plugins should work acceptably.

[bottiger]

Originally Posted by nosoop TF2 Item DB functionality is backwards-compatible other than the patches for per-class item slots (existing behavior should remain unaffected) and other added features. Could always write test cases to be safe... Calling TF2IDB in OnPluginLoad is known (and has been fixed as of the latest commit, which was made after I posted). It's just kind of awkward because TF2II was never exactly built with the schema being unavailable at any point (e.g., unloaded at runtime), so the compatibility shim has to deal with that (what if a different plugin calls OnAllPluginsLoaded before TF2II-compat does?). Also, Bot Weapon Randomizer's just poorly optimized. While not optimal, other plugins should work acceptably.

Can you tell me which part of the per-class item slots is not backwards compatible?

https://github.com/FlaminSarge/tf2id...tf2idb.inc#L55

It looks like here the class is an optional parameter so it should work without having to recompile the plugins right? Actually I'm not sure, maybe the optional parameter is compiled into each plugin?

But it looks like the optional parameter is being checked in the native so it should be fine. Are there any other backwards incompatible issues?

https://github.com/FlaminSarge/tf2id...tf2idb.sp#L371

There are 3 things I need to do before making this the new version of tf2idb.

• Check for backwards incompatibility. Per class items slots looks safe to me.
• Review new code. Unfortunately I forgot which parts I did not write.
• Change the quality mappings so it is done in python file instead of the plugin. It doesn't seem right to start doing these calculations in the plugin when everything else is done in the database.

I see your problem you are trying to get it to load before every other plugin. Maybe sourcemod loads plugins by alphabetical order? If not, I would just give up on it and tell people to modify their plugin to use TF2IDB. It shouldn't be a difficult change and all the important plugins have been migrated.

[Pitbull3]

This doesn't seem to be making changes or updating items_game.txt

Does it not make changes if none are needed to be made?

[nosoop]

Originally Posted by bottiger Can you tell me which part of the per-class item slots is not backwards compatible? There are 3 things I need to do before making this the new version of tf2idb. [...] Review new code. Unfortunately I forgot which parts I did not write. [...] [...] I would just give up on it and tell people to modify their plugin to use TF2IDB.

The behavior has changed, so technically it is a breaking change, but it is a bugfix. Plugins that depend on the old behavior (none, hopefully) will break, of course. Existing stuff remains untouched as far as I can remember.

Feel free to review the changes made in the commits here, starting from "Initial commit", based on your 2016-03-01 release. I'd recommend getting set up with Github, if only for being able to keep track of who made what changes (and what changes have been made) and throwing things into the issue tracker.

Maybe some people just can't be bothered to port things over to TF2IDB, or aren't able to. Can't be helped.

Originally Posted by Pitbull3 This doesn't seem to be making changes or updating items_game.txt

Not sure what you're asking.

TF2 Item DB's Python script processes items_game.txt file and stores the contents into an SQLite database, which the plugin queries. This is an alternative / replacement for TF2 Items Info, which parses items_game.txt into memory and looks up values there. items_game.txt is not written to, only read from.

[bottiger]

Originally Posted by nosoop The behavior has changed, so technically it is a breaking change, but it is a bugfix. Plugins that depend on the old behavior (none, hopefully) will break, of course. Existing stuff remains untouched as far as I can remember. Feel free to review the changes made in the commits here, starting from "Initial commit", based on your 2016-03-01 release. I'd recommend getting set up with Github, if only for being able to keep track of who made what changes (and what changes have been made) and throwing things into the issue tracker. Maybe some people just can't be bothered to port things over to TF2IDB, or aren't able to. Can't be helped.

But as far as I can tell the behavior hasn't changed for plugins that do not specify a class. Am I wrong here?

Unfortunately either you or flamingsarge changed the spacing so looking at a diff is completely useless.

https://github.com/FlaminSarge/tf2id...993c41cd413127

I am rather short on time these days so reviewing every single line is something that I really don't feel like doing.

Right now the 1 thing I really want to make is an automatic gamedata updater, so trying to vet these changes to tf2idb is rather low on my TODO list, especially when it is working correctly for me with no crashes or bugs and I do not need the new features. But I am still interested in trying to get these new features released, but only if I can be reasonably sure they won't crash or something similarly bad.

Update

I've spent about as much extra time on looking through the code as I can stand.

From just looking at the code there are no serious problems and it should run ok. But some changes need to be made to make maintaining the plugin easier.

- You need to cache calls to GetItemSlotName with a class. Some plugins call this hundreds of times a second such as bot weapon randomizer. You can do this by using the same trie cache and doing Format("%i-%i", id, class) as the trie key.
- There's too much new attribute code for me to review but from what I can tell, it won't break any old plugins querying attributes? Someone else needs to check them.
- This handle needs to be closed. https://github.com/FlaminSarge/tf2id...tf2idb.sp#L235
- The quality mappings need to be pulled out of the sourcepawn and into the the python file. It looks like I (or it might have been someone else) hardcoded it into the sourcepawn, but that's only because I didn't realize that items_game.txt has them. All calculations need to be moved into the python file if possible.

"qualities"
{
"normal"
{
"value" "0"
}
"unique"
{
"value" "6"
}
"rarity1"
{
"value" "1"
}
"rarity2"
{
"value" "2"
}

These things won't prevent the plugin from running correctly, but they should be fixed.

[Pitbull3]

Not sure what you're asking. TF2 Item DB's Python script processes items_game.txt file and stores the contents into an SQLite database, which the plugin queries. This is an alternative / replacement for TF2 Items Info, which parses items_game.txt into memory and looks up values there. items_game.txt is not written to, only read from.

I meant the file items_game.txt doesn't seem to have been modified since: 9/28/2016

[xXDeathreusXx]

Originally Posted by Pitbull3 I meant the file items_game.txt doesn't seem to have been modified since: 9/28/2016

Because Valve hasn't added any new items or changed any, yet