[bottiger]

Ok thanks next time just upload the file and I will add it.

[fakuivan]

ENHANCED VERSION

READ THIS bottiger

I've made some changes (again :V) to the python DB builder, you can include these to the release if you want:

Extra features:

#	Features:		1. File locations are loaded trough an external file, eliminating the need of modifying the python code
#				2. Provides support for Environment Variables such as %something% to enhance portability
#				3. Outputs the progress to the console to provide feedback to the user
#				4. Syntax compatible to Python 3
#					a. .iteritems method replaced with .items (obsolete in 3.x)
#					b. "%i" %var syntax replaced with "{}".format(var) (preferred in 3.x)
#				5. Overall cleaner code to make troubleshooting easier

The config file (included in the zip):

"config"
{
"Paths"
{
	"DBPath"	"addons\sourcemod\data\sqlite\tf2idb.sq3"		//The path to the DB file, related to TFPath/DBPath
	"TFPath"	"tf2_dedicated_server\tf"				//If UsesEnvVar is set to 1 then use the path relative to EnvVar/TFPath, else use root
	"UsesEnvVar"	"1"							//Set to 1 if you want to use environment variables, 0 if not
	"EnvVar"	"TF2SERVER"						//Write the Environment variable without the %
}
}

The code:

Spoiler

#	Description: 	This is an enhanced version of the DB builder used by the TF2ItemsDB plugin created by bottiger
#	Author: 		bottiger (base builder owner), fakuivan (enhanced version)
#	Version: 		1.2
#	Features:	1. File locations are loaded trough an external file, eliminating the need of modifying the python code
#				2. Provides support for Environment Variables such as %something% to enhance portability
#				3. Outputs the progress to the console to provide feedback to the user
#				4. Syntax compatible to Python 3
#					a. .iteritems method replaced with .items (obsolete in 3.x)
#					b. "%i" %var syntax replaced with "{}".format(var) (preferred in 3.x)
#				5. Overall cleaner code to make troubleshooting easier 


import os
import vdf
import sqlite3
import traceback
import time

#

print("Loading configs...")
parsedconfig = dict()
cfh = open("{}\\tfidb.configs.txt".format(os.path.dirname(os.path.realpath(__file__))), "r")
parsedconfig = vdf.parse(cfh)

if parsedconfig["config"]["Paths"]["UsesEnvVar"] == "1":
	TF_ENV_VAR = (os.environ[parsedconfig["config"]["Paths"]["EnvVar"]]).replace("\\", "/")
     
if parsedconfig["config"]["Paths"]["UsesEnvVar"] == "0":
	TF_FOLDER = parsedconfig["config"]["Paths"]["DBPath"].replace("\\", "/")
else:
    TF_FOLDER = TF_ENV_VAR + (r'\{}'.format(parsedconfig["config"]["Paths"]["TFPath"])).replace("\\", "/")
    
ITEMS_GAME = TF_FOLDER + r'\scripts\items\items_game.txt'.replace("\\", "/")
DB_FILE = TF_FOLDER + (r'\{}'.format(parsedconfig["config"]["Paths"]["DBPath"])).replace("\\", "/")

#

def resolve_prefabs(item):
	prefabs = item.get('prefab')
	if prefabs:
		prefab_aggregate = {}
		for prefab in prefabs.split():
			prefab_data = data['prefabs'][prefab]
			prefab_data = resolve_prefabs(prefab_data.copy())
			prefab_aggregate.update(prefab_data)
		prefab_aggregate.update(item)
		item = prefab_aggregate
	return item

#

start = time.time()

print("Parsing items_game.txt...")
with open(ITEMS_GAME) as f:
	data = vdf.parse(f)
	data = data['items_game']

#

parse_end = time.time()

print("Finished parsing file, {}s".format(parse_end - start))

print("Connecting to the DB...")
db = sqlite3.connect(DB_FILE)
dbc = db.cursor()

print("Formatting tables...")
dbc.execute('DROP TABLE IF EXISTS new_tf2idb_class')
dbc.execute('DROP TABLE IF EXISTS new_tf2idb_item_attributes')
dbc.execute('DROP TABLE IF EXISTS new_tf2idb_item')
dbc.execute('DROP TABLE IF EXISTS new_tf2idb_particles')
dbc.execute('DROP TABLE IF EXISTS new_tf2idb_equip_conflicts')
dbc.execute('DROP TABLE IF EXISTS new_tf2idb_equip_regions')
dbc.execute('DROP TABLE IF EXISTS new_tf2idb_capabilities')

dbc.execute('CREATE TABLE "new_tf2idb_class" ("id" INTEGER NOT NULL , "class" TEXT NOT NULL , PRIMARY KEY ("id", "class"))')
dbc.execute('CREATE TABLE "new_tf2idb_item_attributes" ("id" INTEGER NOT NULL , "attribute" INTEGER NOT NULL , "value" TEXT NOT NULL, PRIMARY KEY ("id", "attribute") )')
dbc.execute('CREATE TABLE "new_tf2idb_item" ('
	'"id" INTEGER PRIMARY KEY  NOT NULL ,'
	'"name" TEXT NOT NULL,'
	'"item_name" TEXT NOT NULL,'
	'"class" TEXT NOT NULL,'
	'"slot" TEXT,'
	'"quality" TEXT NOT NULL,'
	'"tool_type" TEXT,'
	'"min_ilevel" INTEGER,'
	'"max_ilevel" INTEGER,'
	'"baseitem" INTEGER,'
	'"holiday_restriction" TEXT,'
	'"has_string_attribute" INTEGER'
	')'
)
dbc.execute('CREATE TABLE "new_tf2idb_particles" ("id" INTEGER PRIMARY KEY  NOT NULL , "name" TEXT NOT NULL )')
dbc.execute('CREATE TABLE "new_tf2idb_equip_conflicts" ("name" TEXT NOT NULL , "region" TEXT NOT NULL , PRIMARY KEY ("name", "region"))')
dbc.execute('CREATE TABLE "new_tf2idb_equip_regions" ("id" INTEGER NOT NULL , "region" TEXT NOT NULL , PRIMARY KEY ("id", "region"))')
dbc.execute('CREATE TABLE "new_tf2idb_capabilities"  ("id" INTEGER NOT NULL , "capability" TEXT NOT NULL )')

nonce = int(time.time())
dbc.execute('CREATE INDEX "tf2idb_item_attributes_{}" ON "new_tf2idb_item_attributes" ("attribute" ASC)'.format(nonce))
dbc.execute('CREATE INDEX "tf2idb_class_{}" ON "new_tf2idb_class" ("class" ASC)'.format(nonce))
dbc.execute('CREATE INDEX "tf2idb_item_{}" ON "new_tf2idb_item" ("slot" ASC)'.format(nonce))

# particles

print("Adding particles to table...")
for particle_type,particle_list in data['attribute_controlled_attached_particles'].items():
	for k,v in particle_list.items():
		dbc.execute('INSERT INTO new_tf2idb_particles (id,name) VALUES (?,?)', (k, v['system']) )

# attributes

print("Adding attributes to table...")
attribute_type = {}
for k,v in data['attributes'].items():
	at = v.get('attribute_type')
	if at:
		atype = at
	else:
		if v.get('stored_as_integer'):
			atype = 'integer'
		else:
			atype = 'float'
	attribute_type[v['name']] = (k, atype)

# conflicts

print("Adding conflicts to table...")
for k,v in data['equip_conflicts'].items():
	for region in v.keys():
		dbc.execute('INSERT INTO new_tf2idb_equip_conflicts (name,region) VALUES (?,?)', (k, region))

# items

print("Adding items and capabilities to table...")
for id,v in data['items'].items():
	if id == 'default':
		continue
	i = resolve_prefabs(v)
	baseitem = 'baseitem' in i

	try:
		tool = None
		if 'tool' in i:
			tool = i['tool'].get('type')

		has_string_attribute = False
		if 'attributes' in i:
			for name,info in i['attributes'].items():
				aid,atype = attribute_type[name]
				if atype == 'string':
					has_string_attribute = True
				dbc.execute('INSERT INTO new_tf2idb_item_attributes (id,attribute,value) VALUES (?,?,?)', (id,aid,info['value']))

		dbc.execute('INSERT INTO new_tf2idb_item '
			'(id,name,item_name,class,slot,quality,tool_type,min_ilevel,max_ilevel,baseitem,holiday_restriction,has_string_attribute) '
			'VALUES (?,?,?,?,?,?,?,?,?,?,?,?)', 
			(id,i['name'],i.get('item_name'),i['item_class'],i.get('item_slot'),i.get('item_quality', ''), tool, i.get('min_ilevel'), i.get('max_ilevel'),baseitem,
				i.get('holiday_restriction'), has_string_attribute)
		)

		if 'used_by_classes' in i:
			for prof in i['used_by_classes'].keys():
				dbc.execute('INSERT INTO new_tf2idb_class (id,class) VALUES (?,?)', (id, prof))

		region_field = i.get('equip_region') or i.get('equip_regions')
		if region_field:
			if type(region_field) is str:
				region_field = {region_field: 1}
			for region in region_field.keys():
				dbc.execute('INSERT INTO new_tf2idb_equip_regions (id,region) VALUES (?,?)', (id, region))

		# capabilties
		
		for capabilty in i.get('capabilities', []):
			dbc.execute('INSERT INTO new_tf2idb_capabilities (id,capability) VALUES (?,?)', (id, name))

	except:
		traceback.print_exc()
		print(id)
		raise

#

def replace_table(name):
	dbc.execute('DROP TABLE IF EXISTS %s' % name)
	dbc.execute('ALTER TABLE new_%s RENAME TO %s' % (name,name))
	
#

print("Cleaning DB...")
replace_table('tf2idb_class')
replace_table('tf2idb_item_attributes')
replace_table('tf2idb_item')
replace_table('tf2idb_particles')
replace_table('tf2idb_equip_conflicts')
replace_table('tf2idb_equip_regions')
replace_table('tf2idb_capabilities')
dbc.execute('VACUUM')
db.commit()

whole_finished = int(time.time())

print("Finished, {}s".format(whole_finished - start))

[To final user]if you don't know what an environment variable is, just set it to 0 and put the whole directory in "TFPath"

[Reginald Halifrax]

Has anyone made a port of the currently uploaded Uber Upgrades mod for this? its something i've been hunting for.

[ZAGOR]

[SM] Plugin unusual.smx failed to load: Native "TF2IDB_IsItemUsedByClass" was not found

[fakuivan]

Originally Posted by ZAGOR [SM] Plugin unusual.smx failed to load: Native "TF2IDB_IsItemUsedByClass" was not found

Upload your scripts (tf2idb.sp and unusual.sp)

[fakuivan]

17/12/2015 Update FIX

Fixed an error that breaks this builder, it has to do with Valve introducing case typos in "The Beggar's Bazooka" in the "blast radius decreased" attribute that should be writen as "Blast radius decreased" . The fix can be seen in the 135th line

This fix is only for Python 3.5, install it on your desktop if you have the 2 version on your server and then upload the database to your server. Keep in mind that this is supposed to be an error that valve HAS TO FIX, if this is not the case I'll try to make a fix for python 2

[bottiger]

I have fixed the issue by storing all attribute names as lower case. Please download tf2idb.zip.

Fakuivan, I am sorry I cannot accept your environmental variables patch. It is nice to have more options but it is not backwards compatible and I don't want to use environmental variables myself.

Please put the code to check for environmental variables after the settings so people can choose which one to use.

Originally Posted by ZAGOR [SM] Plugin unusual.smx failed to load: Native "TF2IDB_IsItemUsedByClass" was not found

The official tf2idb does not have the native TF2IDB_IsItemUsedByClass. Please refrain from using unofficial versions of tf2idb. If you want this feature added, then please post the code here for people to review.

[ZAGOR]

Originally Posted by bottiger I have fixed the issue by storing all attribute names as lower case. Please download tf2idb.zip. Fakuivan, I am sorry I cannot accept your environmental variables patch. It is nice to have more options but it is not backwards compatible and I don't want to use environmental variables myself. Please put the code to check for environmental variables after the settings so people can choose which one to use. The official tf2idb does not have the native TF2IDB_IsItemUsedByClass. Please refrain from using unofficial versions of tf2idb. If you want this feature added, then please post the code here for people to review.

Its okey but now i get this errors with tf2idb ,

L 12/19/2015 - 12:26:28: [unusual.smx] [UNUSUAL] unusual_list.cfg can't read line : 2 
L 12/19/2015 - 12:26:28: [unusual.smx] [UNUSUAL] unusual_list.cfg can't read line : 4 
L 12/19/2015 - 12:26:28: [SM] Plugin encountered error 15: Array index is out of bounds
L 12/19/2015 - 12:26:28: [SM] Displaying call stack trace for plugin "unusual.smx":
L 12/19/2015 - 12:26:28: [SM]   [0]  Line 903, unusual.sp::PanelEffect()
L 12/19/2015 - 12:26:28: [SM]   [1]  Line 854, unusual.sp::QltymenuAnswer()

[bottiger]

That isn't a problem with tf2idb, that is a problem with unusual.sp.

[SowlHash]

L 02/02/2016 - 00:31:05: [SM] Unable to load extension "socket.ext": dlopen(/Users/SowlHash/Library/Application Support/Steam/SteamApps/common/Team Fortress 2/tf/addons/sourcemod/extensions/socket.ext.dylib, 2): image not found
L 02/02/2016 - 00:31:05: [SM] Unable to load plugin "itemsapi.smx": Required extension "Socket" file("socket.ext") not running
L 02/02/2016 - 00:31:05: [SM] Unable to load plugin "tf2ibwrl.smx": Required extension "TF2Items" file("tf2items.ext.2.ep2v") not running

Im running a server on OS X, and I cannot seem to get TF2ItemBotWeaponRandomizer to work with Death's TF2IDB or the original TF2IDB, can anyone help me?

Sorry if theres a format error in this post, or if this is a stupid question, I am relatively new to all this.
Thanks!