Raised This Month: $51 Target: $400
 12% 

[Tutorial] Data Structures With Indexes


Post New Thread Reply   
 
Thread Tools Display Modes
Author Message
rhelgeby
Veteran Member
Join Date: Oct 2008
Location: 0x4E6F72776179
Old 05-20-2011 , 22:25   [Tutorial] Data Structures With Indexes
Reply With Quote #1

Data Structures With Indexes

Update: This tutorial is about static data structures. For dynamic type safe storage in SourcePawn see my Object Library.

Basics

We often need to store data in a structured manner. A way to get a simple data structure is using arrays and enums. That way we can sort of get small value objects (structs). Have a look at the following code:

PHP Code:
// Data structure definition. Elements can be tagged. Arrays and strings
// can also be used.
enum MyDataStruct 
{
    
Data_SomeVar
    
bool:Data_SomeBool
    
Float:Data_SomeFloat,
    
String:Data_SomeString[32]
}

// Maximum number of objects to store. Not all indexes has to be used.
const MAX_OBJECTS 20;

// Data. Note the use of MyDataStruct in the last dimension.
new MyObjects[MAx_OBJECTS][MyDataStruct];

// Number of objects.
new ObjectCount 0;

TestCase()
{
    
// Creating a test object.
    
MyObjects[ObjectCount][Data_SomeVar] = 5;
    
MyObjects[ObjectCount][Data_SomeBool] = true;
    
MyObjects[ObjectCount][Data_SomeFloat] = 3.14159265;
    
strcopy(MyObjects[ObjectCount][Data_SomeString], 32"Test string.");
    
    
// After each insertion the counter should be updated.
    
ObjectCount++;
    
    new var = 
MyObjects[0][Data_SomeVar];
    new 
bool:boolVar MyObjects[0][Data_SomeBool];
    new 
Float:floatVar MyObjects[0][Data_SomeFloat];
    
    new 
String:stringVar[32];
    
strcopy(stringVarsizeof(stringVar), MyObjects[0][Data_SomeString];

The object reference is the index (first dimension) in the MyObjectArray and the object data is values in the second dimension.

The advantage with this structure is that it's really fast for direct lookup on values. It's also easy to loop through objects. The compiler also does tag checking so you get some sort of type safety. How fast you can search through it depends on how you order stuff, but there's a trick to get fast search too, which is explained later below.

Obviously it will use a constant amount of memory, whether all indexes is used or not. So this structure is not suitable for large amounts of indexes. If you get above 500-1000 entries you should reconsider your design - or use a database. If the structure itself is small you need to use really many indexes to spend just a few KB of RAM. But this is perfectly suitable for smaller amounts like storing a player class data or cached client settings.


Indexes - Looking up stuff fast

With many objects a linear search in a loop might be too slow, especially if you have to search for several objects in the same game event.

Usually you look up objects by a certain value, like a name. To speed up this search ADT tries can be used as indexes. A trie is a dynamic structure that maps a key (string) to a value. Its data is structured in a way that we can get very fast lookup on keys.

When creating a object we can use the object name (or other string value) as the key and set the value to the object index in the array. To find a object, all you need to do is to look it up by its name and you quickly get the object index in return.

Again this will result in some redundancy and more memory usage but it's usually just a matter of a few KB, which is a lot of data if you just store numbers and short strings.

PHP Code:
// Example of player class attributes.
enum ClassAttributes 
{
    
String:Class_Name[32],
    
Float:Class_Speed,
    
Class_OtherAttribute
}

const 
MAX_CLASSES 20;

// Class data.
new ClassData[MAX_CLASSES][ClassAttributes];

// Number of classes.
new ClassCount 0;

// Handle to trie with class name index.
new Handle:ClassNameIndex;

CreateClass(const String:name[], Float:speedotherAttribute)
{
    
// Set class data.
    
strcopy(ClassData[ClassCount][Class_Name], 32name);
    
ClassData[ClassCount][Class_Speed] = speed;
    
ClassData[ClassCount][Class_OtherAttribute] = otherAttribute;
    
    
// Add name to index.
    
SetTrieValue(ClassNameIndexnameClassCount);
    
    
ClassCount++;
}

FindClass(const String:name[])
{
    new 
classIndex = -1;
    
    
// Lookup the class name in the trie index. The value is inserted into the
    // classIndex variable.
    
if (!GetTrieValue(ClassNameIndexnameclassIndex))
    {
        
// Not found.
        
return -1;
    }
    
    return 
classIndex;
}

TestCase()
{
    
// The trie should be created somewhere, but we do it here to make
    // it simple.
    
ClassNameIndex CreateTrie();
    
    
// Create some test classes.
    
CreateClass("test class"2502);
    
CreateClass("other"4003);
    
CreateClass("third"3006);
    
CreateClass("something"2001);
    
    
// Find a class by name.
    
new theClass FindClass("third");
    
    
// Check if found.
    
if (theClass >= 0)
    {
        
// Found it. Do your stuff.
    
}

There's a similar tutorial in the AMX Mod X forum about enums and arrays: http://forums.alliedmods.net/showthread.php?p=1319714

My tutorial is the SourcePawn version. It's the same concepts as the other one, but I figured this would be useful anyways. The other one didn't have my idea of using indexes.
__________________
Richard Helgeby

Zombie:Reloaded | PawnUnit | Object Library
(Please don't send private messages for support, they will be ignored. Use the forum.)

Last edited by rhelgeby; 09-26-2013 at 18:22.
rhelgeby is offline
Send a message via MSN to rhelgeby
DarkEnergy
SourceMod Donor
Join Date: Apr 2008
Location: Georgia Tech, MSECE
Old 05-21-2011 , 02:11   Re: [Tutorial] Data Structures With Indexes
Reply With Quote #2

very nice, too bad pawn is specifically designed not to be OOP
these data structs can store alot of data, it will increase memory usage and compile time, but memory is cheap anyway (if your single plugin uses less than 1MB)
__________________
War3:Source Developer
"Your CPU is just a bunch of Muxes"
DarkEnergy is offline
GoD-Tony
Veteran Member
Join Date: Jul 2005
Old 05-21-2011 , 10:29   Re: [Tutorial] Data Structures With Indexes
Reply With Quote #3

I like this idea and I think I have a use for it already.

Quote:
Originally Posted by DarkEnergy View Post
these data structs can store alot of data, it will increase memory usage and compile time, but memory is cheap anyway (if your single plugin uses less than 1MB)
I think it's still useful if you need the speed and are making use of the arrays you're creating.
GoD-Tony is offline
rhelgeby
Veteran Member
Join Date: Oct 2008
Location: 0x4E6F72776179
Old 05-21-2011 , 10:42   Re: [Tutorial] Data Structures With Indexes
Reply With Quote #4

Assuming you're talking about the JIT? I don't worry about compile time since plugins are loaded when the server is pretty busy anyways. These plugins are too small and I think you have to make really big plugins for it to be noticeable. The Zombie:Reloaded plugin is 160 KB, loads very fast - even on an old server. Does anyone know a SM plugin bigger than this?

I don't worry that much about memory usage either, but of course I try to avoid overuse. As mentioned, if a plugin use less than 1-2 MB it's fine. That's huge if you're just storing numbers and even a few strings.

My main focus in this tutorial is the use of tries as indexes for fast lookup. Since I've never seen it anywhere in the forum I thought it would be useful. And you can of course have multiple indexes on an array if you need fast lookup by various stuff.

When talking about performance I used to be careful about doing expensive stuff. I used to worry about ADT arrays and other stuff like that, even the native call overhead. I've seen other plugins caching stuff like results from IsPlayerConnected, which has almost no benefit (even if used in OnGameFrame). Apparently there's no worries according to this discussion: http://forums.alliedmods.net/showthr...11#post1162011

I still prefer using real arrays with enums because the compiler does tag checking. A value object in the examples above can also be inserted in an ADT array with PushArrayArray. You create a single dimension array using the enum and push that array into the ADT array. The drawback is that you have to get the whole array back even though you just want a single value.

Maybe there's an extension that provides natives for creating value objects, where the compiler is able to do tag checks?
__________________
Richard Helgeby

Zombie:Reloaded | PawnUnit | Object Library
(Please don't send private messages for support, they will be ignored. Use the forum.)

Last edited by rhelgeby; 05-21-2011 at 20:25.
rhelgeby is offline
Send a message via MSN to rhelgeby
strontiumdog
Veteran Member
Join Date: Jan 2007
Location: BC, Canada
Old 05-21-2011 , 18:27   Re: [Tutorial] Data Structures With Indexes
Reply With Quote #5

Nice tutorial Richard.
Left4DoD is 165kb compiled at the moment. Takes 20secs to compile on my machine.
But a lot of the code in it is inefficient because I'm too lazy to optimize it!
As you say, SM is pretty fast and runs stuff quickly.
__________________
Plugins | TheVille
Zombie Mod for DoD:S - l4dod.theville.org
strontiumdog is offline
DarkEnergy
SourceMod Donor
Join Date: Apr 2008
Location: Georgia Tech, MSECE
Old 05-22-2011 , 16:12   Re: [Tutorial] Data Structures With Indexes
Reply With Quote #6

war3source combined is 1.2M compiled code, but thats the sum of 90+ plugins, obviously quite some redundancy.
__________________
War3:Source Developer
"Your CPU is just a bunch of Muxes"
DarkEnergy is offline
Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -4. The time now is 04:21.


Powered by vBulletin®
Copyright ©2000 - 2024, vBulletin Solutions, Inc.
Theme made by Freecode