[Backstabnoob]

I'm not sure if this is a bug or if I'm just talking bullshit so I'd like to discuss it here first before I'll report it as a bug. Sorry if it bothers anyone; just tell me and I'll wipe the thread.

Arrays seem to allocate unnecessary amount of memory. The syntax for creating an array is this:
ArrayCreate(cellsize=1, reserved=32). The problem is with the reserved value, which is basically the amount of elements that are pre-allocated to prevent the array's resizing all the time. However, if this value is <= 8, it's not taken into account at all because it seems the minimum value is 8 from this code:
https://github.com/alliedmodders/amx...astructs.h#L53

This would be fine, however the array always gets initialized with cursize=0:
https://github.com/alliedmodders/amx...astructs.h#L75

So if I want to create a dynamic array that I'm sure will have X elements and no higher, where X is for example two, that's 4x more memory allocated than necessary.

The solution to this should be simply initializing the array with cursize=1.

Again, I'm not sure if this is correct, so please discuss ^_^

Test code:

PHP Code:


#define RESERVED_MEMORY X
#include < amxmodx >

public plugin_init( )
{
    register_concmd( "getmemory", "_getmemory" )
}

public _getmemory( )
{
    
    new Array: iArray
    
    for( new i; i < 100000; i ++ )
    {
        iArray = ArrayCreate( 32, RESERVED_MEMORY )
        
        ArrayPushCell( iArray, 12345678 )
        ArrayPushString( iArray, "some string" )
        ArrayPushCell( iArray, 12345678 )
        ArrayPushString( iArray, "some string" )
        ArrayPushCell( iArray, 12345678 )
        ArrayPushString( iArray, "some string" )
        ArrayPushCell( iArray, 12345678 )
        ArrayPushString( iArray, "some string" )
    }
} 


With X=1 and 8, the memory usage was 120MB. With X=9, the memory usage was a bit higher, like 130-140MB. With pushing more than 8 elements into the array with X<=8, the memory usage doubled right away to 240MB which is caused by this code:
https://github.com/alliedmodders/amx...astructs.h#L59
That could also be changed as it's still a waste in most cases.

[Nextra]

The relevant code is inside the ArrayCreate native: https://github.com/alliedmodders/amx...tructs.cpp#L79

The vector is created with cursize 0 and then immediately grows to the desired reserved size. It will sit there until your data outgrows that reserved size. GrowIfNeeded is only hit with cursize == 0 if you use ArrayCreate(1, 0) and then push elements. Most of the time it would be too much to worry about wasting space of a couple of cells of memory, so the code uses 8 as a baseline size. Otherwise it would have to re-allocate four times before hitting that size, and I'd argue that most cellarrays in plugins will contain 8+ elements at some point anyway.

Doubling the space inside the vector if we hit the limit is a simple and common allocation strategy with this type of container. Pushing it to the extreme with 100.000 of them is guaranteed to give you some odd memory usage patterns. This is rarely a concern for the common use case, and you can always specify a suitable reserved size if absolutely necessary.

Most of the time you don't even have to worry about this, though. There is a reason the default for reserved is 32.

[Backstabnoob]

I'm working on a MySQL data layer which uses one dynamic array per one row in the database. The elements are the columns, so I know exactly how many I'll need and being forced to use 8 despite only needing 2 at all times is a huge waste for me. Like I said, if the array is created with 1 or 8 reserved changes absolutely nothing in regards of memory usage. Using more does, though.

It's probably not that much of a deal with a couple of arrays, but when you're dealing with thousands of then, it becomes a problem. Using 4x more memory then I'd ever need with this particular "class" (it's an API which let's you define the amount of columns yourself and 8 will almost always be more than needed) is a huge drawback to an otherwise extremely fast and efficient way to deal with MySQL.

This persists on both 1.8.2 and 1.8.3. There's an ArrayResize() native on 1.8.3 which probably could work, but I'd guess it completely reallocates the array.

Edit: My code might have been unclear. Even when creating those arrays with reserved=1 and pushing no data into them, the memory usage is the same. It just seems to jump to 8 if it's any lower than that no matter what.

[Backstabnoob]

Doh! Looks like I was wrong. This has been fixed in 1.8.3, I tested it wrong.

My bad.

[asherkin]

You should consider flipping your system to use one array per column instead, this also lets you keep data typing sane.

[Backstabnoob]

I thought about that, however it would be quite difficult to do that as the rows are per-player basis and don't need to be unique. I'd also have to create a new array and somehow copy the data from those arrays into it per player. Right now I don't need to do that, this is pretty fast and easy to work with.