[Keeper]

After the update last night, dlsym is not returning any addresses from the binaries.

using the vtablecheck (rebuilt with new binary info) and tfscan.sh ( as well as Mani's vtable functions ) I can't get any addresses.

Here's the output

sdktools.games.ep2.txt
Invalid vtable symbol "_ZTV9CTFPlayer"
Invalid vtable symbol "_ZTV9CTFPlayer"
Invalid vtable symbol "_ZTV9CTFPlayer"
Invalid vtable symbol "_ZTV9CTFPlayer"
Invalid vtable symbol "_ZTV9CTFPlayer"
Invalid vtable symbol "_ZTV9CTFPlayer"
Invalid vtable symbol "_ZTV9CTFPlayer"
Invalid vtable symbol "_ZTV9CTFPlayer"
Invalid vtable symbol "_ZTV9CTFPlayer"
Invalid vtable symbol "_ZTV9CTFPlayer"
Invalid vtable symbol "_ZTV9CTFPlayer"
Invalid vtable symbol "_ZTV9CTFPlayer"
Invalid vtable symbol "_ZTV9CTFPlayer"
Invalid vtable symbol "_ZTV9CTFPlayer"
sm-tf2.games.txt
Invalid vtable symbol "_ZTV9CTFPlayer"

_ZTV9CTFPlayer is in binary, but for some reason dlsym isn't finding it.

nm server.so | grep _ZTV9CTFPlayer
00bdb380 d _ZTV9CTFPlayer

This is above my head, so I was hoping somebody here could give me a push in the right direction. That "d" in front of the symbol from the output of nm scares me.

EDIT: It seems that the new optimizations hid internal functions. So I guess it's time to do my own scanning

[BAILOPAN]

core/MemoryUtils.cpp in the SM tree has a function for grabbing hidden symbols. props to DS.

[Keeper]

Yeah, I saw that and adapted it for our use ( as a temporary try to fix our issues ). I removed the caching part ... but now that I realized that dladdr doesn't work either, I'm going to have to go back and cache so we can have the address to symbol lookup

Thanks for the info tho. Helps to know I definitely went in the right direction.

So does all this "optimization" really help? Will performance really be enhanced? I'm trying to figure out if all this really served a purpose.

Thanks again!

[BAILOPAN]

all this "optimization" ... I'm trying to figure out if all this really served a purpose.

If you meant Valve's GCC optimizations, not sure. Someone who's looked at them should comment.

If you meant our caching optimizations, which you mentioned removing and then adding back in -

It's a trivial 200-line hash table implementation, 30% of which is a copy-pasted hash function.

My benchmarks showed that a single string comparison cost about 40 nanoseconds, so a worst case hit would cost 2ms. That's pretty bad. Hash table would essentially be free. Adding an entry into the hash table costs 1,100 nanoseconds, so up-front conversion of the whole thing costs about 60ms. That's really cheap, but just to be safe we incrementally fill the table as we search.

You can do even better by getting rid of the malloc() call - easily doable by pre-allocating the hash table memory parallel to the ELF section size. I tried this just now and it's about a 5X win, bringing the cost for a complete table fill down to 14ms.

I don't know what your workloads are like, but the cache is not necessary for the dlhsym() kernel to work. SM always assumes worst-case since it's extensible. My guess is all of this happens on server startup so as long as it says under half a second or so, it won't matter (but it's not the last cookie that makes you fat).

[BAILOPAN]

On the Valve end, the only thing I heard is that they re-added fPIC or something which is a perf loss. Hiding the symbols is about lowering the binary size.

[Mani]

I think we only really used the dladdr() function for working out vfunc offsets and that is really only a tool for displaying them in the console rather than dynamically working them out on the fly.

It was a reasonable way for a user to be able to set the offsets up without needing access to the shell.

It would be quite nice to just have a database somewhere that holds all the symbols/addresses which is updated per Valve release.

Mani

[Keeper]

Ya, I should have been more specific. I was talking about VALVe's optimizations. Seems like a silly thing to do to remove 4 megs from the binary. There is so much code that could be cleaned up that would probably do the same thing. But I guess a switch on gcc is much easier