[vancelorgin]

*Windows only* - I'm switching to linux dev, so this is sorta a fair well to the win32 thiscall convention. Yet a fuckin gain, not for noobs, but don't be intimidated by its ugliness - it's perfectly safe, if used right

Wanna hook stuff? Well, back in hl1, all you needed to do was wrap the server dll and you could act as a proxy between the mod and the engine, as all the funcs were simple exports and struct memebers. Not anymore - valve has moved to interfaces. Understandable - much more flexible, faster, overall better. Sadly, hooking is a bit tougher. Lucky for you I've put together a little toolkit for doing so.

A little, tiny primer on what virtual funcs are. Lets take IVEngineServer engine for example. Now, the first thing inside engine is a pointer to a table full of addresses. These addresses are the addrs of the virtual funcs of that class. For example the 43rd address in IVEngineServer's virtual function table is a pointer to UserMessageBegin.

So lets say you want to hook all messages being sent from the server to clients. How can you do this? Well, first, slap this somewhere into your project:

class CCallGate {
public:
	CCallGate(){
		m_ulOrigFunc = 0;

		m_ulPreCall = 0;
		m_ulPostCall = 0;
		
		m_ulOldRet = 0;
		m_ulThis = 0;

		m_ulOldESP = 0;
		m_ulNewESP = 0;
	}

	CCallGate(void* pOrigFunc, void* pPreCall, void* pPostCall){
		CCallGate();

		BuildGate(pOrigFunc, pPreCall, pPostCall);
	}

	void Build(void* pOrigFunc, void* pPreCall, void* pPostCall){
		BuildGate(pOrigFunc, pPreCall, pPostCall);
	}

	void* Gate(){
		return (void*)m_szGate;
	}

	unsigned long RetAdd(){
		return m_ulOldRet;
	}
	
private:
	void BuildGate(void* pOrigFunc, void* pPreCall, void* pPostCall){
		m_ulOrigFunc = (unsigned long)pOrigFunc;

		m_ulPreCall = (unsigned long)pPreCall;
		m_ulPostCall = (unsigned long)pPostCall;
	
		char* pCur = m_szGate;

		pCur[0] = '\x8F'; pCur[1] = '\x05'; pCur += 2; //pop m32
		*(unsigned long*)pCur = (unsigned long)&m_ulOldRet; pCur += 4;

		pCur[0] = '\x89'; pCur[1] = '\x0D'; pCur += 2; //mov m32, ecx
		*(unsigned long*)pCur = (unsigned long)&m_ulThis; pCur += 4;

		if(m_ulPostCall != NULL){
			pCur[0] = '\x89'; pCur[1] = '\x25'; pCur += 2; //mov m32, esp
			*(unsigned long*)pCur = (unsigned long)&m_ulOldESP; pCur += 4;
		}

		if(m_ulPreCall != NULL){
			*pCur = '\x51'; pCur++; //push ecx

			*pCur = '\xA1'; pCur++; //mov eax, m32
			*(unsigned long*)pCur = (unsigned long)&m_ulPreCall; pCur += 4;

			pCur[0] = '\xFF'; pCur[1] = '\xD0'; pCur += 2; //call eax
		
		}else{
			pCur[0] = '\x83'; pCur[1] = '\xEC'; pCur[2] = '\x04'; pCur+= 3; //sub esp, 0x04
		
		}

		pCur[0] = '\x8B'; pCur[1] = '\x0D'; pCur += 2; //mov ecx, m32
		*(unsigned long*)pCur = (unsigned long)&m_ulThis; pCur += 4;

		*pCur = '\xB8'; pCur++; //mov eax, u32        this mov push ret
		*(unsigned long*)pCur = (unsigned long)pCur + 4 +  3 + 5 +  1; pCur += 4;

		pCur[0] = '\x89'; pCur[1] = '\x04'; pCur[2] = '\x24'; pCur += 3; //mov [esp], eax

		*pCur = '\x68'; pCur++; //push u32
		*(unsigned long*)pCur = m_ulOrigFunc; pCur += 4;

		*pCur = '\xC3'; pCur++; //ret

		if(m_ulPostCall != NULL){
			pCur[0] = '\x89'; pCur[1] = '\x25'; pCur += 2; //mov m32, esp
			*(unsigned long*)pCur = (unsigned long)&m_ulNewESP; pCur += 4;

			pCur[0] = '\x8B'; pCur[1] = '\x25'; pCur += 2; //mov esp, m32
			*(unsigned long*)pCur = (unsigned long)&m_ulOldESP; pCur += 4;

			pCur[0] = '\xFF'; pCur[1] = '\x35'; pCur += 2; //push m32
			*(unsigned long*)pCur = (unsigned long)&m_ulThis; pCur += 4;
			
			*pCur = '\x50'; pCur++; //push eax
			
			*pCur = '\xA1'; pCur++; //mov eax, m32
			*(unsigned long*)pCur = (unsigned long)&m_ulPostCall; pCur += 4;

			pCur[0] = '\xFF'; pCur[1] = '\xD0'; pCur += 2; //call eax
			
			pCur[0] = '\x8B'; pCur[1] = '\x25'; pCur += 2; //mov esp, m32
			*(unsigned long*)pCur = (unsigned long)&m_ulNewESP; pCur += 4;
		}

		pCur[0] = '\xFF'; pCur[1] = '\x35'; pCur += 2; //push m32
		*(unsigned long*)pCur = (unsigned long)&m_ulOldRet; pCur += 4;

		*pCur = '\xC3'; pCur++; //ret
	}

	char m_szGate[0x60];

	unsigned long m_ulOrigFunc;

	unsigned long m_ulPreCall;
	unsigned long m_ulPostCall;
	
	unsigned long m_ulOldRet;
	unsigned long m_ulThis;

	unsigned long m_ulOldESP;
	unsigned long m_ulNewESP;
};

inline bool DeProtect(void* pMemory, unsigned int uiLen){
#ifndef __LINUX__
	DWORD dwIDontCare;

	return (VirtualProtect(pMemory, uiLen, PAGE_EXECUTE_READWRITE, &dwIDontCare) ? true : false);
#else
#error jackass
	return (!mprotect(pMemory, uiLen, PROT_READ | PROT_WRITE | PROT_EXEC) ? true : false);
#endif
}

#define VTBL( classptr ) (*(DWORD*)classptr)
#define PVFN_( classptr , offset ) (VTBL( classptr ) + offset)
#define VFN_( classptr , offset ) *(DWORD*)PVFN_( classptr , offset )
#define PVFN( classptr , offset ) PVFN_( classptr , ( offset * sizeof(void*) ) )
#define VFN( classptr , offset ) VFN_( classptr , ( offset * sizeof(void*) ) )

Yich. What's all that? Well, it's a class that assembles a gate function for virtual functions that will call your functions before and/or after the original function is called. Wtf, you may ask? Basically, it handles the ugly crap required for you to hook virtual class funcs. The format for the callbacks are:

void __cdecl PreFunc(classtype* pclassinst, <args>){ }
returntype __cdecl PreFunc(returntype retval, classtype* pclassinst, <args>){ return retval; }

Back to our example:

CCallGate GateEngine_UserMessageBegin; //gates
CCallGate GateEngine_MessageEnd;

...

bf_write* bfw = NULL;
int imsgtype = 0;

//the funcs to call
bf_write* PostUserMessageBegin(bf_write* pBitBuf, IVEngineServer* pEngine, IRecipientFilter *filter, int msg_type){
	bfw = pBitBuf;
	imsgtype = msg_type;

	return pBitBuf;
}

void PostMessageEnd(void* pNothing, IVEngineServer* pEngine){
	if(!bfw)
		return;
	
	bf_read r(bfw->GetData(), bfw->GetNumBytesWritten());

//TODO: anything..

	bfw = NULL;
}

...

Load:
	DeProtect((void*)VTBL(engine), 1024); //deprotect the virtual function table, a length of some abritrary ass amount
	DWORD* pUserMessageBegin = (DWORD*)PVFN(engine, 43); //get a pointer to usermessagebegin
	DWORD* pMessageEnd = (DWORD*)PVFN(engine, 44); //get a pointer to messageend
	GateEngine_UserMessageBegin.Build((void*)*pUserMessageBegin, NULL, PostUserMessageBegin); //build the gate for pUserMessageBegin with no precallback and PostUserMessageBegin as a postcallback
	GateEngine_MessageEnd.Build((void*)*pMessageEnd, NULL, PostMessageEnd); // /s/UserMessageBegin/MessageEnd
	*pUserMessageBegin = (DWORD)GateEngine_UserMessageBegin.Gate(); //set engine's vtable's pUserMessageBegin entry to the address of your newly constructed gate
	*pMessageEnd = (DWORD)GateEngine_MessageEnd.Gate(); //restart the computer!one

Voila. Now your funcs are called every time *ANYTHING* calls engine->BeginUserMessage - beit server.dll, some plugin, your plugin. How'd I get the addr's of Begin/End message? Well, I'm lazy, so I put engine->UserMessageBegin and MessageEnd in my load func, _asm int 03'd, and loaded a debugger. But you could really just count down the interface declaration, the first virtual func is 0, second is 1, third is 2, and I think you can guess the rest. Some other nifty funcs are 21 = CreateEdict and 22 = RemoveEdict. Please note that this works on absolutly any class with virtual funcs in it, including, for example, CBaseEntity. Go ahead, hook think. You know you want to.

[BAILOPAN]

This is really nice, lance, thanks for posting this to the public.

It's great that some of the hardcore hackers are willing to share stuff usually these things are pretty black-boxed.

As for a linux port, be wary... gcc tends to do things differently with certain compiler flags and such. Linux isn't pretty for hacks like these.

For example, you've already used mprotect wrong ;] it returns 0 on success, -1 on failure, and pMemory must be aligned to _SC_PAGESIZE (usually 4K I think).

[vancelorgin]

Hence the #error I was expecting somebody to point out my horrible attempt at mprotect. It was theoretical as I didn't even have linux installed at the time I wrote it.

As for gcc being mean - I'm expecting it to be, however inline asm is alot more annoying and the linux debuggers all suck as well.

[Geesu]

We need an "over my head" emoticons

[XAD]

Originally Posted by vancelorgin Wanna hook stuff? Well, back in hl1, all you needed to do was wrap the server dll and you could act as a proxy between the mod and the engine, as all the funcs were simple exports and struct memebers. Not anymore - valve has moved to interfaces. Understandable - much more flexible, faster, overall better. Sadly, hooking is a bit tougher. Lucky for you I've put together a little toolkit for doing so.

Very nice info... got me thinking (a bad thing )...

Why the machine code?? Why so complicated (or is it about stability)??
With your info about "virtual" I checked the vtable and then thought why not change the pointers in the original vtable to your own functions??
I tested the engine->LogPrint function on my linux box and has "routed" the engines virtual call to my function in my plugin which is printing a message and then calls the original function... (My test: when I enter "say hello" in the console my message is printed and then the log line.)

	*((int*)&OrigLogPrint) = (*(int**)engine)[76];
	(*(int**)engine)[76] = (int)MyLogPrint;

Shouldn't this also work on a Windows box??

The code would be so much simplier as it only copies function pointers casted to int (yea, not nice but what the heck)... One copy to save the original function, and one copy to override the engine's table wih my function...

OR, did I miss something badly??

/X

PS! By using "IVEngineServer::LogPrint" you get the byte offset + 1...

[blaubaer]

offsets nice idea... but when it dosn't work under linux: we have a problem.

the most server runs with linux ;-/

mfg bear

[XAD]

Originally Posted by blaubaer offsets nice idea... but when it dosn't work under linux: we have a problem.

Reread the previous post again.... /X

[blaubaer]

i have write this post sime hour before and press post at this time *lol

[vancelorgin]

If you read my post, you'll see that I'm switching to linux dev....

If MSVC will let you put class->virtualfunc into ... params, then I wouldn't trust it worth a damn - it'd probably just spit out the address of the func, which is not what we want anyway (we want the address of it in the table) - you could, however, detour that func. That's another class of mine

Regarding 'why the machine code' - for thiscall funcs on a windows box (class methods), it is required that the address of the object be put into ecx (this). Beyond that trying to call a dword ptr as a thiscall in msvc requires inline asm as well. I got tired of writing that crap over and over, so just made it a class. Wasn't much trouble, really :s The way linux does thiscalls is one reason I'm switching [so much easier], but the real reason is the server binaries have.. uh.. some nice strings in them

About a custom vtable - I was doing that a little while ago. I replaced the vtable pointer on some health_vial's I spawned to my own custom table with some funcs overwritten on it. It worked fine for a while, until one of the physics funs was called (it worked until the ents stopped rolling around - ent::physicssleep or something) - then it just crashed. I copied like 0x5000 bytes on the vtable, so I doubt it overshot). I was working on finding out wtf I forgot, but then my mission became to switch to linux.

[XAD]

Originally Posted by vancelorgin About a custom vtable - I was doing that a little while ago. I replaced the vtable pointer on some health_vial's I spawned to my own custom table with some funcs overwritten on it...

Hmm, I don't change the vtable pointer but the function pointers in the vtable...
I'm now implemented it using a function-structure based on the IVEngineServer class (as in MetaMod) and is copying all virtual functions in the engine, which also make it easy to calculate the function index, not nice C++ thing but a simple solution..

Macros:

#define ENGINEFUNC(func) ( (int)( (void**)&g_pOrigEngine.func - (void**)&g_pOrigEngine ) )
#define SET_ENGINEFUNC(func,myfunc) ( (*(void***)engine)[ENGINEFUNC(func)] = (void*)&myfunc )

Plugin code:

// Define a copy of the original engine functions...
static ivengineserver_t g_pOrigEngine;

// Override LogPrint to MyLogPrint...
SET_ENGINEFUNC(LogPrint,MyLogPrint);

/X