[CS:GO] I need some assistance with hooking and detouring a function

LionDoge · **Author**

Hello,
I've been trying to make it so you can penetrate every wall, however I quickly found out that the Trace function that is called when doing the penetration calculation has a max distance given as one of the arguments, which means that the bullet will just stop if the wall is too thick.

I figured that I maybe could hook the function and change it's argument to a bigger distance. I managed to find the signature, but I can't figure out the arguments and calling convention stuff at all, everything leads to the game crashing. This is the message I get before the server crashes:

Code:

[DHOOKS] FATAL: Failed to find return address of original function. Check the arguments and return type of your detour setup.

This is my setup so far:

Code:

Handle hTraceToExit;

public void OnPluginStart()
{
	Handle hGameData = LoadGameConfigFile("penetration.games");
	
	hTraceToExit = DHookCreateDetour(Address_Null, CallConv_FASTCALL, ReturnType_Bool, ThisPointer_Ignore);
	DHookSetFromConf(hTraceToExit, hGameData, SDKConf_Signature, "TraceToExit");

	DHookAddParam(hTraceToExit, HookParamType_VectorPtr, .flag=DHookPass_ByVal);
	DHookAddParam(hTraceToExit, HookParamType_VectorPtr, .flag=DHookPass_ByVal);
	DHookAddParam(hTraceToExit, HookParamType_VectorPtr, .flag=DHookPass_ByRef);
	DHookAddParam(hTraceToExit, HookParamType_ObjectPtr, .flag=DHookPass_ByRef);
	DHookAddParam(hTraceToExit, HookParamType_ObjectPtr, .flag=DHookPass_ByRef);
	DHookAddParam(hTraceToExit, HookParamType_Float, .flag=DHookPass_ByVal);
	DHookAddParam(hTraceToExit, HookParamType_Float, .flag=DHookPass_ByVal);

	DHookEnableDetour(hTraceToExit, false, Hook_TraceToExit);
}

public MRESReturn Hook_TraceToExit(Handle hParams)
{
	PrintToServer("TraceToExit called!");
	return MRES_Handled;
}

Gamedata:

Code:

"Games"
{
	"csgo"
	{
		"Signatures"
		{
			"TraceToExit"
			{
				"library"	"server"
				"windows"	"\x55\x8B\xEC\x83\xEC\x4C\xF3\x0F\x10\x75\x2A"
				"linux"		"\x55\x89\xE5\x57\x56\x53\x81\xEC\xFC\x00\x00\x00\xF3\x0F\x10\x45\x2A"
			}
		}
	}
}

I even looked at the game code to see which arguments it expects, here they are:

Code:

static bool TraceToExit( Vector start, Vector dir, Vector &end, trace_t &trEnter, trace_t &trExit, float flStepSize, float flMaxDistance )

I'm confused with the arguments here, since the Vectors seem to be passed by value here does it mean that I can't use HookParamType_VectorPtr when hooking? should I use a object in that case?

I'm also wondering if it's even possible at all to do what I want to, next thing that comes to mind is just straight up modifying the value in the binary...
Sorry if I missed anything important. Thanks!

I'm using SourceMod on Windows 10 however it would also be nice to get it working on Linux.