Raised This Month: $32 Target: $400
 8% 

[RESULTS] Server-Side Bot Detection Using Artificial Neural Networks


Post New Thread Reply   
 
Thread Tools Display Modes
Author Message
SWAT_88
SourceMod Donor
Join Date: Jan 2008
Location: Austria
Old 07-27-2015 , 15:24   [RESULTS] Server-Side Bot Detection Using Artificial Neural Networks
Reply With Quote #1

Dear contributors,

Sorry for the delay. I was not allowed to publish my thesis until I've finished my final exam.
If you are interested in the results of my Master's Thesis, have a look here.

It's written completely in English and most of the results and data (diagrams, heat maps, etc.) are also included in the thesis.
You can find the results in the chapter "Evaluation", the diagrams and heat maps are shown in the appendix A.

If you are interested in the recorded data or the source code, I've uploaded everything to a shared folder using Google Drive.
You can also find the link to this folder in the appendix A.

To all players on my server.
Thank you very much for your contribution. I really appreciate it.
Without you my master thesis would not have been possible.


Have a nice day :)
__________________

Last edited by SWAT_88; 04-24-2022 at 05:01. Reason: Updated Google Short-Url to Firebase Short-Url
SWAT_88 is offline
RedSword
SourceMod Plugin Approver
Join Date: Mar 2006
Location: Quebec, Canada
Old 07-28-2015 , 10:06   Re: [RESEARCH] Play for science with GLaDOS
Reply With Quote #2

If anyone is wondering, it is for the games CSS and HL2DM.

Note that I tried to connect to your first server...
Code:
Connecting to [An-ip-address]:27095...
STEAM validation rejected
And you say you need 100 players; but shouldn't you get some pros in that 100 (or else your neural would give false positive) ?

Good luck with your thesis,

Red

post scriptum : my ping is too high; so I guess you only target Europeans
__________________
My plugins :
Red Maze
Afk Bomb
RAWR (per player/rounds Awp Restrict.)
Kill Assist
Be Medic

You can also Donate if you appreciate my work

Last edited by RedSword; 08-13-2015 at 16:50. Reason: SWAT88 wanting the IP removed
RedSword is offline
SWAT_88
SourceMod Donor
Join Date: Jan 2008
Location: Austria
Old 07-28-2015 , 11:46   Re: [RESEARCH] Play for science with GLaDOS
Reply With Quote #3

Quote:
Originally Posted by RedSword View Post
If anyone is wondering, it is for the game CSS.

Note that I tried to connect to your first server...
--snip--
And you say you need 100 players; but shouldn't you get some pros in that 100 (or else your neural would give false positive) ?

Good luck with your thesis,

Red

post scriptum : my ping is too high; so I guess you only target Europeans
First of all, thank you very much for your help. Highly appreciated.
What was your ping? I can try to raise the minimum ping. Currently it is set to 100.

@Pros:
I need pros and noobs to get correct results
During the training of the artificial brain there are no false positives because the brain is in training. It can only fail to recognize players or bots, but then I need to change the number of the neurons or the layout of the neurons in the brain.
If everything fails, there is still a backup method called "symbolic regression", I will try that method as well to distinguish between bots and humans.

Movement and Aiming is one part of the detection, but there are also other ways of detecting bots, like "how often did the user change his weapon during a round", or "shooting at objects without a reason", and so on.
When using artificial neural networks, or symbolic regression, you do not need to create a relation between the data, the network or the symbolic regression methods will figure it out for you.
You just have to tell the network which data belongs to bots, and which data belongs to humans.

@Error:
Is this a permanent error? I cannot reproduce it. Maybe it's a client problem.

Again thanks for testing.
Best regards
__________________

Last edited by SWAT_88; 08-13-2015 at 15:07.
SWAT_88 is offline
friagram
Veteran Member
Join Date: Sep 2012
Location: Silicon Valley
Old 07-28-2015 , 12:35   Re: [RESEARCH] Play for science with GLaDOS
Reply With Quote #4

https://forums.alliedmods.net/misc.php?do=showrules

Quote:
•Do not advertise (including hosting services, game servers and gaming communities).
__________________
Profile - Plugins
Add me on steam if you are seeking sp/map/model commissions.
friagram is offline
psychonic

BAFFLED
Join Date: May 2008
Old 07-28-2015 , 13:00   Re: [RESEARCH] Play for science with GLaDOS
Reply With Quote #5

Quote:
Originally Posted by friagram View Post
He's a long-time member, with released plugins, just asking for help gathering data for his thesis.

He's not advertising any particular community nor permanent servers.

I think we can let this one slide.

Last edited by psychonic; 07-28-2015 at 14:14.
psychonic is offline
SWAT_88
SourceMod Donor
Join Date: Jan 2008
Location: Austria
Old 07-28-2015 , 14:11   Re: [RESEARCH] Play for science with GLaDOS
Reply With Quote #6

Quote:
Originally Posted by psychonic View Post
He's a long-time member, with released plugins, just asking for help gathering data for his thesis.

He's not advertising any particularly community nor permanent servers.

I think we can let this one slide.
Thank you psychonic.

Asking for some data is not the kind of thing I usually do. But scientific research often requires data gathering (I'm not overwhelming happy about it either).
I'm currently studying "Game-Engineering" in Vienna, and my master thesis has to be related to gaming. Therefore I thought, using sourcemod for advanced anti cheat algorithms would be nice.

If it turns out that server-side bot detection is possible, I can also upload my results and even write a sourcemod plugin that is utilizing this technique for a better anti cheat system.
So this data gathering might be useful for us all.

In the meantime I want to thank all the people who already have contributed.
And I also want to thank all the moderators who are not deleting this thread.

Best regards
__________________

Last edited by SWAT_88; 07-29-2015 at 03:26.
SWAT_88 is offline
RedSword
SourceMod Plugin Approver
Join Date: Mar 2006
Location: Quebec, Canada
Old 07-29-2015 , 04:26   Re: [RESEARCH] Play for science with GLaDOS
Reply With Quote #7

@Error it is still the case @ [An-ip-address]:27095

I can connect fine to 27085 though.

Also, I'm not entirely sure which players you're recording for your neural network, but be careful as it seems you're saving bots data :
Code:
[GLaDOS] Saving data for [BOT] Calvin (281 records)
.

Regarding my ping, it seems to go up to 119 on your 27085 server.

Red
__________________
My plugins :
Red Maze
Afk Bomb
RAWR (per player/rounds Awp Restrict.)
Kill Assist
Be Medic

You can also Donate if you appreciate my work

Last edited by RedSword; 08-13-2015 at 16:50. Reason: SWAT88 wanting the IP removed
RedSword is offline
SWAT_88
SourceMod Donor
Join Date: Jan 2008
Location: Austria
Old 07-29-2015 , 16:16   Re: [RESEARCH] Play for science with GLaDOS
Reply With Quote #8

Quote:
Originally Posted by RedSword View Post
@Error it is still the case @27095

I can connect fine to 27085 though.

Also, I'm not entirely sure which players you're recording for your neural network, but be careful as it seems you're saving bots data :
-snip-
Regarding my ping, it seems to go up to 119 on your 27085 server.

Red
Saving the bot data was my intention. I need data from humans and bots to train the artificial brain.
After the training my neural network should be able to tell, if a player is a bot or a human, therefore I need both data.

There is a flag in my recorded data which tells me if the current line was recorded from a bot or a human.
I need this flag for training. Later (i.e. during testing), this flag is not used and also not available (because I don't know if a player is a cheater or not).

You are right, I can probably collect the bot data on my home computer, but the server is running and the bots are moving. So I thought I also record them on the server, saves me some time :)

@Ping:
I've raised the max ping to 140 on both servers.

@27095:
Apparently a file called "steam_appid.txt" was missing, I fixed that. Can you try it again?

Thank you very much for your feedback :)
__________________

Last edited by SWAT_88; 08-13-2015 at 15:08.
SWAT_88 is offline
RedSword
SourceMod Plugin Approver
Join Date: Mar 2006
Location: Quebec, Canada
Old 07-29-2015 , 23:06   Re: [RESEARCH] Play for science with GLaDOS
Reply With Quote #9

Quote:
Originally Posted by SWAT_88 View Post
Saving the bot data was my intention.
I understand, but I believe custom bots can also exist (i.e. DoD:S bots are unofficial iirc, but if they were to be ported to css the behaviour wouldn't be the same I believe), and that those may not be recognized then. I mean I believed you would detect if a behaviour was human or not, not "probably human" or "probably bot". I thought that would ease ambiguity.

And @ 27095 = same behaviour; did you reboot your server =3 *cough* :$ ?.

edit : 23h20 (GMT -5, +/- 1 maybe @ summer timer) server crashed; wasn't doing much special (remaining alive bot had a shotty).
__________________
My plugins :
Red Maze
Afk Bomb
RAWR (per player/rounds Awp Restrict.)
Kill Assist
Be Medic

You can also Donate if you appreciate my work

Last edited by RedSword; 07-29-2015 at 23:21.
RedSword is offline
SWAT_88
SourceMod Donor
Join Date: Jan 2008
Location: Austria
Old 07-30-2015 , 07:19   Re: [RESEARCH] Play for science with GLaDOS
Reply With Quote #10

Quote:
Originally Posted by RedSword View Post
And @ 27095 = same behaviour; did you reboot your server =3 *cough* :$ ?.
Of course I've restarted the server ^^
But the problem is getting more mysterious.

Apparently the file "steam_appid" was always there.
I just searched for it in the wrong place. So the missing "steam_appid" file was not the problem.

I've downloaded both servers (css and hl2dm) using steamcmd back then.
The file and folder structure on both servers is exactly the same, so there are no files missing on the hl2dm server.

However, I redownloaded and validated the hl2dm server using that same command again.
The validation process doesn't had to re-download some files (which means everything is up-to-date).
I also downloaded a new copy of hl2dm in a separate folder and did a "diff -r" on both directories, they are identical.

Nevertheless I also copied the file "steam_appid.txt" with and without the extension to the mod folder.
Can you test it again?

It's really hard for me to test this behavior, because I don't experience any problems.
Regarding to my data folder, 4 people already have joined my server successfully.

I'm no quite sure, but maybe it's a client problem. I don't know.
Maybe update your HL2DM installation, or do a "verifiy cache".

Quote:
Originally Posted by RedSword View Post
edit : 23h20 (GMT -5, +/- 1 maybe @ summer timer) server crashed; wasn't doing much special (remaining alive bot had a shotty).
Server is restarting everyday at 5:20 AM (GMT+1), so it's not a crash. But thanks for the info.
__________________

Last edited by SWAT_88; 07-30-2015 at 07:35.
SWAT_88 is offline
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -4. The time now is 09:34.


Powered by vBulletin®
Copyright ©2000 - 2024, vBulletin Solutions, Inc.
Theme made by Freecode