Raised This Month: $12 Target: $400
 3% 

[Linux] Automated restart scripts sometimes messing up


Post New Thread Reply   
 
Thread Tools Display Modes
Author Message
ghostofmybrain
Veteran Member
Join Date: Mar 2010
Old 08-16-2018 , 16:54   [Linux] Automated restart scripts sometimes messing up
Reply With Quote #1

So, I am experiencing a problem where occasionally my restart scripts aren't working correctly on one server. The normal process is this: Every morning (as scheduled in crontab), a script (start.sh) runs that will run a bunch of other scripts. These other scripts will connect to my running screens, kill the screen, and then start another server up with the correct properties.

However, I noticed today, for some reason sometimes instead of killing the screen, it will just make a whole new game server, and I'll have a bunch of servers running in a bunch of different screens with serial hostports.

I'm hoping a fresh set of eyes can help me identify the problem. I'm going to post each step in the process starting with crontab:

/etc/crontab
Code:
20 11 * * * gouser /home/gouser/start.sh
~/start.sh (only relevant pieces included)
Code:
#!/bin/bash

/home/gouser/go/surf/surf.sh
sleep 10
~/go/surf/surf.sh (only the relevant part)
Code:
#!/bin/bash

screen -X -S surf quit
sleep 4
cd /home/gouser/go/surf
taskset -c 6 /usr/bin/screen -dmS surf ./srcds_run -game csgo [etc]
My dedicated server box also reboots once per month.

Today, I noticed that my servers hadn't updated automatically this morning when the script should have run at 11:20 server time (which is like 5:20 my time) so I ran the script manually and noticed there were three screens running a surf server, and I had three surf servers running. (All of these screens were called "surf", so the script didn't know which one to connect to and kill.) They were being assigned new hostports as the current one was taken, and they were just there running unbeknownst to me.

Since the server machine last rebooted on the 1st of the month, and there's only 3 surf servers running (as opposed to 16) I can only assume this happening willy-nilly. I would like to help maybe figuring out why sometimes the script doesn't kill the screen and instead starts a new one up.
__________________
Boycott ESEA
My servers
ghostofmybrain is offline
Maxximou5
AlliedModders Donor
Join Date: Feb 2013
Old 08-16-2018 , 17:24   Re: [Linux] Automated restart scripts sometimes messing up
Reply With Quote #2

Perhaps you accidentally sent an Xoff signal, which terminates all input (which I believe blocks quit commands). In doing so, you won't quit the server and will make another server. Add a check for processes with the same name and kill it as a backup.
If it happens again, you could attach the session and see if any input is being allowed.

Curious, why not use something like LGSM?
Maxximou5 is offline
ghostofmybrain
Veteran Member
Join Date: Mar 2010
Old 08-16-2018 , 17:40   Re: [Linux] Automated restart scripts sometimes messing up
Reply With Quote #3

Quote:
Perhaps you accidentally sent an Xoff signal
I'm not sure how that could be happening. I don't really ever do things to the actual server. Is there anything you'd suggest that might be causing this to happen without my input?

Quote:
Curious, why not use something like LGSM?
Because I don't want to. I've been using these scripts for significantly longer than LGSM has been around, and I simply have no interest in using any other system when this one is already set up and (almost) flawless (if I could figure out this one thing).
__________________
Boycott ESEA
My servers
ghostofmybrain is offline
Maxximou5
AlliedModders Donor
Join Date: Feb 2013
Old 08-16-2018 , 17:54   Re: [Linux] Automated restart scripts sometimes messing up
Reply With Quote #4

Quote:
Originally Posted by ghostofmybrain View Post
I'm not sure how that could be happening. I don't really ever do things to the actual server. Is there anything you'd suggest that might be causing this to happen without my input?
I really can't say, I've only ever known due to bad commands being sent or user input.

Beyond killing the service itself, you could add a simple grep look for all names matching and quit/kill those servers in case this ever happens again. As for the root problem of how it happened, unless it happens again I haven't the faintest idea. Perhaps someone else has had this happen as I don't use screen much anymore.
Maxximou5 is offline
ghostofmybrain
Veteran Member
Join Date: Mar 2010
Old 08-27-2018 , 17:01   Re: [Linux] Automated restart scripts sometimes messing up
Reply With Quote #5

Quote:
Originally Posted by Maxximou5 View Post
you could add a simple grep look for all names matching and quit/kill those servers
Do you know how to do this? Would I add it to the startup script?

It's weird because if I try to just run the startup script when the extra screens are running, it will say "there are multiple screens with that name", so I am not even sure how it's getting to the next line and even launching the server?
__________________
Boycott ESEA
My servers
ghostofmybrain is offline
Maxximou5
AlliedModders Donor
Join Date: Feb 2013
Old 08-27-2018 , 21:26   Re: [Linux] Automated restart scripts sometimes messing up
Reply With Quote #6

If you can and do have full console logs, check for what happened when you sent the command quit. That's honestly the most important factor here, you can see if there was any character or input send before which would cause it to fail, or if you have a parameter set to simply restart the server on the quit command sent.

As for a loop, there's a ton of different ways, but without your full codes I can only suggest looking at examples and adding it however it fits your current setup. Any of the following should get you what you want.

Example 1
Example 2
Example 3
Example 3
Maxximou5 is offline
ghostofmybrain
Veteran Member
Join Date: Mar 2010
Old 08-28-2018 , 06:13   Re: [Linux] Automated restart scripts sometimes messing up
Reply With Quote #7

Those are my full codes. The only thing I took out was pieces of start.sh that started other servers, and things in my startup like my authkey

I'll turn logging on and see if i can notice anything next time it happens. Thank you.
__________________
Boycott ESEA
My servers

Last edited by ghostofmybrain; 08-28-2018 at 06:13.
ghostofmybrain is offline
Triniayo
Senior Member
Join Date: Apr 2011
Location: #include <germany>
Old 08-29-2018 , 03:50   Re: [Linux] Automated restart scripts sometimes messing up
Reply With Quote #8

You could try this script:

https://github.com/crazy-max/csgo-server-launcher

Really simple to set up, and works 100%.
__________________
If you need any help, feel free to add me on Steam.


Triniayo is offline
Send a message via Skype™ to Triniayo
ghostofmybrain
Veteran Member
Join Date: Mar 2010
Old 08-29-2018 , 06:05   Re: [Linux] Automated restart scripts sometimes messing up
Reply With Quote #9

No thanks, I'd rather figure out what is wrong with mine it's of just ignoring the problem.
__________________
Boycott ESEA
My servers
ghostofmybrain is offline
Ejziponken
AlliedModders Donor
Join Date: Apr 2008
Old 08-29-2018 , 13:54   Re: [Linux] Automated restart scripts sometimes messing up
Reply With Quote #10

or try this
https://forums.alliedmods.net/showthread.php?t=87291
Ejziponken is offline
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -4. The time now is 15:06.


Powered by vBulletin®
Copyright ©2000 - 2024, vBulletin Solutions, Inc.
Theme made by Freecode