systemd configuration for TF2 servers and maintenance scripts

avi9526 · **Author**

It's not a question. Just decided to share my current systemd config. Not tested much, but seems working. Maybe will be useful for you, maybe not. Use at own risk.

I have 10 TF2 servers instances that is partially symlinked (only static files,… at least I think so) from one installation (6.7G for installation and <20M for each game folder). hlstatsX:CE present as well. My servers installed in /opt/valve folder, and hlstatsX:CE in /opt/hlstatsx folder - edit paths to match your system.

So here is my systemd configs.

1. Server instance startup script (only for one server, they all similar, only map names and numbers is different)

/etc/systemd/system/tf2-server-4.service

Code:

[Unit]

Description=Team Fortress 2 Server 4
After=network.target
Wants=hlstatsx.service
RequiresMountsFor=/opt

[Service]

Type=forking

User=steam
Group=steam

WorkingDirectory=/opt/valve/tf2-server-4/
Environment=

ExecStart=/usr/bin/tmux  -S ./tmux.sock new-session -d -n "TF2" -s "srcds" "./srcds_run -game tf  -debug -tvdisable +maxplayers 32 +ip 0.0.0.0 -port 27018 +clientport  27008 +tv_port 27028 +map mvm_manndarin_final"

ExecStop=/usr/bin/tmux -S ./tmux.sock send -t "srcds" "exit" ENTER

KillSignal=SIGCONT

TimeoutStartSec=0
TimeoutStopSec=5

Restart=on-failure
RestartSec=10

# Security
PrivateTmp=true
PrivateDevices=false
MemoryLimit=750M
DevicePolicy=closed
ProtectSystem=full
ProtectHome=true

[Install]

WantedBy=default.target

Notice "Wants=hlstatsx.service" line - it autostart hlstatsx service when TF2 server starting. You must have installed "tmux" (available in most linux distros). Connect to server console using command:

Code:

sudo -u steam tmux -S /opt/valve/tf2-server-4/tmux.sock attach-session

(don't forget to change path)

Then run from root

Code:

systemctl daemon-reload

As I said above - other servers configs are almost the same, so not posting it.

2. BASh scripts to start and stop all servers.

/opt/valve/maintenance/tf2_start

Code:

#!/bin/bash
if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root"
   exit 1
fi

systemctl start tf2-server-1 &
systemctl start tf2-server-2 &
systemctl start tf2-server-3 &
systemctl start tf2-server-4 &
systemctl start tf2-server-5 &
systemctl start tf2-server-6 &
systemctl start tf2-server-7 &
systemctl start tf2-server-8 &
systemctl start tf2-server-9 &
systemctl start tf2-server-10 &

wait

/opt/valve/maintenance/tf2_stop

Code:

#!/bin/bash
if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root"
   exit 1
fi

systemctl stop tf2-server-1 &
systemctl stop tf2-server-2 &
systemctl stop tf2-server-3 &
systemctl stop tf2-server-4 &
systemctl stop tf2-server-5 &
systemctl stop tf2-server-6 &
systemctl stop tf2-server-7 &
systemctl stop tf2-server-8 &
systemctl stop tf2-server-9 &
systemctl stop tf2-server-10 &

wait

"&" - parallel execution. "wait" - wait till all parallel tasks finished before exit script (required).

3. BASh script to update original installation and check instances.

Code:

#!/bin/bash
if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root"
   exit 1
fi

/opt/valve/maintenance/tf2_stop

wait

sudo -u steam bash -c "cd /opt/valve/steamcmd; ./steamcmd.sh +runscript tf2-server-common_update.cmd"

for i in {1..10}; do cp -ax /opt/valve/tf2-server-common/core /opt/valve/tf2-server-$i/; done
echo "----------------------------------------------------------------------"
diff -r /opt/valve/tf2-server-common /opt/valve/tf2-server-1 | grep common
echo "----------------------------------------------------------------------"

"diff -r …" command shown differences between symlinked instance and original installation. After major changes in updates new files may appear and manual action must be taken - payoff for using symlinked server instances.

4. BASh scripts to compress old logs and clear old temp data and log archives

You need to install lbzip2 - multi-threaded version of bzip2

Text files that stores locations of log files that must be compressed/cleared:

/opt/valve/maintenance/tf2_logs-rotation.list

Code:

/opt/valve/tf2-server-1/tf/logs/
/opt/valve/tf2-server-2/tf/logs/
/opt/valve/tf2-server-3/tf/logs/
/opt/valve/tf2-server-4/tf/logs/
/opt/valve/tf2-server-5/tf/logs/
/opt/valve/tf2-server-6/tf/logs/
/opt/valve/tf2-server-7/tf/logs/
/opt/valve/tf2-server-8/tf/logs/
/opt/valve/tf2-server-9/tf/logs/
/opt/valve/tf2-server-10/tf/logs/

Log cleanup

/opt/valve/maintenance/tf2_logs-rotation

Code:

#!/bin/bash
echo "Start cleaning logs - $(date '+%Y-%m-%d - %H:%M:%S')"
#--------------------------------------------------------------------------------------------------
List="/opt/valve/maintenance/tf2_logs-rotation.list"
# Log regex
LogRegex=".*\.log\$"
# Days pass from last modification (see.: man find, mtime)
Old="+1"
# Command to compress old logs (lbzip2 is multicore version of bzip2)
Archive="lbzip2 -z"
# Archive filename extension"
ArchiveRegex=".*\.log\.bz2\$"
# Days pass from archive modification (see.: man find, mtime)
ArchiveOld="+7"
#--------------------------------------------------------------------------------------------------
if [[ ! -f "$List" ]];
then
    echo "File list \"$List\" not found"
    exit 1
fi

while read i; do
    echo -n "Removing archives in \"$i\" older that $ArchiveOld days ... "
    if [[ -d "$i" ]];
    then
        find $i -maxdepth 2 -mtime $ArchiveOld -type f -regextype posix-extended -iregex "$ArchiveRegex" -exec rm '{}' \;
        echo "Done"
    else
        echo "Fail - Folder does not exist"
    fi
    echo -n "Compressing logs in  \"$i\" older than $Old days ... "
    if [[ -d "$i" ]];
    then
        find $i -maxdepth 2 -mtime $Old -type f -regextype posix-extended -iregex "$LogRegex" -exec $Archive '{}' \;
        echo "Done"
    else
        echo "Fail - Folder does not exist"
    fi
done < "$List"
#--------------------------------------------------------------------------------------------------
echo "Done cleaning logs - $(date '+%Y-%m-%d - %H:%M:%S')"
echo

List of replay folders to be cleared

/opt/valve/maintenance/tf2_replays-clear.list

Code:

/opt/valve/tf2-server-1/tf/replay/
/opt/valve/tf2-server-2/tf/replay/
/opt/valve/tf2-server-3/tf/replay/
/opt/valve/tf2-server-4/tf/replay/
/opt/valve/tf2-server-5/tf/replay/
/opt/valve/tf2-server-6/tf/replay/
/opt/valve/tf2-server-7/tf/replay/
/opt/valve/tf2-server-8/tf/replay/
/opt/valve/tf2-server-9/tf/replay/
/opt/valve/tf2-server-10/tf/replay/
/var/www/downloads/tf2/replays/tf2-server-1/
/var/www/downloads/tf2/replays/tf2-server-2/
/var/www/downloads/tf2/replays/tf2-server-3/
/var/www/downloads/tf2/replays/tf2-server-4/
/var/www/downloads/tf2/replays/tf2-server-5/
/var/www/downloads/tf2/replays/tf2-server-6/
/var/www/downloads/tf2/replays/tf2-server-7/
/var/www/downloads/tf2/replays/tf2-server-8/
/var/www/downloads/tf2/replays/tf2-server-9/
/var/www/downloads/tf2/replays/tf2-server-10/

Replays cleanup script

/opt/valve/maintenance/tf2_replays-clear

Code:

#!/bin/bash
echo "Start replay cleaning - $(date '+%Y-%m-%d - %H:%M:%S')"
#--------------------------------------------------------------------------------------------------
List="/opt/valve/maintenance/tf2_replays-clear.list"

if [[ ! -f "$List" ]];
then
    echo "File list \"$List\" not found"
    exit 1
fi

while read i; do
    echo -n "Cleaning folder \"$i\" ... "
    if [[ -d "$i" ]];
    then
        find $i -maxdepth 3 -mtime +7 -type f -exec rm '{}' \;
        echo "Done"
    else
        echo "Fail - Folder does not exist"
    fi
done < "$List"
#--------------------------------------------------------------------------------------------------
echo "Done replay cleaning - $(date '+%Y-%m-%d - %H:%M:%S')"
echo

List of temp folders

/opt/valve/maintenance/tf2_temp-clear.list

Code:

/opt/valve/tf2-server-1/tf/download/user_custom/
/opt/valve/tf2-server-2/tf/download/user_custom/
/opt/valve/tf2-server-3/tf/download/user_custom/
/opt/valve/tf2-server-4/tf/download/user_custom/
/opt/valve/tf2-server-5/tf/download/user_custom/
/opt/valve/tf2-server-6/tf/download/user_custom/
/opt/valve/tf2-server-7/tf/download/user_custom/
/opt/valve/tf2-server-8/tf/download/user_custom/
/opt/valve/tf2-server-9/tf/download/user_custom/
/opt/valve/tf2-server-10/tf/download/user_custom/

Temp folder cleanup script

/opt/valve/maintenance/tf2_temp-clear

Code:

#!/bin/bash
echo "Start cleaning temporary files - $(date '+%Y-%m-%d - %H:%M:%S')"
#--------------------------------------------------------------------------------------------------
List="/opt/valve/maintenance/tf2_temp-clear.list"

if [[ ! -f "$List" ]];
then
    echo "File list \"$List\" not found"
    exit 1
fi

while read i; do
    echo -n "Cleaning folder \"$i\" ... "
    if [[ -d "$i" ]];
    then
        find $i -maxdepth 3 -mtime +7 -type f -exec rm '{}' \;
        echo "Done"
    else
        echo "Fail - Folder does not exist"
    fi
done < "$List"
#--------------------------------------------------------------------------------------------------
echo "Done cleaning temporary files - $(date '+%Y-%m-%d - %H:%M:%S')"
echo

One cleanup script to execute all above

/opt/valve/maintenance/tf2_cleanup

Code:

#!/bin/bash

./tf2_logs-rotation
./tf2_replays-clear
./tf2_temp-clear

5. systemd service and timer files for cleanup script

/etc/systemd/system/tf2-cleanup.service

Code:

[Unit]

Description=Cleanup and rotate TF2 servers logs and other temporary files
RequiresMountsFor=/opt
After=local-fs.target time-sync.target

[Service]

Type=oneshot

User=steam
Group=steam

WorkingDirectory=/opt/valve/maintenance/

ExecStart=/opt/valve/maintenance/tf2_cleanup
IOSchedulingClass=idle
Nice=10

# Security
PrivateTmp=true
PrivateDevices=false
MemoryLimit=100M
DevicePolicy=closed
ProtectSystem=full
ProtectHome=true

[Install]

WantedBy=default.target

/etc/systemd/system/tf2-cleanup.timer

Code:

[Unit]

Description=Cleanup and rotate TF2 servers logs and other temporary files

[Timer]

#OnBootSec=20min
OnUnitActiveSec=1d

[Install]

WantedBy=timers.target

Then run

Code:

systemctl daemon-reload
systemctl enable tf2-cleanup.timer
systemctl start tf2-cleanup.timer

6. hlstatsX:CE service startup config

/etc/systemd/system/hlstatsx.service

Code:

[Unit]

Description=hlstatsX:CE
After=network.target
RequiresMountsFor=/opt
StopWhenUnneeded=yes

[Service]

Type=forking

User=hlstatsx
Group=hlstatsx

WorkingDirectory=/opt/hlstatsx/
Environment=

ExecStart=/opt/hlstatsx/run_hlstats start

SuccessExitStatus=1

ExecStop=/opt/hlstatsx/run_hlstats stop

TimeoutStartSec=0
TimeoutStopSec=5

Restart=on-failure
RestartSec=10

# Security
PrivateTmp=true
PrivateDevices=true
MemoryLimit=100M
CPUQuota=10%
DevicePolicy=closed
ProtectSystem=full
ProtectHome=true

[Install]

WantedBy=default.target

Notice the line "StopWhenUnneeded=yes" its will stop hlstatsx service after all services that is wanting it was stopped.

The run

Code:

systemctl daemon-reload

7. hlstatsX:CE logs rotate BASh script, service and timer configs

You need to install lbzip2 - multi-threaded version of bzip2

/opt/hlstatsx/maintenance/log-rotation

Code:

#!/bin/bash

echo "Start cleaning logs - $(date '+%Y-%m-%d - %H:%M:%S')"
#--------------------------------------------------------------------------------------------------
# Path where logs are stored
LogPath="/opt/hlstatsx/logs/"
# Log regex
LogRegex=".*hlstats.*-[0-9]{2}\$"
# Days pass from last modification (see.: man find, mtime)
Old="+1"
# Command to compress old logs (lbzip2 is multicore version of bzip2)
Archive="lbzip2 -z"
# Archive filename extension"
ArchiveRegex=".*hlstats.*\.bz2\$"
# Days pass from archive modification (see.: man find, mtime)
ArchiveOld="+30"
#--------------------------------------------------------------------------------------------------
for i in $LogPath; do
    echo "Processing path: $i"
    echo "Compressing logs older than: $Old days"
    find $i -maxdepth 1 -mtime $Old -type f -regextype posix-extended -iregex "$LogRegex" -exec $Archive '{}' \;
    echo "Done"
done

for i in $LogPath; do
    echo "Processing path: $i"
    echo "Removing archives older that: $ArchiveOld days"
    find $i -maxdepth 1 -mtime $ArchiveOld -type f -regextype posix-extended -iregex "$ArchiveRegex" -exec rm '{}' \;
    echo "Done"
done
#--------------------------------------------------------------------------------------------------
echo "Done cleaning logs - $(date '+%Y-%m-%d - %H:%M:%S')"
echo

systemd service file for log rotation

/etc/systemd/system/hlstatsx-cleanup.service

Code:

[Unit]

Description=Cleanup and rotate hlstatsX:CE logs
RequiresMountsFor=/opt
After=local-fs.target time-sync.target

[Service]

Type=oneshot

User=hlstatsx
Group=hlstatsx

WorkingDirectory=/opt/hlstatsx/maintenance/

ExecStart=/opt/hlstatsx/maintenance/log-rotation
IOSchedulingClass=idle
Nice=10

# Security
PrivateTmp=true
PrivateDevices=false
MemoryLimit=100M
DevicePolicy=closed
ProtectSystem=full
ProtectHome=true

[Install]

WantedBy=default.target

systemd timer for log rotation

/etc/systemd/system/hlstatsx-cleanup.timer

Code:

[Unit]

Description=Timer to cleanup and rotate hlstatsX:CE logs

[Timer]

#OnBootSec=20min
OnUnitActiveSec=1d

[Install]

WantedBy=timers.target

The run

Code:

systemctl daemon-reload
systemctl enable hlstatsx-cleanup.timer
systemctl start hlstatsx-cleanup.timer

8. hlstatsX:CE awards calculation script required to run every 15min by hlstatsX:CE manual. Service and timer configs to do so

/etc/systemd/system/hlstatsx-awards.service

Code:

[Unit]

Description=hlstatsX:CE Awards
After=network.target
RequiresMountsFor=/opt
After=local-fs.target 

[Service]

Type=oneshot

User=hlstatsx
Group=hlstatsx

WorkingDirectory=/opt/hlstatsx/

ExecStart=/opt/hlstatsx/hlstats-awards.pl
IOSchedulingClass=idle
Nice=10

# Security
PrivateTmp=true
PrivateDevices=true
MemoryLimit=100M
DevicePolicy=closed
ProtectSystem=full
ProtectHome=true

[Install]

WantedBy=default.target

/etc/systemd/system/hlstatsx-awards.timer

Code:

[Unit]

Description=hlstatsX:CE Awards Timer

[Timer]

#OnBootSec=20min
OnUnitActiveSec=15m

[Install]

WantedBy=timers.target

Then run

Code:

systemctl daemon-reload
systemctl enable hlstatsx-awards.timer
systemctl start hlstatsx-awards.timer

Hope I didn't miss anything. If you ask I can make post about how I made 10 TF2 servers from one installation. But I think there is enough good (with less crashes per day) examples in internet.