Batch file / create bz2's in mass quantities

[SlimeCounty]

So, I was compressing tons of custom models, sounds, vmt's and vtf's (custom source materials, etc) this morning, and I mean TONS, doing each manually, getting them ready for my fastdl server. My buddy Squirrel pops into group chat, knows what I'm doing, and throws this batch file at me.

Works like a flipping charm. I thought I'd post it somewhere on these forums, as I feel this is something pretty damn useful for source server owners who compress/upload custom content in large quantities.

I'll attach the file below, but this file assumes 2 things...
7zip is installed in C:\Program Files\7-Zip\7z.exe
and that you want your files compressed to the .bz2 format.

HOW IT WORKS
First, you must have 7zip installed.
Place the .bat file into a directory containing all the files you wish to compress and drag those files into the .bat file, stand back, and watch as it compresses each file into it's own .bz2.

Here's the script, in case you need to change something to suit it for your needs.
Simply copy that into a new txt file, and name it whateveryouwant.bat.


@Echo Off
:Start
If [%1==[ Goto :EOF
PushD %~pd1
"C:\Program Files\7-Zip\7z.exe" u -tbzip2 %~nx1.bz2 %1
Shift
PopD
Goto Start

[ReFlexPoison]

Wow, Imma have to try this. :)

[Groger]

Does it crawls trough all the directories to? Cause you can allready drag & drop multiple files on the default bzip2.exe

[SlimeCounty]

Does it crawls trough all the directories to? Cause you can allready drag & drop multiple files on the default bzip2.exe

No, it only compresses what you drop in it. Only time I've had any experience with bz2 was with 7zip, and there's no bzip2.exe in my installation.

[Powerlord]

One thing I miss having moved to a Windows server from a Linux server is the ability to wget maps and things directly to my fast download server, and then be able to bzip2 -k *.bsp to convert them all to bz2.

Edit: -k was to keep the original bsp files so I could then use sftp to upload the files to my game server.

[SlimeCounty]

That does sound nice.

[checkster]

This will save some time for those who do not have a auto bzip on they're cite. Nice share bro.

[Fearts]

This is very nice indeed. What compression level does it use?


EDIT:

My friend DHack sent me this link:

http://www.brainless.us/downloads.aspx?cid=1

[natino]

can you use these compressed files by just putting them in the directory or on a webserver?

[Powerlord]

can you use these compressed files by just putting them in the directory or on a webserver?
I've heard that they will be used for downloads if you put them on the game server, but they're intended for a fast download webserver.

Side note: The game server itself doesn't understand bz2, so you must put the uncompressed maps on the game server itself.

[Aqua Dark]

Just use MinGW to do all files in a directory so say you have all your files in C:\ToBebzip2 it would find and bzip2 all the files you have in it and will keep the directory structure too.

To use this you'll need MinGW (here (https://sourceforge.net/projects/mingw/files/Installer/mingw-get-inst/)) and MSYS (here (http://sourceforge.net/projects/mingw/files/MSYS/Base/msys-core/msys-1.0.11/MSYS-1.0.11.exe/)) If you need help here's the setup and install guild http://www.mingw.org/wiki/InstallationHOWTOforMinGW When installing MinGW make sure you get the bzip2 lib.

Once installed open MSYS run this command
find 'C:\YOUR PATH' -type f -exec bzip2 -9 '{}' \;
So say your files/folders are in C:\bzip2 and you have the folders "models" and "maps" in it. Your path would be C:\bzip2 and it will find all the files and folders inside C:\bzip2 and .bz2 them for you.

[SlimeCounty]

That sounds slightly more complicated than simply dragging files into a batch folder which uses a prog many server owners already have, but hey, alternative methods are always helpful, thanks for sharing!

[Aqua Dark]

That sounds slightly more complicated than simply dragging files into a batch folder which uses a prog many server owners already have, but hey, alternative methods are always helpful, thanks for sharing!
For the most part just click nest on the installers. The hardest part would be clicking "Download latest repository catalogues" in the MinGW installer it's the only thing you click on other then next. For the cmd popup in the MSYS installer type "y" "y" "C:\MinGW" and your done.

I like this because if you got mass files to do it looks something like this (http://216.52.148.164/bz2list.txt) not just two folders with maybe 10 files where your .bat would be faster, but that's just me.

[cruzzin]

its not working for me cmd comes up then shuts down.

[Aqua Dark]

its not working for me cmd comes up then shuts down.
Did you type this into it before it closed "y" "y" "C:\MinGW"?

[SlimeCounty]

its not working for me cmd comes up then shuts down.

Are you talking about the batch file method or Aqua's preferred MinGW method?
If you're talking about the batch file I posted, keep in mind that you don't run the batch file itself, you drag the files you wish to compress into it, and of course you must have 7zip installed, which brings up another point I made in the first post, that you must check the location of your 7zip installation against what is there in the script of the batch file.

"C:\Program Files\7-Zip\7z.exe"

If that does not match your install location, change it accordingly.
Hope that helps.

[minimoney1]

I don't know why you guys make it so complicated. In Windows 7 (and Vista), you could move all the files you want to bz2 to a folder, then in the search bar (up top) put in "." (without the quotes) and it lists all files and folders. Then drag all of them into the bzip2 executable file and it will auto bz2 all of them.

[SlimeCounty]

I don't know why you guys make it so complicated.

Well, there is no bz2 executable with my particular installation of 7zip, but hell yeah, that does sound nice, clean and easy.

I WHOLEHEARTEDLY DISAGREE WITH SLIMECOUNTRY, THIS IS FAR EASIER, AND SOMETIMES IT'S JUST WHAT YOU NEED!

Awesome! Whatever works, maybe I'll give that a shot!

[Aqua Dark]

I don't know why you guys make it so complicated. In Windows 7 (and Vista), you could move all the files you want to bz2 to a folder, then in the search bar (up top) put in "." (without the quotes) and it lists all files and folders. Then drag all of them into the bzip2 executable file and it will auto bz2 all of them.
Yes that works but now your not using the best compression by doing that, using the .bat or the MSYS you can set the compression level so the files are smaller then just dropping them on a bzip2.exe. As you can see in my command line I have set -9 (or --best) to use the best compression for bzip2 v.s. drag and drop -1 (or --fast). I'm not sure if SlimeCounty as set it in his .bat or not, but it dose make a big differenced for people who have slower connection speeds even if it's only 5MB.

[invermere]

I couldn't get that first script to work, so I made a slight variation. Not sure if it's a Win7 thing (?) but after doing a little research on command line and 7z command line I had to change the output pathname to get the absolute path, like the input path needs to be. Oh an also specified the Ultra compression for the archive... not sure if that's on by default. Don't ask me why, but this works for me and if anyone else has issues with the OP try this.


@Echo Off
:Start
If [%1==[ Goto :EOF
PushD %~pd1
"C:\Program Files (x86)\7-Zip\7z.exe" u -tbzip2 -mx9 %1.bz2 %1
Shift
PopD
Goto Start


I would include the batch file, but it says invalid file type, so you can just make your own and put this in there. Simply drag all the files you want to compress on to the file and it will go to town.

[Groger]

You guys heard of sourceRCS yet? This automaticly detects if a file is added and compresses it and uploads it to your fast download. Works like a charm :)

http://www.brainless.us/download/8/SourceRSC

[invermere]

You guys heard of sourceRCS yet? This automaticly detects if a file is added and compresses it and uploads it to your fast download. Works like a charm :)

http://www.brainless.us/download/8/SourceRSC

They way I had mine set up it started downloading a bunch of stuff on to my local machine, which eats up my bandwidth and doesn't transfer all that fast, which makes sense because it needs to pull data from the GS to put on the FTP. I would ideally want it to check the GS, compile a list of whatever it needs to, then check the fastDL and compare the lists. I don't think this solution is ideal for shared hosting clients, since 3 computers are in the mix? Maybe I just have it misconfigured.

[Groger]

On my local box it compares the gameserver to the fastdownload and than only compresses and uploads the files it needs. But i have them on the same box so perhaps thats the difference yeah.

[masher]

They way I had mine set up it started downloading a bunch of stuff on to my local machine, which eats up my bandwidth and doesn't transfer all that fast, which makes sense because it needs to pull data from the GS to put on the FTP. I would ideally want it to check the GS, compile a list of whatever it needs to, then check the fastDL and compare the lists. I don't think this solution is ideal for shared hosting clients, since 3 computers are in the mix? Maybe I just have it misconfigured.

I'm the author of SourceRSC, and it should compare the game server files to the fastdl's before downloading anything from the game server. Maybe double check the paths. You can hit up our support forum if you'd like further help.

[deadmau54]

My god... this is god given, thank you SlimeCounty for this!

[Dragonthunder]

Thanks SlimeCounty, you just saved me a lot of time!

[dildoughy]

Just wanted to leave a message on this 4-year-old topic to say that I use this little thing every single day and it has saved my ass SO. MANY. TIMES. I can't imagine trying to individually archive hundreds of maps by hand. Thanks! :)

[Werewolf_UK]

How would I modify this to convert all the files in a specific folder? In my case I've made a folder on the desktop named 'BZ2 Compression Folder' (so it's path is "C:\Users\admin\Desktop\BZ2 Compression Folder") and I want it so I can dump files in and run the batch file (which also sits on the desktop)

The batch file would need to convert all the files in the named folder then delete the originals.

[airplanelover12]

Wow, Imma have to try this. :) in note pad do this:

:start
start
goto start

save that as lol.bat

[BustedRider]

This isnt realy easy to read for a french guys -_- which i wouldnt have to do each of them 1 by 1 :S

[YoloMySolo]

This isnt realy easy to read for a french guys -_- which i wouldnt have to do each of them 1 by 1 :S

You would just have to download 7Zip ( http://www.7-zip.org/download.html ) and the file provided is the batch file needed. After you have both of those, select all the files you want compressed and drag them into the batch file.

[BustedRider]

Lmao i tought u need to launch batch lol

[BustedRider]

Worked perfectly thx

[Elitcky]

I'm really sorry for this bump admins. But im so sure than people who is making sv's on 2017 didn't know about this like me.

All 2016 to current Date and i didn't knew about this thread, Tons of files which i had to compress 1 by 1 and now i see this... Omg. What a Helpful thread.


PD: don't warn me for bump thread :(( It's just an amazing thread bros. :c i'm so excited than i found this thing xD

[fragnichtnach]

[..] and drag those files into the .bat file, stand back, and [..]

I should read carefully to the end. Missed that part and was searching 15 minutes for a solution.

Thanks for that awesome script!

[eyal282]

Can you add something that allows me to drag a folder and it will do everything needed?

Also I noticed this file didn't get processed, maybe a name thing?

Jack--o-Lantern Happy Top.vmt

[CryWolf]

if youre on Linux is quite simple

apt install bzip2
cd ../moddir/maps/
bzip2 -k *
on windows
http://gnuwin32.sourceforge.net/packages/bzip2.htm
drag all the files over .exe and it will compress without deleting them.

[Tim q]

Somebody knows what should i type to delete files after converse?

[eyal282]

https://forums.alliedmods.net/showthread.php?t=341229