[Silvers]

About:

• Demonstration and proof-of-concept to Read and Write .VPK files in SourceMod.
• Based on my plugin: Helms Deep Patch (VPK Maker).

Details:

• Supports VPK format version 1 and 2.
• Can read and write files synchronously and asynchronously.
• The MAX_PROCESS define is 0.3 seconds for asynchronous processing before waiting 0.1 seconds.
• Does not create archives.
• Does not write format version 2 footers.
• Use the test plugin examples as a guide for reading and writing.

• ToDo: Extract to temp file and rename on completion.
• ToDo: Tidy up some error handling (mostly for missing files).
• ToDo: Some natives may return true on error or wrong file count.
• ToDo: Remove some of the commented out debug code.
• ToDo: Convert some code to use methodmaps. - Done for v1.1 release.
• Should not leak handles but have not tested. Please report any issues.

Related Plugins:

• VScript File Replacer

Thanks:

• Dragokas - For requesting a method to Read VPK files.
• Lux - For suggestions and ideas.
• JoinedSenses - For sending me an example of version 2 format.

Admin Commands: (requires "z" flag)

PHP Code:

// From the "vpk_api_test" plugin.
// Tests all features of the plugin.
// Creates a folder named "vpk_api_test" in your servers main directory (where addons etc is located).
sm_vpk_api_test 


Example Output:

Spoiler

PHP Code:

-- VPK: Write Sync: create success: true. Packaged (6) files to "vpk_api_test/new_vpk_sync.vpk". 5137 bytes (5.01 KB). Took 0.002230 seconds to process.


Test File List: "scripts/kb_act.lst". 1740 bytes (1.69 KB)
Test File List: "gameinfo.txt". 1670 bytes (1.63 KB)
Test File List: "mapcycle.txt". 95 bytes (95.00 B)
Test File List: "maplist.txt". 1282 bytes (1.25 KB)
Test File List: "motd.txt". 38 bytes (38.00 B)
Test File List: "scripts/gameserverconfig.vdf". 91 bytes (91.00 B)
-- VPK: File List: (6) files in "vpk_api_test/new_vpk_sync.vpk". Took 0.005316 seconds to process


Test Write Async: "scripts/kb_act.lst". 1740 bytes (1.69 KB)
Test Write Async: "gameinfo.txt". 3410 bytes (3.33 KB)
Test Write Async: "mapcycle.txt". 3505 bytes (3.42 KB)
Test Write Async: "maplist.txt". 4787 bytes (4.67 KB)
Test Write Async: "motd.txt". 4825 bytes (4.71 KB)
Test Write Async: "scripts/gameserverconfig.vdf". 4916 bytes (4.80 KB)
-- VPK: Write Async: Packaged (6) files to "vpk_api_test/new_vpk_async.vpk". 5137 bytes (5.01 KB). Took 0.007080 seconds to process


Test Extracted Sync: List: "vpk_api_test/sync/maplist.txt". -1 bytes (-1.00 B)
Test Extracted Sync: List: "vpk_api_test/sync/scripts/game_sounds.txt". 6616 bytes (6.46 KB)
Test Extracted Sync: List: "vpk_api_test/sync/scripts/hudlayout.res". 20553 bytes (20.07 KB)
Test Extracted Sync: List: "vpk_api_test/sync/resource/modevents.res". 36328 bytes (35.47 KB)
-- VPK: Extracted Sync: (3) files from "pak01_dir.vpk" to "vpk_api_test/sync". 63588 bytes (62.09 KB). Took 0.178592 seconds to process.


Test Extracted Sync All: List: "vpk_api_test/all/scripts/kb_act.lst". 1740 bytes (1.69 KB)
Test Extracted Sync All: List: "vpk_api_test/all/gameinfo.txt". 1670 bytes (1.63 KB)
Test Extracted Sync All: List: "vpk_api_test/all/mapcycle.txt". 95 bytes (95.00 B)
Test Extracted Sync All: List: "vpk_api_test/all/maplist.txt". 1282 bytes (1.25 KB)
Test Extracted Sync All: List: "vpk_api_test/all/motd.txt". 38 bytes (38.00 B)
Test Extracted Sync All: List: "vpk_api_test/all/scripts/gameserverconfig.vdf". 91 bytes (91.00 B)
-- VPK: Extracted Sync All: (6) files from "vpk_api_test/new_vpk_sync.vpk" to "vpk_api_test/all". 4916 bytes (4.80 KB). Took 0.005634 seconds to process.


Test Extracted Async: "vpk_api_test/async/scripts/hudlayout.res". 20553 bytes (20.07 KB)
Test Extracted Async: "vpk_api_test/async/resource/modevents.res". 56881 bytes (55.54 KB)
Test Extracted Async: "vpk_api_test/async/scripts/game_sounds.txt". 63497 bytes (62.00 KB)
-- VPK: Extracted Async: (3) files from "pak01_dir.vpk" to "vpk_api_test/async". 3795195 bytes (3.61 MB). Took 0.185302 seconds to process


-- VPK Header: File: "pak01_dir.vpk". Signature: 55AA1234. Version: 1. Header: 3795183 [3795195] bytes (3.61 MB). Took 0.000130 seconds to process.


-- VPK: Showing random file: "models/props/cs_assault/vents.dx90.vtx"
-- VPK: "pak01_dir.vpk" has (22272) files. Took 0.089648 seconds to process. 


Natives for developers:

Spoiler

PHP Code:

// Callbacks for asynchronous packaging/extracting
typeset VPK_OnExtractedFiles
{
    /**
     * @brief Callback called whenever extracting files has finished.
     *
     * @param filename           The path of the VPK to open
     * @param destination        The folder path where extract the files
     * @param aSave              ArrayList handle containing a list of files extracted
     *
     * @mnoreturn
     */
    function void(const char filename[PLATFORM_MAX_PATH], const char destination[PLATFORM_MAX_PATH], ArrayList aSave);
}

typeset VPK_OnPackagedFiles
{
    /**
     * @brief Callback called whenever packaging files has finished.
     *
     * @param filename           The path of the VPK saving to
     * @param aSave              ArrayList handle containing a list of files packed
     *
     * @mnoreturn
     */
    function void(const char filename[PLATFORM_MAX_PATH], ArrayList aSave);
}



/**
 * @brief Reads the header from the specified VPK file
 *
 * @param filename               The path of the VPK to open
 * @param signature              The VPK signature bytes (e.g. "55AA1234")
 * @param version                The VPK version format (1 or 2)
 * @param fileSize               The size of the VPK archive or header including all PreLoad bytes
 *
 * @return                       true on success, false if file missing or other error
 */
native bool VPK_GetHeader(const char filename[PLATFORM_MAX_PATH], char signature[16] = "", int &version = 0, int &fileSize = 0);

/**
 * @brief Lists all the files packed into the specified VPK file
 *
 * @param filename               The path of the VPK to open
 * @param aList                  ArrayList handle returned containing file list
 *
 * @return                       -1 on failure (missing file or other error), number of files contained in VPK
 */
native int VPK_GetFileList(const char filename[PLATFORM_MAX_PATH], ArrayList aList);

/**
 * @brief Extracts the listed files from the specified filename to the destination folder
 *
 * @param filename               The path of the VPK to open
 * @param destination            The folder path where extract the files
 * @param aSave                  ArrayList handle containing a list of files to extract OR null to extract all (when null, list of extracted files is returned)
 * @param VPK_OnExtractedFiles   Callback for asynchronous extraction of files. With no argument specified or INVALID_FUNCTION the function will act synchronously
 *
 * @return                       true on success, false if file missing, unable to create extraction path or other error. Always true if specifying a VPK_OnExtractedFiles callback.
 */
native bool VPK_ExtractFiles(const char filename[PLATFORM_MAX_PATH], const char destination[PLATFORM_MAX_PATH], ArrayList &aSave, VPK_OnExtractedFiles callback = INVALID_FUNCTION);

/**
 * @brief Writes the listed files to the specified folder and filename
 *
 * @param filename               The path of the VPK to write to including the name
 * @param aSave                  ArrayList handle containing a list of files to pack
 * @param VPK_OnPackagedFiles    Callback for asynchronous packaging of files. With no argument specified or INVALID_FUNCTION the function will act synchronously
 * @param versionVPK             VPK version format to write. Must be 1 or 2.
 *                               Version 1 = Alien Swarm, Dota 2, Left 4 Dead, Left 4 Dead 2, Portal 2, Source Filmmaker.
 *                               Version 2 = Counter-Strike: Global Offensive, Counter-Strike: Source, Day of Defeat: Source, Half-Life: Source, Half-Life 2, Half-Life 2: Deathmatch, Portal, Team Fortress 2
 *
 * @return                       true on success, false if a file is missing, unable to create path or other error. Always true if specifying a VPK_OnPackagedFiles callback.
 */
native bool VPK_WriteFiles(const char filename[PLATFORM_MAX_PATH], ArrayList aSave, VPK_OnPackagedFiles callback = INVALID_FUNCTION, int versionVPK = 1); 


Changes:

1.2 (28-Jan-2024)
    - Fixed memory leak caused by clearing StringMap/ArrayList data instead of deleting.

1.1 (04-Dec-2021)
    - Changes to fix warnings when compiling on SourceMod 1.11.
    - Converted some code to use MethodMaps.

1.0 (26-Oct-2021)
    - Initial release.

Installation:

• Download the .zip and extract the files to their respective folders in your servers \addons\sourcemod\ folder.

[Dragokas]

Cool!
Highly anticipated feature to bypass valve_fs slow read.
Congratzzz!

[NoroHime]

is this way to solution write stringtable error? like this
https://github.com/ValveSoftware/Sou...es/issues/3451

[Dragokas]

NoroHime, it can also help to cook potatoes if they burn.

[alasfourom]

Does this mean it can read any vpk file that was installed in the server?

[Shao]

Originally Posted by NoroHime is this way to solution write stringtable error? like this https://github.com/ValveSoftware/Sou...es/issues/3451

Afaik this is because map makers don't stringtabledictionary and buildcubemaps before uploading their maps, to fix it you need to re-upload the proper .bsp files with said cvars done and those map makers get rather upset when you do so while they often claim that everything is fine. (P.S. It's not and they are often old uploads that they are too lazy to fix themselves in the end.)

So I don't see how this plugin could help it since players happen to have different maps anyway, then again I hardly understand how it's supposed to work.

[Dragokas]

alasfourom, you can read any vpk with SM OpenFile .use_valve_fs = true (if vpk is installed).

This plugin made by my request to bypass Valve slow FS read: Issue #1577

[Bacardi]

...can this plugin edit files inside game VPKs what are signatured ?
Counter-Strike: Source sv_pure system relies in these key signatures,
sv_pure not check files inside VPK, only loosed files.

[Silvers]

Originally Posted by Bacardi ...can this plugin edit files inside game VPKs what are signatured ? Counter-Strike: Source sv_pure system relies in these key signatures, sv_pure not check files inside VPK, only loosed files.

Assuming they are VPK archives, where you have: "pak01_dir.vpk", "pak01_001.vpk" etc then no. If their format is VPK2 the footer is written, not sure how much that is required for the game to read. Maybe one day I'll add support for VPK archives, or whatever they call that style of having multiple VPKs belonging to one, but since this was more of a proof-of-concept and pretty much unused I feel it's more effort and energy I could put elsewhere.

[Bacardi]

yes, cs:s own archives.

I was thinking, can some people edit materials parameters and still can keep VPK signature untouch,
then sv_pure system just let them use like normally.

...some years ago, I got cs:s VPK files in use, which files were edit inside VPK but it had right (Valve) signature.
This should not be possible unless someone got both "keys" to recompile cs:s VPK package.

I don't have those files anymore, those went when I formatted my PC every other year

*edit
I have misspell "key" as "signature". Sorry