AlliedModders - TOGs SourcePawn Syntax Converter [MS Excel Based for now]

AlliedModders (https://forums.alliedmods.net/index.php)

- Snippets and Tutorials (https://forums.alliedmods.net/forumdisplay.php?f=112)

- - TOGs SourcePawn Syntax Converter [MS Excel Based for now] (https://forums.alliedmods.net/showthread.php?t=290805)

TOGs SourcePawn Syntax Converter [MS Excel Based for now]

I wasnt sure which forum section to post this in, but since there are syntax highlighters and other indirectly related stuff to SP coding, I thought this the best fit.

I created some code in VBA in a Microsoft Excel environment to convert SourcePawn plugins from old syntax into new syntax. While the project is not yet finished, I thought I would release it for others to use, and for feedback (provided I have time to implement much feedback). It REALLY should be ported over to another language so that MS Excel isnt required. Despite all that, here it is. Perhaps in the future, myself or another will port my code to PHP and make a web version...

Also, it isnt perfect (and probably never will be). Here is the identified TO-DO's, should I find the time for them...the point is that the tool gets you most of the way there, taking a lot of the work out of it (if not all).

To Do and Stuff Not Yet Handled:

Find expressions or variables that are coerced to data types and replace with view_as<type>() [Not sure how to best detect this without false positives at this time...]
~~Break apart lines where variables of multiple types are declared.~~ ~~Done.~~ Doesnt always work - need to revise.
~~Need to add dynamic size detection, similar to what is already done for char, for other variable types.~~ Done.
May need to build up list of SM forwards again and push to 2D array with type to add them as (typically void) for checking if functions should be ints, check the type for fwds. They are only a problem when cross referenced in a recursive include, and added to the runtime list that makes them ints (typical for custom forwards from plugins, so it seems). Done.
~~Inclusions of includes - would need a defined path to include folder - DONT CONVERT INCLUDE, but use it to get defines, enums and forwards~~ Done.
Conversion of strings created with sizes set on run time (this would require pushing all defines to an array, including defines in includes, and checking if the size inside a bracket is a define or a variable - variable being dynamic) Done.
~~Add detection of multiple declarations of the same variable type (other than int), similar to what is already done for char, but for other types.~~ Done.
Detection of functions that dont fit the several other lists but return something other than void, detectable via the return lines. I'm thinking of having it process entire functions at a time, allowing it to detect the text "return ....;" and adjust the type accordingly, and find the variable declare, if local, and its type. To support global variable types, those would need to be pushed to an array, since they are done processing. Done.
Processing of includes that are already in new syntax. Currently, it processes them, but might not work correctly on identifying some return type of natives, etc.
Some combinations of formatting can throw tool off, such as line breaks inserted in the middle of a single code statement, or possibly replacing some spaces with tabs.

Options that would be nice but aren't required to compile:

Replacing code with new SP 1.8 classes

Tasks to improve efficiency:

Remove function types that don't exist in SP 1.6- (1.8 classes)
General optimization of code

Things handled by this program:

Conversion of variable declarations ("String:" --> "char ", "bool:" --> "bool", etc.).
Adding "int" in front of variable declarations that are not specified.
Adding void in front of function header parameters that don't have a type declared. This has overrides, noted below. Handling of optional parameters is now supported.
Detection of code that is in comments, including multi-line comments (/* */), even if mid-line of code. Anything inside comments or quotes is not processed.
Storing lists of function types, numeric defines (for processing of dynamic variable size declarations), enums (to identify variable "types"), forwards and their function types (for processing what their callback function types should be), and natives (same idea as forwards). These are also detected within includes.
Recursive reading of includes (and includes stated inside includes), both with default path (name inside <>), or string paths, and with exception to the <sourcemod> include. The sourcemod include is in most plugins, it's contents are already pre-added to lists, and this cuts down on processing time. In the future, this may include an option to also convert the includes (disabled by default), but not at this time. The processing looks for numeric defines, forwards, natives, and enums.
Skips most pre-compilation processing statements (starting with #), except for processing defines and includes.
Switching of where brackets go for ~~char's~~ any variables in function headers (i.e. after char, not after variable name). Variables with multiple brackets (dimensions) are now supported.
Replacement/removal of some depreciated functions or constants (FCVAR_PLUGIN, FindSendPropOffs, GetClientAuthString, etc.)
Menu detection for function header type declaration as int.
Removal of multiple of the same type declarations in a single line (i.e. char sSomeString[1], char sAnotherString[1]; --> char sSomeString[1], sAnotherString[1]; ). This applies to all variable types now.
Processing of entire functions to check return types in order to override the default "void" that would be added when not specified. Code detects type returned, and if it is a local or global variable, it looks it up to find that variables type and sets the function type accordingly. This does NOT process returning functions that return values at this time.
Splits declarations of multiple different variable types into as many separate lines as there are types. Note: This piece DOES NOT work if any of the multiple variables being declared set defaults in that line (e.g. iSomeInt = 1, Float:sSomeChar = 0.0;).
Converting of variable declarations with dynamic sizes to the new format.

Usage/Instructions:

Note:The changelog is available with the spreadsheet in the "changelog" sheet.

I hope this is helpful! To see the code in Excel, press Alt+F11, or right click the tab and click "View Code". This tool will NOT edit the source file, but will create a new file with the same name then " newsyntax" appended. If a file already exists with that name, it will get a number appended in addition to that.

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

You could also use Visual Basic Script *.vbs as a single file. Works for all Windows versions and should work for Linux with Wine aswell, i think (i saw MS Scripting Control in winetricks).

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by Timocop (Post 2472270)

You could also use Visual Basic Script *.vbs as a single file. Works for all Windows versions and should work for Linux with Wine aswell, i think (i saw MS Scripting Control in winetricks).

I actually started the conversion to vbs and havent finished it because of some of the differences. Ideally, I'd rather just port the logic to php and make a web page for people to use, but we'll see.

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

convert back to old syntax would be useful

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by friagram (Post 2472328)

convert back to old syntax would be useful

lol...but why would someone want to do that?! I'm sure there are times it would be useful though...

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

what the fuck vba :psyduck:

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by Miu (Post 2472393)

what the fuck vba :psyduck:

^This. lol. I already had a lot of code available from past projects I've done at work, so after a few drinks one night...

Anyways, when I finish the project, I'll probably make a PHP page to do the same. For now though, the development is easier in VBA, specifically for the includes processing, which I'm working on now. I dont know if you can easily follow a recursive include model from PHP without the user uploading all of those files as well...

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Updated code and OP description.

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Updated to v1.1.1.

Here's the changelog (also viewable from the tool as another sheet).

Spoiler

Code:

Date                Version        Description of Changes

11/24/16        1.1.0        Started changelog.

11/25/16        1.1.1        Separated out functions by type into separate code modules.

11/25/16        1.1.1        Replace AlreadyIncluded function to use IsStringInArray, since they did the same thing.

11/25/16        1.1.1        Added version number to main sheet and whited out backgrounds to remove gridlines.

11/25/16        1.1.1        Converted ga_sFwdNames to be a 2D array, storing names of forwards and their types. Converted function for determining if SM function type should be int to be a generic function to determine type. Created functions for processing 2D arrays other than the previously create int ones for ga_iIgnoreBounds and ga_iCommentsOnly.

11/25/16        1.1.1        Fixed UpdateBracketCount being able to not have the correct locations for ignore sections by calling GetIgnoreSections just before it.

11/25/16        1.1.1        Updated engine to process entire functions before writing to output. This will allow additional processing and then editing of the lines.

11/25/16        1.1.1        Replaced Trim() functions in code with Trim2() funciton I created that also removes tabs (I thought Trim() did this, but it appears not to).

11/25/16        1.1.1        Pushed global variables + types to array for processing function types.

11/25/16        1.1.1        Miscellaneous renaming of variables, most notably ga_sFunctionNames to ga_sTypes.

11/25/16        1.1.1        Added overriding of function types in headers using detection of return types - returning functions not currently supported for overrides.

11/25/16        1.1.1        Edited includes forward names detection to detect types and push to what is now a 2D array with function name. Returns int if type not specified.

11/25/16        1.1.1        Added handling for multiple float declarations on a single line.

11/25/16        1.1.1        Fixed multiline comments not always processing correctly when the end is on a separate line.

11/25/16        1.1.1        Edited MakeReplacement to pass the ignore array so that it has the ability to be used for replacements without using the global arrays, and using local ones. Currently, this added functionality it not being used though.

11/25/16        1.1.1        Fixed "const" being counted as a type in header parameters. The code previously just checked if there was only 1 word in the comma splits string sections. Now, it checks for 1, and if not 1, checks if 2 and first is "const". I cant think of additional key words off the top of my head, but there may be more to be added.

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Looks nice but please don't even convert it to php, convert it to javascript otherwise we'll see bunch of pirated plugin saved in databases..

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by TheWho (Post 2472919)

Looks nice but please don't even convert it to php, convert it to javascript otherwise we'll see bunch of pirated plugin saved in databases..

Good point - it can be modified to store the script, whereas js you can see the code. Be my guest when it comes to converting it though.

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Updated to v1.1.2. Changelog is in file. Added a lot of fixes to special circumstances and finished quite a few things on the to-do list.

P.S.: I meant to leave the old version up for reference, but accidentally clicked the remove button when uploading this version....Why is there no cancel!?!?

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Might as well make it a cross compiler, too. That way you can convert and compile in one step. That'd be cool. Any update on it being ported to PHP?

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by Headline (Post 2475698)

Might as well make it a cross compiler, too. That way you can convert and compile in one step. That'd be cool. Any update on it being ported to PHP?

Until the tool can handle everything, it wont be able to compile...as far as porting, nothing planned at this time....or the next 3 months for that matter (I have a 6 hr test I am studying for mid-March).

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Updated to version 1.1.4. Change log is in file. The main changes regard comments being processed correctly (in a couple situations, they could be processed as code previously).

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Welp, can't use this because Excel Starter doesn't allow running macros >:l

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by abrandnewday (Post 2495245)

Welp, can't use this because Excel Starter doesn't allow running macros >:l

Lol.

Fuck excell.

OP should copy paste to vba

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

You can make Notepad++ plugins too. Notepad++ can also run scripts though I'm not sure what language the scripts use.

Not sure why something couldn't be made in the format of one of those two options.

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

A php site would be nice

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Yes, a PHP site would indeed be wonderful. Upload source code, convert. Problem is, someone could set the site up to harvest any source code entered into it which would mean bad things for private plugins that are entered in.

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by abrandnewday (Post 2495279)

I mean, online compilers probably do that anyway lol
Just keep the php site open source, and no problem

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Or use Javascript instead.

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by Headline (Post 2495448)

I mean, online compilers probably do that anyway lol
Just keep the php site open source, and no problem

S'why I don't use online compilers. I keep a "clean" (unmodified) SourceMod install in a folder with new syntax-updated versions of many includes from these forums (updated by me, added for compatibility so I can compile things with custom includes) that I use for compiling my own plugins. Drag and drop the .sp file on compile.exe for a "test" compile so I can check for errors (because compile.exe's console window stays up). If I know something is fine, I'll drop it onto spcomp.exe for a speedy compile with no console window just to generate the .smx file in the same folder (another nice benefit, doesn't put them in /scripting/compiled).

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by abrandnewday (Post 2495462)

You could instead use Notepad++ to compile for you (since you already mentioned you use it). Install the NPPExec addon from the plugin manager. Follow instructions here: https://forums.alliedmods.net/showpo...&postcount=164

With NPP doing the compiling, it will stay in the NPP console and you can double click the error/warning to go straight to the line it came from. Then, map the macro to a shortcut key. So, when all is said and done, I press ctrl+shift+b to compile a script that is anywhere and the .smx will appear next to it in the same folder. Note: The includes will need to be in the include folder relative to your compiler path.

Currently, I run this script to compile:

Spoiler

I used to run a simpler script but found that on windows, pseudo mapped locations (through vsubst) seemed to add an extra slash, where it didnt for full paths. So, hence the checking for the slash at the end before adding it.

-----------------------------------------------

As far as converting the whole thing to PHP, I was planning on doing it some time, but am too busy until around April, so it likely wouldnt happen until then. As far as exporting the code for you:

Spoiler

When April comes around, I'll check back in on this, unless someone else beats me to it. Perhaps we can get a github repo going with a PHP or java version created.

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by ThatOneGuy (Post 2495578)

Spoiler

So do I except I modified mine ages ago and shrunk it down after removing useless shit. Should've noted that drag & drop was my OLD method until I started using that script about....fuck I forget when, it's been a while.

Spoiler

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by abrandnewday (Post 2495606)

Yup...that is what mine looked like until recently when my vsubst drives were causing the path to have a double / near the end if in root mapped folder of the vsubst "drive". For 98% of users out there, the shorter version is all that is needed.

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

why not just a transcompiler written in Python?

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by nergal (Post 2520376)

why not just a transcompiler written in Python?

Firstly because I didnt know VBA -> Python translators exist (though I just looked and there is one). Secondly, because I have only learned basic python and have issues getting the proper tools set up to even utilize python. Also, then I assume you couldnt make a web based program for converting? Good idea though...would you be willing to take that on? I found this one here: http://vb2py.sourceforge.net/

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by ThatOneGuy (Post 2520379)

I meant a python transcompiler that converts old sp to new syntax.

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by nergal (Post 2520403)

I meant a python transcompiler that converts old sp to new syntax.

Well, if such a thing is out there, it would be nice to be made available to others. If you're proposing I develop it, again, I dont know python (yet), and would encourage you to develop it if that is your idea. I'm still leaning towards php with the code made open source, as then it can be utilized on the web by anyone, anywhere, with no need to install anything to run the tool.

Side Note: Perhaps someone can get me started with the right setup for coding/compiling scripts in Python. I've installed python 3.5 in the past and read several tutorials to learn the language....seems pretty straight forward....but I couldnt figure out what I was doing in the software and wasnt committed enough to spend forever looking for tutorials on how to compile, etc. Every tutorial I did find looked nothing like what I was looking at on my screen or had options that I was missing, etc. So, I dropped it for the time.

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by ThatOneGuy (Post 2520431)

Python isn't compiled, but rather interpreted. I use Pycharm as my IDE and it works out of the box well

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by Headline (Post 2520667)

Python isn't compiled, but rather interpreted. I use Pycharm as my IDE and it works out of the box well

Nice. I've tried visual studio and the default install from python.org, and ran into the roadblocks mentioned (and didnt care enough at the time to work too hard to resolve the issues). I'll look into pycharm though!

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by T1MOXA (Post 2534433)

Error when converting https://forums.alliedmods.net/showthread.php?t=71867

Spoiler

Thanks for the feedback! I just patched this for you. Try the new version (1.1.6) in the OP.

Note: The processor doesnt handle splitting lines into multiple when multiple variable types are declared and the default values are set in the same line. This plugin has an instance of that. After fixing that, some view_as<>() stuff, and the AskPluginToLoad2 function, this is what I got.

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

How in the world do you even get this too work?

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

@ThatOneGuy :3 Thanks!

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]

Quote:

Originally Posted by T1MOXA (Post 2534847)

@ThatOneGuy :3 Thanks!

Your welcome!

Quote:

Originally Posted by SnowTigerVidz (Post 2534767)

How in the world do you even get this too work?

It is a Microsoft Excel based tool for now. All you need to do to use it is:

Open the tool. If there is a yellow bar at the top asking if you want macros enabled, say yes (that allows the code to run). If there is a dialogue asking if ActiveX should run, say yes. This allows the button to work, though you could say no and run the code directly from the VBA window.
Make sure your include path is set. The includes arent edited, but are used to define variable types, natives, forwards, etc.
Click the button, and find the .sp file you want converted, and click ok.
The file will now be processed. The original file is left as is, and the new one is in the same location and has the text " newsyntax" added to the end. After this, the tool is done.
After processing, you will see the following at the top of the code:

PHP Code:

/* put the line below after all of the includes! #pragma newdecls required */

Move the line of code (#pragma newdecls required) to be just past the last inlude. I may automate this step in the future as well.
Make final edits for anything not handled by the processor (see original post for what all that is) until it compiles.

Re: TOGs SourcePawn Syntax Converter [MS Excel Based for now]