Raised This Month: $159 Target: $400
 39% 

PawnUnit - Testing tool for SourcePawn


Post New Thread Reply   
 
Thread Tools Display Modes
Author Message
rhelgeby
Veteran Member
Join Date: Oct 2008
Location: 0x4E6F72776179
Old 09-10-2012 , 17:57   PawnUnit - Testing tool for SourcePawn
Reply With Quote #1

PawnUnit

PawnUnit is a unit testing tool embedded in SourceMod plugins.

Feature summary
  • Easy to write tests. Complexity is hidden as far as SourcePawn allows, so that the test code is clean.
  • Fully automated testing. No interaction is required by the user, unless the developer choose to make a console command for starting tests manually.
  • Embedded in plugin with minimal impact on plugin code. Nearly no need to modify existing plugin code other than including PawnUnit and starting it.

It was supposed to support asynchronous testing so it could wait for events, timer callbacks and such, but SourcePawn is limited here and there's currently no elegant way of doing it. PawnUnit will still support suspending and resuming tests so developers could hack in their own solutions for asynchronous testing.

This is an early version that seems to work fine with simple tests, but it may have some bugs. Suspending and resuming tests isn't working as expected yet. Other possible improvements is how it's writing messages to the console, and eventually formatting some HTML/XML test results.

Project on Google Code
Files also mirrored at the Google Code project: http://code.google.com/p/pawnunit/

Credits

PawnUnit is mainly based on ideas from:
Data structures

Test cases are organized in containers to allow grouping of tests.

Hierarchy/Containers
  1. TestCollection
    A collection of test cases. Can be related to a module or component. It also has two optional before and after callbacks that can be called before and after each test case.
  2. TestCase
    A named test case. Has one or more test phases.
  3. TestPhase
    A callback used to execute test code. Added for eventual future asynchronous testing.

    Asynchronous test cases would usually need several phases where the first phase trigger something that it need to wait for while the rest of the plugin continue as usual. Once it's complete the next phase will be executed and results can be tested.

Example (subset of the bundled example)
PHP Code:
/*
 * ============================================================================
 *
 *  PawnUnit Demo
 *
 *  File:          pawnunitdemo.sp
 *  Type:          Main
 *  Description:   Example usage of PawnUnit.
 *
 *  Copyright (C) 2012  Richard Helgeby
 *
 *  This program is free software: you can redistribute it and/or modify
 *  it under the terms of the GNU General Public License as published by
 *  the Free Software Foundation, either version 3 of the License, or
 *  (at your option) any later version.
 *
 *  This program is distributed in the hope that it will be useful,
 *  but WITHOUT ANY WARRANTY; without even the implied warranty of
 *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 *  GNU General Public License for more details.
 *
 *  You should have received a copy of the GNU General Public License
 *  along with this program.  If not, see <http://www.gnu.org/licenses/>.
 *
 * ============================================================================
 */

// Comment out to not require semicolons at the end of each line of code.
#pragma semicolon 1

#include <sourcemod>
#include <pawnunit>

#define PLUGIN_VERSION "1.0.0-dev"

/**
 * Record plugin info.
 */
public Plugin:myinfo =
{
    
name "PawnUnit Demo",
    
author "Richard Helgeby",
    
description "Example usage of PawnUnit.",
    
version PLUGIN_VERSION,
    
url "http://code.google.com/p/pawnunit/"
};

new 
TestCollection:ExampleCollection INVALID_TEST_COLLECTION;

/**
 * Plugin is loading.
 */
public OnPluginStart()
{
    
// Build tests.
    
InitTestCases();
    
    
PawnUnit_Run(ExampleCollection);
    
PawnUnit_PrintResults(ExampleCollection);
}

/**
 * Plugin is unloading.
 */
public OnPluginEnd()
{
    
// Delete collection and test cases. Not really necessary in this example
    // plugin since they're only created once anyways.
    
PawnUnit_DeleteTestCollection(ExampleCollection);
}

/**
 * Returns whether a number is a prime number.
 *
 * Source: http://stackoverflow.com/questions/1538644/c-determine-if-a-number-is-prime
 *
 * @param number    Number to test.
 *
 * @return          True if prime, false otherwise.
 */
bool:IsPrime(number)
{
    if (
number <= 1)
    {
        return 
false;
    }
    
    for (new 
2numberi++)
    {
        if (
number == && != number)
        {
            return 
false;
        }
    }
    
    return 
true;
}

/******************
 *   Test cases   *
 ******************/

InitTestCases()
{
    
// Tests cases should be added to a collection. Collections can be used for
    // organizing test code or grouping test cases that need certain stuff to
    // be done before and after each test.
    
ExampleCollection PawnUnit_CreateTestCollection("Example tests");
    
    
// Create a test case.
    
new TestCase:primeTest PawnUnit_CreateTestCase("Prime number test");
    
    
// Add a test phase to it (callback for the actual test code).
    
PawnUnit_AddTestPhase(primeTestPrimeTest);
    
    
// Add the test case to the collection.
    
PawnUnit_AddTestCase(ExampleCollectionprimeTest);
}

/**
 * This is the test case callback (there's only one phase in this example and
 * therefore only one callback).
 */
public TestControlAction:PrimeTest(TestCase:testCase)
{
    
// Assert is a macro that returns an error code if the expression is
    // logically false.
    
    // True and false are simple assert functions that return whether the actual
    // value passed are true or false.
    
    
Assert(False(IsPrime(-1)));
    
Assert(False(IsPrime(0)));
    
Assert(False(IsPrime(1)));

    
Assert(True(IsPrime(2)));
    
Assert(True(IsPrime(3)));
    
Assert(False(IsPrime(4)));
    
Assert(True(IsPrime(5)));
    
Assert(False(IsPrime(6)));
    
Assert(True(IsPrime(7)));
    
Assert(False(IsPrime(8)));
    
Assert(False(IsPrime(9)));
    
Assert(False(IsPrime(10)));
    
Assert(True(IsPrime(11)));
    
Assert(False(IsPrime(12)));
    
Assert(True(IsPrime(13)));
    
Assert(False(IsPrime(14)));
    
Assert(False(IsPrime(15)));
    
Assert(False(IsPrime(16)));
    
Assert(False(IsPrime(17)));
    
Assert(True(IsPrime(17)));

    
// Indicates that the test passed. If any of the assert calls above failed
    // it would return earlier with an error code.
    
return Test_Continue;

Example output (full version)
The prime test failed because there were two asserts for no. 17 (typo, but a great example anyways).
Code:
Running test "Prime number test" at phase 0...
TEST FAILED...
Running test "Should pass 5 phases" at phase 0...
Running test "Should pass 5 phases" at phase 1...
Running test "Should pass 5 phases" at phase 2...
Running test "Should pass 5 phases" at phase 3...
Running test "Should pass 5 phases" at phase 4...
Test passed...
Running test "Should fail on third phase" at phase 0...
Running test "Should fail on third phase" at phase 1...
Running test "Should fail on third phase" at phase 2...
TEST FAILED...
Running test "Should pass" at phase 0...
Test passed...
Running test "Should fail" at phase 0...
TEST FAILED...
Running test "CellEquals failing" at phase 0...
TEST FAILED...
Running test "CellEquals passing" at phase 0...
Test passed...
Running test "FloatEquals failing" at phase 0...
TEST FAILED...
Running test "FloatEquals passing" at phase 0...
Test passed...
Running test "FloatEquals passing with margin" at phase 0...
Test passed...
Running test "FloatEquals failing with margin" at phase 0...
TEST FAILED...
Running test "Suspending for 2 seconds" at phase 0...
Testing suspended...
Testing suspended...
Tests in collection "Example tests"
Test name:                          Status:
===============================================================================
Prime number test                   Value was not false
Should pass 5 phases                [PASSED]
Should fail on third phase          Failed on purpose.
Should pass                         [PASSED]
Should fail                         Failed on purpose.
CellEquals failing                  Expected 1, but was 2
CellEquals passing                  [PASSED]
FloatEquals failing                 Expected 1.000000, but was 2.000000
FloatEquals passing                 [PASSED]
FloatEquals passing with margin     [PASSED]
FloatEquals failing with margin     Expected 1.000000, but was 1.009999
Suspending for 2 seconds
-------------------------------------------------------------------------------
Tests passed:        5
Tests failed:        6
-------------------------------------------------------------------------------
Download
The pawnunit-dev-r10.zip archive contains a standalone develop environment with compilers and scripts included. Download pawnunit-dev-src-r10.zip if you want the source code only.
Attached Files
File Type: zip pawnunit-dev-r10.zip (2.39 MB, 664 views)
File Type: zip pawnunit-dev-src-r10.zip (12.0 KB, 456 views)
__________________
Richard Helgeby

Zombie:Reloaded | PawnUnit | Object Library
(Please don't send private messages for support, they will be ignored. Use the forum.)

Last edited by rhelgeby; 08-18-2013 at 20:09.
rhelgeby is offline
Send a message via MSN to rhelgeby
rhelgeby
Veteran Member
Join Date: Oct 2008
Location: 0x4E6F72776179
Old 09-10-2012 , 18:31   Re: PawnUnit - Testing tool for SourcePawn
Reply With Quote #2

(Reserved for future use)

I'm open for suggestions about this tool.
__________________
Richard Helgeby

Zombie:Reloaded | PawnUnit | Object Library
(Please don't send private messages for support, they will be ignored. Use the forum.)
rhelgeby is offline
Send a message via MSN to rhelgeby
Zephyrus
Cool Pig B)
Join Date: Jun 2010
Location: Hungary
Old 09-12-2012 , 13:18   Re: PawnUnit - Testing tool for SourcePawn
Reply With Quote #3

oh nooo thats not how you check if a number is prime, you only have to check until i=sqrt(num) and i havent gone into more complex algorithms yet :p
__________________
Taking private C++/PHP/SourcePawn requests, PM me.

Last edited by Zephyrus; 09-12-2012 at 13:19.
Zephyrus is offline
rhelgeby
Veteran Member
Join Date: Oct 2008
Location: 0x4E6F72776179
Old 09-13-2012 , 16:23   Re: PawnUnit - Testing tool for SourcePawn
Reply With Quote #4

IsPrime is not my code. It's just something I grabbed to demonstrate a test. It doesn't matter if there's a faster way to do it as long as this works.
__________________
Richard Helgeby

Zombie:Reloaded | PawnUnit | Object Library
(Please don't send private messages for support, they will be ignored. Use the forum.)
rhelgeby is offline
Send a message via MSN to rhelgeby
alongub
Veteran Member
Join Date: Aug 2009
Location: Israel
Old 02-03-2013 , 08:44   Re: PawnUnit - Testing tool for SourcePawn
Reply With Quote #5

This is just awesome, I was looking for a proper testing tool for SourceMod for ages.
__________________
alongub is offline
rhelgeby
Veteran Member
Join Date: Oct 2008
Location: 0x4E6F72776179
Old 02-03-2013 , 08:47   Re: PawnUnit - Testing tool for SourcePawn
Reply With Quote #6

It's still a work in progress, but it's suitable for regular simple unit tests.

I noticed that it's not good at handling native errors within the tests. The test fails and it moves on to next test, but it doesn't tell you what's wrong. I figured out I had to check the error logs for such errors.

Do you need asynchronous test cases (such as waiting for timer callbacks)? I haven't figured out an elegant way to do this in SourcePawn (though it works fine in the JavaScript version).
__________________
Richard Helgeby

Zombie:Reloaded | PawnUnit | Object Library
(Please don't send private messages for support, they will be ignored. Use the forum.)

Last edited by rhelgeby; 02-03-2013 at 08:54.
rhelgeby is offline
Send a message via MSN to rhelgeby
alongub
Veteran Member
Join Date: Aug 2009
Location: Israel
Old 02-03-2013 , 09:05   Re: PawnUnit - Testing tool for SourcePawn
Reply With Quote #7

Quote:
Originally Posted by rhelgeby View Post
Do you need asynchronous test cases (such as waiting for timer callbacks)? I haven't figured out an elegant way to do this in SourcePawn (though it works fine in the JavaScript version).
Yes, definitely. This will be extremely useful in testing database callbacks.
__________________

Last edited by alongub; 02-03-2013 at 09:05.
alongub is offline
rhelgeby
Veteran Member
Join Date: Oct 2008
Location: 0x4E6F72776179
Old 02-03-2013 , 11:32   Re: PawnUnit - Testing tool for SourcePawn
Reply With Quote #8

Yeah, I suppose there's a great need for it when a lot of common stuff to do in SourceMod is asynchronous too. So far I have no idea on how to implement it without using some wrappers, but that will also mess with the code you're testing, so I don't like that solution.
__________________
Richard Helgeby

Zombie:Reloaded | PawnUnit | Object Library
(Please don't send private messages for support, they will be ignored. Use the forum.)
rhelgeby is offline
Send a message via MSN to rhelgeby
alongub
Veteran Member
Join Date: Aug 2009
Location: Israel
Old 02-03-2013 , 14:37   Re: PawnUnit - Testing tool for SourcePawn
Reply With Quote #9

Quote:
Originally Posted by rhelgeby View Post
Yeah, I suppose there's a great need for it when a lot of common stuff to do in SourceMod is asynchronous too. So far I have no idea on how to implement it without using some wrappers, but that will also mess with the code you're testing, so I don't like that solution.
Let's wait for SourcePawn 2.0?
__________________
alongub is offline
rhelgeby
Veteran Member
Join Date: Oct 2008
Location: 0x4E6F72776179
Old 02-03-2013 , 23:29   Re: PawnUnit - Testing tool for SourcePawn
Reply With Quote #10

Sure. I'll suspend this project for two years. zzzZZZ

It's possible some how, but probably not as elegant as I'd want it. I'll figure out something some time.
__________________
Richard Helgeby

Zombie:Reloaded | PawnUnit | Object Library
(Please don't send private messages for support, they will be ignored. Use the forum.)
rhelgeby is offline
Send a message via MSN to rhelgeby
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -4. The time now is 16:16.


Powered by vBulletin®
Copyright ©2000 - 2025, vBulletin Solutions, Inc.
Theme made by Freecode