Raised This Month: $32 Target: $400
 8% 

Efficient Queries


Post New Thread Reply   
 
Thread Tools Display Modes
Author Message
Puppetmaster
Senior Member
Join Date: Jun 2015
Location: Probably at a computer.
Old 07-08-2015 , 00:34   Efficient Queries
Reply With Quote #1

What are the best methods of making queries? In the past I have tried separating queries with the ; character and that seems to work fine until the queries become too long. Is there an upper limit on query size?

I have recently begun using threaded queries and they are nice and do not pause the main thread which allows me to do things in the background. The main issue that I have had is that the execution time for them is extremely inconsistent, some take a few milliseconds while other times the same query will take many times longer. Is it better to open multiple threaded connection handles to the same database and load balance them when I need a large number of threaded queries at once?
__________________

GZS Servers
Puppetmaster is offline
RedSword
SourceMod Plugin Approver
Join Date: Mar 2006
Location: Quebec, Canada
Old 07-08-2015 , 02:26   Re: Efficient Queries
Reply With Quote #2

You do not have a size limit (AFAIK), but if your DBMS is MySQL, Sourcemod uses a certain MySQL client flag that prevent doing multiple queries for security (I guess you're using SQLite because you don't seem to have that problem ?). To run multiple query at once you cannot create a transaction as you would normally do (by this I mean via SQL w/ "START TRANSACTION;" and "COMMIT;"). You should instead use SQL_CreateTransaction w/ AddQuery & SQL_ExecuteTransaction to use transactions.

I'd say to use one big query preferably, as the overall overhead should be lighter than making many new query and sending them manually. It however depends of many things. Is your database on the same computer ? How important is it to have a query be executed ? Do they all need to be executed together ?

Red
__________________
My plugins :
Red Maze
Afk Bomb
RAWR (per player/rounds Awp Restrict.)
Kill Assist
Be Medic

You can also Donate if you appreciate my work
RedSword is offline
Potato Uno
Veteran Member
Join Date: Jan 2014
Location: Atlanta, Georgia
Old 07-08-2015 , 04:07   Re: Efficient Queries
Reply With Quote #3

Can't you deadlock the server if you do this? (Or maybe I'm thinking of something else.)

https://wiki.alliedmods.net/SQL_(SourceMod_Scripting)
Potato Uno is offline
Miu
Veteran Member
Join Date: Nov 2013
Old 07-08-2015 , 09:21   Re: Efficient Queries
Reply With Quote #4

i dont think separate queries like that are properly supported, just use transactions
Miu is offline
RedSword
SourceMod Plugin Approver
Join Date: Mar 2006
Location: Quebec, Canada
Old 07-08-2015 , 13:22   Re: Efficient Queries
Reply With Quote #5

Quote:
Originally Posted by Potato Uno View Post
Can't you deadlock the server if you do this? (Or maybe I'm thinking of something else.)

https://wiki.alliedmods.net/SQL_(SourceMod_Scripting)
Why would you deadlock a server by sending a transaction ? /confused
__________________
My plugins :
Red Maze
Afk Bomb
RAWR (per player/rounds Awp Restrict.)
Kill Assist
Be Medic

You can also Donate if you appreciate my work
RedSword is offline
Powerlord
AlliedModders Donor
Join Date: Jun 2008
Location: Seduce Me!
Old 07-08-2015 , 13:48   Re: Efficient Queries
Reply With Quote #6

If you fail to properly lock/unlock tables / columns, it's possible to deadlock anything in SQL.
__________________
Not currently working on SourceMod plugin development.

Last edited by Powerlord; 07-08-2015 at 13:48.
Powerlord is offline
Puppetmaster
Senior Member
Join Date: Jun 2015
Location: Probably at a computer.
Old 07-08-2015 , 20:11   Re: Efficient Queries
Reply With Quote #7

Quote:
Originally Posted by RedSword View Post
You do not have a size limit (AFAIK), but if your DBMS is MySQL, Sourcemod uses a certain MySQL client flag that prevent doing multiple queries for security (I guess you're using SQLite because you don't seem to have that problem ?). To run multiple query at once you cannot create a transaction as you would normally do (by this I mean via SQL w/ "START TRANSACTION;" and "COMMIT;"). You should instead use SQL_CreateTransaction w/ AddQuery & SQL_ExecuteTransaction to use transactions.

I'd say to use one big query preferably, as the overall overhead should be lighter than making many new query and sending them manually. It however depends of many things. Is your database on the same computer ? How important is it to have a query be executed ? Do they all need to be executed together ?

Red
Would this be the correct way of doing it?

Code:
Handle dbc;
Transaction trans;
char errorc[255];
dbc = SQL_Connect("default", false, errorc, sizeof(errorc));
if(dbc != INVALID_HANDLE)
{
    trans = SQL_CreateTransaction();
    SQL_AddQuery(trans, "UPDATE webData SET amazingVariable = 5 WHERE id = 6;", 0);
    SQL_AddQuery(trans, "UPDATE webData SET amazingVariable = 15 WHERE id = 7;", 0);
    SQL_ExecuteTransaction(dbc, trans);
    CloseHandle(dbc)
}
__________________

GZS Servers

Last edited by Puppetmaster; 07-09-2015 at 02:25. Reason: Updated it for future reference with the information below it in other replies
Puppetmaster is offline
RedSword
SourceMod Plugin Approver
Join Date: Mar 2006
Location: Quebec, Canada
Old 07-09-2015 , 01:37   Re: Efficient Queries
Reply With Quote #8

Well, I believe you always need a ";" with non-select queries.

Other than that the transaction part looks ok; though I've never used the "Transaction" "type". I used a handle. There should be no differences though (beside type safety in your). Also I'd do "Transaction trans;" instead of "new Transaction:trans;" (I believe it should work) ; you're currently mixing 1.7 syntax with non-1.7 syntax I think (the Transaction type is a 1.7 feature if I'm correct).

Red
__________________
My plugins :
Red Maze
Afk Bomb
RAWR (per player/rounds Awp Restrict.)
Kill Assist
Be Medic

You can also Donate if you appreciate my work

Last edited by RedSword; 07-09-2015 at 01:38. Reason: oops @ CloseHandle
RedSword is offline
SM9
Veteran Member
Join Date: Sep 2013
Location: United Kingdom
Old 07-09-2015 , 07:17   Re: Efficient Queries
Reply With Quote #9

Make sure you always use threaded queries to avoid Deadlocks, also using none threaded queries is likely to cause lag spikes on your server.
SM9 is offline
Potato Uno
Veteran Member
Join Date: Jan 2014
Location: Atlanta, Georgia
Old 07-09-2015 , 08:00   Re: Efficient Queries
Reply With Quote #10

Is there a reason why non-threaded queries exist then? Everyone says not to use them like as if it's AIDS (for good reason too). Can't we make the API use threaded by default, and only use non-threaded as a fallback? (Rather than allowing people to voluntarily use them?)
Potato Uno is offline
Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -4. The time now is 19:11.


Powered by vBulletin®
Copyright ©2000 - 2024, vBulletin Solutions, Inc.
Theme made by Freecode