Raised This Month: $51 Target: $400
 12% 

Download webpage to string directly


Post New Thread Reply   
 
Thread Tools Display Modes
Author Message
Dr. Greg House
Professional Troll,
Part-Time Asshole
Join Date: Jun 2010
Old 03-24-2012 , 07:18   Download webpage to string directly
Reply With Quote #1

I was wondering if it is possible to download a webpage directly into a string variable.
So far I have found two comparable possibilities:

https://forums.alliedmods.net/showthread.php?t=67640
The implemented example on how to query webservers doesn't work for me. The file is being created, however it remains empty (0 bytes filesize) and no errors are being shown in the logs.

https://forums.alliedmods.net/showthread.php?t=152216
This one seemed to be what I was looking for, but as far as I see this doesn't give me to possibility to save the webpage directly into a string variable.

Am I missing anything on these two extensions? Is there another possibility?

Precisely what I want to achieve is get the community id of a client on the server and retrieve the total playtime of l4d2 he has. This can be done by getting the webpage:
steamcommunity.com/profiles/INSERTCOMMUNITYIDHERE/stats/L4D2?l=english
and looking for this phrase:
Code:
					<div id="tsblTitle">Total Playtime:<br />Past 2 weeks:</div> 					<div id="tsblVal">136h 15m 1s<br />27h 18m</div>
Thanks in advance.

Last edited by Dr. Greg House; 03-24-2012 at 07:23.
Dr. Greg House is offline
GoD-Tony
Veteran Member
Join Date: Jul 2005
Old 03-25-2012 , 01:42   Re: Download webpage to string directly
Reply With Quote #2

Quote:
Originally Posted by Dr. Greg House View Post
https://forums.alliedmods.net/showthread.php?t=152216
This one seemed to be what I was looking for, but as far as I see this doesn't give me to possibility to save the webpage directly into a string variable.

Am I missing anything on these two extensions? Is there another possibility?
You want the "cURL write function test" from the example zip.
__________________
GoD-Tony is offline
Dr. Greg House
Professional Troll,
Part-Time Asshole
Join Date: Jun 2010
Old 03-27-2012 , 10:32   Re: Download webpage to string directly
Reply With Quote #3

Thank you, but unfortunately I'm afraid there's another problem.

Code:
/* Plugin Template generated by Pawn Studio */

#pragma semicolon 1

#pragma dynamic 32767 // Without this line will crash server!!

#include <sourcemod>
#include <cURL>

#define USE_THREAD                1
#define TEST_FOLDER                "data/curl_test"
    
    
new CURL_Default_opt[][2] = {
#if USE_THREAD
    {_:CURLOPT_NOSIGNAL,1},
#endif
    {_:CURLOPT_NOPROGRESS,1},
    {_:CURLOPT_TIMEOUT,30},
    {_:CURLOPT_CONNECTTIMEOUT,60},
    {_:CURLOPT_VERBOSE,0}
};

#define CURL_DEFAULT_OPT(%1) curl_easy_setopt_int_array(%1, CURL_Default_opt, sizeof(CURL_Default_opt))

public Plugin:myinfo = 
{
    name = "New Plugin",
    author = "PP(R)TH: Dr. Gregory House",
    description = "<- Description ->",
    version = "1.0",
    url = "<- URL ->"
}

public OnPluginStart()
{
    // Add your own code here...
    
    RegAdminCmd("sm_playtimetest", PlaytimeTest_Used, ADMFLAG_KICK);
}

public Action:PlaytimeTest_Used(client, args)
{
    
    new Handle:curl = curl_easy_init();
    if(curl != INVALID_HANDLE)
    {
        CURL_DEFAULT_OPT(curl);
        curl_easy_setopt_function(curl, CURLOPT_WRITEFUNCTION, WriteFunction);
        curl_easy_setopt_string(curl, CURLOPT_URL, "http://steamcommunity.com/profiles/76561197986272671/stats/L4D2?l=english");
        ExecCURL(curl, client);
    }
    
    return Plugin_Handled;
}

public WriteFunction(Handle:hndl, const String:buffer[], const bytes, const nmemb)
{
    
}

public onComplete(Handle:hndl, CURLcode: code, any:data)
{
    if(code != CURLE_OK)
    {
        
    }
    
    CloseHandle(hndl);
}

stock ExecCURL(Handle:curl, client)
{
#if USE_THREAD
    curl_easy_perform_thread(curl, onComplete, client);
#else
    new CURLcode:code = curl_load_opt(curl);
    if(code != CURLE_OK) {
        PrintTestCaseDebug(client, "curl_load_opt Error");
        PrintcUrlError(code);
        CloseHandle(curl);
        return;
    }
    
    code = curl_easy_perform(curl);
    
    onComplete(curl, code, client);

#endif
}
I copied this together from the given example. Afais there's no way to get a threaded solution in this form, because I can't pass an identifying value to WriteFunction so that I could know which client executed the command. Due to that it's also not possible to write to multiple buffers at the same time to store the webpages in.
Dr. Greg House is offline
Thrawn2
Veteran Member
Join Date: Apr 2009
Old 03-27-2012 , 10:53   Re: Download webpage to string directly
Reply With Quote #4

Huh? The stuff you "copy+pasted" together does exactly that...
it passes the client in it's call to curl_easy_perform_thread. only it does not use the passed param afterwards...

Take a look at the curl.inc:
PHP Code:
/**
 * Perform a file transfer, using thread
 * @ param Handle             hndl                 The handle of the curl to be used. May be INVALID_HANDLE if not essential.
 * @ param CURL_OnComplete    perform_callback     The complete callback.
 * @ param cell_t             value                Value to set.
 * @ noreturn
 */
native curl_easy_perform_thread(Handle:hndlCURL_OnComplete:perform_callbackany:value=0);

/**
 * called if curl_easy_perform_thread() or curl_easy_send_recv() Complete
 * @ param Handle      hndl        The curl handle
 * @ param CURLcode    code        The CURLcode code, see cURL_header.inc
 * @ param data                    Data passed to curl_easy_perform_thread()
 * @ noreturn
 */
funcenum CURL_OnComplete
{
    public(
Handle:hndlCURLcodecode),
    public(
Handle:hndlCURLcodecode any:data),
}; 
__________________
einmal mit profis arbeiten. einmal.
Thrawn2 is offline
Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -4. The time now is 13:04.


Powered by vBulletin®
Copyright ©2000 - 2024, vBulletin Solutions, Inc.
Theme made by Freecode