[Solved] getting data from get request not work

newone123 · **Author**

FIXED BY CURL
1. AmxxCurl
2. LINUX: download and move curl_amxx_i386.so to amxmodx/modules (server)
3. write curl_amxx in amxmodx/configs/modlues.ini (server)
4. put the includes: curl.inc and curl_consts.inc in amxmodx\scripting\include (server and compiler)
FIX

Code:

#include <amxmodx>
#include <amxmisc>
#include <curl>

#define CURL_BUFFER_SIZE 512

public plugin_init()
{
   register_plugin("curl http get", "1.0", "Polarhigh")
   
   new data[1]
   data[0] = fopen("addons/amxmodx/alliedmods_main_page.html", "wb")
   server_print("curl start")
   
   new CURL:curl = curl_easy_init()
   curl_easy_setopt(curl, CURLOPT_BUFFERSIZE, CURL_BUFFER_SIZE)
   curl_easy_setopt(curl, CURLOPT_URL, "https://forums.alliedmods.net")
   curl_easy_setopt(curl, CURLOPT_WRITEDATA, data[0])
   curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, "write")
   curl_easy_perform(curl, "complite", data, sizeof(data))
}

public write(data[], size, nmemb, file)
{
   new actual_size = size * nmemb;
   
   fwrite_blocks(file, data, actual_size, BLOCK_CHAR)
   
   return actual_size
}

public complite(CURL:curl, CURLcode:code, data[])
{
  if(code == CURLE_WRITE_ERROR)
     server_print("transfer aborted")
  else
     server_print("curl complete")
  
  fclose(data[0])
  curl_easy_cleanup(curl)
}

The Problem
code

Code:

#include <amxmodx>
#include <sockets>

#define READ_TASK 1
#define TOPIC "/forumdisplay.php?f=11"
#define HOST "forums.alliedmods.net"
new g_Socket
new g_Data[1001]

public plugin_init() {
	register_clcmd("say !newz", "get_news")
}

public get_news() {
	new error 
	g_Socket = socket_open(HOST, 80, SOCKET_TCP, error)
	if(error == 0) {
		new request[501]
		formatex(request, charsmax(request), "GET %s HTTP/1.1 ^n Host: %s ^n Connection: close", TOPIC, HOST)
		socket_send2(g_Socket, request, charsmax(request))
		set_task(0.1 , "read_socket", READ_TASK, _, _, "b")
		set_task(30.0 , "close_socket")
	} else client_print(0, print_chat, "error")
}

public read_socket() {
	if(socket_is_readable(g_Socket)) {
		new countByte 
		countByte = socket_recv(g_Socket, g_Data, charsmax(g_Data))
		if(countByte > 0) {
			remove_task(READ_TASK)
		}
	}
}

public close_socket() {
	socket_close(g_Socket)
	new path[] = "addons/j/data.txt"
	new file = fopen(path,"w")
	if(file) fputs(file, g_Data)
	fclose(file)
}

result

Code:

HTTP/1.1 400 Bad Request

Server: cloudflare

Date: Thu, 13 May 2021 05:39:43 GMT

Content-Type: text/html

Content-Length: 155

Connection: close

CF-RAY: -



<html>

<head><title>400 Bad Request</title></head>

<body>

<center><h1>400 Bad Request</h1></center>

<hr><center>cloudflare</center>

</body>

</html>

Dragos · *Last edited by Dragos; 05-13-2021 at 01:59.*

Have You Tried other sites?
Because alliedmods are using Cloudflare, it blocks the weird request and terminates your conexion.

newone123 · *Last edited by newone123; 05-13-2021 at 02:29.*

Quote:

Originally Posted by Dragos

Have You Tried other sites?
Because alliedmods are using Cloudflare, it blocks the weird request and terminates your conexion.

i try on newegg (its work for my in another language)

Code:

#define TOPIC "/global/il-en/black-acer-nitro-ed270-xbmiipx-um-he0aa-x01-27/p/N82E16824011367?Item=N82E16824011367&cm_sp=Homepage_dailydeals-_-P1_24-011-367-_-05122021"
#define HOST "newegg.com"

result:

Code:

HTTP/1.0 408 Request Time-out

Server: AkamaiGHost

Mime-Version: 1.0

Date: Thu, 13 May 2021 06:26:32 GMT

Content-Type: text/html

Content-Length: 218

Expires: Thu, 13 May 2021 06:26:32 GMT



<HTML><HEAD>
<TITLE>Request Timeout</TITLE>
</HEAD><BODY>
<H1>Request Timeout</H1>
The server timed out while waiting for the browser's request.<P>
Reference #2.d6fbce17.1620887192.0
</BODY></HTML>

**HamletEagle** · *Last edited by HamletEagle; 05-13-2021 at 02:31.*

You can't connect to https websites using sockets. Use curl.

newone123 · 05-13-2021 , 02:36 Re: getting data from get request not work

Quote:

Originally Posted by HamletEagle

You can't connect to https websites using sockets. Use curl.

can you send me please tutorial for amx on this subject ?
and why this tutorial not removed ? https://forums.alliedmods.net/showthread.php?t=151401

newone123 · 05-13-2021 , 02:46 Re: getting data from get request not work

found this: https://github.com/Polarhigh/AmxxCurl/releases
but i dont know how install this

Black Rose · *Last edited by Black Rose; 05-13-2021 at 04:25.*

Quote:

Originally Posted by newone123

and why this tutorial not removed ? https://forums.alliedmods.net/showthread.php?t=151401

Even though that specific example doesn't work it's still relevant.

Quote:

Originally Posted by newone123

found this: https://github.com/Polarhigh/AmxxCurl/releases
but i dont know how install this

1. Download the zip for your server platform.
2. Unzip and place the binary in "cstrike\addons\amxmodx\modules".
3. Open "cstrike\addons\amxmodx\configs\modules.i ni" and put "curl_amxx" on a new row in there.
Simple as that. More information here:
https://forums.alliedmods.net/showthread.php?t=285656

Bad request is not from HTTPS conflict, it's because your request header is incorrectly formatted. When SSL/TLS is forced you should get 301 Moved Permanently.
HTTP has a very specific format and is not tolerant to mistakes, you can't just throw in spaces because you want to. You also need the double newline at the end of the request. Should look like this:

Code:

formatex(request, charsmax(request), "GET %s HTTP/1.1^nHost: %s^nConnection: close^n^n", TOPIC, HOST)

newone123 · 05-13-2021 , 04:37 Re: getting data from get request not work

Quote:

Originally Posted by Black Rose

Even though that specific example doesn't work it's still relevant.
Bad request is not from HTTPS conflict, it's because your request header is incorrectly formatted. When SSL/TLS is forced you should get 301 Moved Permanently.
HTTP has a very specific format and is not tolerant to mistakes, you can't just throw in spaces because you want to. You also need the double newline at the end of the request. Should look like this:

Code:

formatex(request, charsmax(request), "GET %s HTTP/1.1^nHost: %s^nConnection: close^n^n", TOPIC, HOST)

lets start here, i change like you said and get this:

Code:

HTTP/1.1 301 Moved Permanently

Server: AkamaiGHost

Content-Length: 0

Location: https://www.newegg.com/global/il-en/black-acer-nitro-ed270-xbmiipx-um-he0aa-x01-27/p/N82E16824011367?Item=N82E16824011367&cm_sp=Homepage_dailydeals-_-P1_24-011-367-_-05122021

Date: Thu, 13 May 2021 08:31:43 GMT

Connection: close

i know 200 range is good, its good also ? and if yes can i get the all page and how ?

**HamletEagle** · *Last edited by HamletEagle; 05-13-2021 at 05:05.*

Quote:

Originally Posted by HamletEagle

You can't connect to https websites using sockets. Use curl.

Do I need to repeat? You can not connect to https using sockets. You will get 301 moved permanently.

newone123 · *Last edited by newone123; 05-13-2021 at 06:27.*

curl works