Raised This Month: $ Target: $400
 0% 

Solved ??? instead of Cyrillic letters on debian after SQL_SetCharset


Post New Thread Reply   
 
Thread Tools Display Modes
Author Message
damage220
Member
Join Date: Jul 2022
Location: Ukraine
Old 11-30-2023 , 13:30   ??? instead of Cyrillic letters on debian after SQL_SetCharset
Reply With Quote #1

Hello. I have following table
PHP Code:
  CREATE TABLE `users` (
    `
idint(11NOT NULL AUTO_INCREMENT,
    `
namevarchar(255NOT NULL,
    
PRIMARY KEY (`id`),
  ) 
ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
After creating db tuple, I call SQL_SetCharset with value "utf8mb4".
PHP Code:
_dbh SQL_MakeDbTuple(hostuserpassdb);
is_success SQL_SetCharset(_dbh"utf8mb4");
server_print("[%f] [SQL_SetCharset] is_success: %d"get_gametime(),
    
is_success); 
It works perfect on my local machine but remote one - all Cyrillic letters are replaced with question marks. Does anyone know why?

Local machine
Linux distro: Arch Linux
Database: mariadb from 11.1.2-MariaDB, client 15.2 for Linux (x86_64) using readline 5.1

Remote machine
Linux distro: Debian 11
Database: mariadb Ver 15.1 Distrib 10.5.21-MariaDB, for debian-linux-gnu (x86_64) using EditLine wrapper

A few more words about the issue. For example, I have nickname "Привет".
PHP Code:
                         Local machine Remote machine
  without SQL_SetCharset 
Привет  | Привет
  
with SQL_SetCharset    | Привет  | ??????
  
SQL_SetCharset returns 1             
It seems that code parser replaces first Cyrillic letter with html code.

Well, I can remove SQL_SetCharset and then it would work on remote machine, but my test environment becomes broken. Also I want to know the reason why encoding on debian is forced without SQL_SetCharset call?

Last edited by damage220; 12-03-2023 at 13:06.
damage220 is offline
bigdaddy424
Senior Member
Join Date: Oct 2021
Location: Jupiter
Old 11-30-2023 , 22:50   Re: ??? instead of Cyrillic letters on debian after SQL_SetCharset
Reply With Quote #2

wild guess but try the first one
https://stackoverflow.com/a/4534452
__________________
bigdaddy424 is offline
damage220
Member
Join Date: Jul 2022
Location: Ukraine
Old 12-01-2023 , 17:46   Re: ??? instead of Cyrillic letters on debian after SQL_SetCharset
Reply With Quote #3

Quote:
Originally Posted by bigdaddy424 View Post
wild guess but try the first one
https://stackoverflow.com/a/4534452
Then I will have to prefix each async query with "SET NAMES utf8mb4". I guess I will switch from Debian to Ubuntu and see if it fixes the problem.
damage220 is offline
DJEarthQuake
Veteran Member
Join Date: Jan 2014
Location: Astral planes
Old 12-02-2023 , 09:45   Re: ??? instead of Cyrillic letters on debian after SQL_SetCharset
Reply With Quote #4

One MUST install matching LOCALES.
DJEarthQuake is offline
damage220
Member
Join Date: Jul 2022
Location: Ukraine
Old 12-02-2023 , 15:29   Re: ??? instead of Cyrillic letters on debian after SQL_SetCharset
Reply With Quote #5

Quote:
Originally Posted by DJEarthQuake View Post
One MUST install matching LOCALES.
Could you please instruct on how to do that? Because in my /etc/locale.gen file I have uncommented these lines
PHP Code:
en_US.UTF-8 UTF-8
ru_RU
.UTF-8 UTF-
When I enter locale-gen in the terminal, I get
PHP Code:
Generating locales (this might take a while)...
  
en_US.UTF-8... done
  ru_RU
.UTF-8... done
Generation complete

I also set locale using systemd
PHP Code:
localectl set-locale LANG=en_US.UTF-
But it did not helped. I also can see that on my local machine locale prints
PHP Code:
LANG=C
LC_CTYPE
="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-
But on remote one I get
PHP Code:
LANG=C
LANGUAGE
=
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_PAPER="C"
LC_NAME="C"
LC_ADDRESS="C"
LC_TELEPHONE="C"
LC_MEASUREMENT="C"
LC_IDENTIFICATION="C"
LC_ALL
Does this mean that locale is not installed?

Last edited by damage220; 12-02-2023 at 15:31.
damage220 is offline
DJEarthQuake
Veteran Member
Join Date: Jan 2014
Location: Astral planes
Old 12-03-2023 , 05:54   Re: ??? instead of Cyrillic letters on debian after SQL_SetCharset
Reply With Quote #6

Code:
"C" != "en_US.UTF-8"
UTF-8 is required on both. Install it.
DJEarthQuake is offline
damage220
Member
Join Date: Jul 2022
Location: Ukraine
Old 12-03-2023 , 13:02   Re: ??? instead of Cyrillic letters on debian after SQL_SetCharset
Reply With Quote #7

Quote:
Originally Posted by DJEarthQuake View Post
Code:
"C" != "en_US.UTF-8"
UTF-8 is required on both. Install it.
I found the reason. It is Debian that overrides default LANG variable (set in /etc/default/locale) in /root/.profile. Neither Arch Linux nor Ubuntu does this.
PHP Code:
# Installed by Debian Installer:
#  no localization for root because C
#  cannot be properly displayed at the Linux console
LANG=C
LANGUAGE
=
Once these variables were commented out, sqlx started to use correct charset. Thanks.

Last edited by damage220; 12-03-2023 at 13:20.
damage220 is offline
damage220
Member
Join Date: Jul 2022
Location: Ukraine
Old 12-03-2023 , 13:44   Re: ??? instead of Cyrillic letters on debian after SQL_SetCharset
Reply With Quote #8

Actually, I just realized there was no issue with setting charset. It was just the terminal that failed to display Cyrillic letters. To my shame I forgot to check them in the game in the first place.
damage220 is offline
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -4. The time now is 06:55.


Powered by vBulletin®
Copyright ©2000 - 2024, vBulletin Solutions, Inc.
Theme made by Freecode