Open-AudIT

What's on your network?
It is currently Wed Apr 25, 2018 10:36 pm

All times are UTC + 10 hours




Post new topic Reply to topic  [ 10 posts ] 
Author Message
PostPosted: Thu Apr 23, 2015 7:26 pm 
Offline
Newbie

Joined: Thu Apr 09, 2015 4:30 pm
Posts: 25
Hi, I'm using the latest version of OpenAudit here in Afghanistan. We have several languages in use here and they all use arabic characters. We're storing people's names in the system owner field of the database and while that works beautifully for English names we've just tried putting a Dari name in, and this is what is displayed (بت�سش�تب��ب���س����) when we look at this report /open-audit/index.php/main/system_display/1

Is there something that we've missed doing to allow it to display non-english text or is open audit (either database or the web gui) only designed to work for English characters?

Thanks,
Stephen


Top
 Profile  
Reply with quote  
PostPosted: Fri Apr 24, 2015 8:29 am 
Offline
Site Admin
User avatar

Joined: Mon Jun 07, 2004 11:48 am
Posts: 1944
Location: Brisbane, Australia
Strings should all be stored in the DB as UTF-8, so it "should" work fine.
Likely it's the web output.

Can you post the name itself here and I can test with it?

_________________
Support and Development hours available from Opmantek.
Please consider a purchase to help make Open-AudIT better for everyone.


Top
 Profile  
Reply with quote  
PostPosted: Fri Apr 24, 2015 2:50 pm 
Offline
Newbie

Joined: Thu Apr 09, 2015 4:30 pm
Posts: 25
Hi Mark,

Thanks for your reply. I can't read Dari and I don't have anyone here with me right now who can type an actual name. So here are just some Dari words I copied from elsewhere that I'm testing with.

دفتری میباشد

I'm not certain, but I think I might have just also found dari in the last logged on user field under windows details on another computer - it is displayed on the web page for open audit as: عÙ�Ù�Ù�ات خاÙ�Ù�_2@computer1

Regards,
Stephen


Top
 Profile  
Reply with quote  
PostPosted: Mon Jun 08, 2015 3:11 pm 
Offline
Newbie

Joined: Thu Apr 09, 2015 4:30 pm
Posts: 25
Hi Mark,
Did you manage to find out why non-english characters were not displaying?
Thanks,
Stephen


Top
 Profile  
Reply with quote  
PostPosted: Tue Jun 09, 2015 1:26 pm 
Offline
Site Admin
User avatar

Joined: Mon Jun 07, 2004 11:48 am
Posts: 1944
Location: Brisbane, Australia
Pretty sure the error is on the 'display' side.
Maybe the browser or OS the page is being viewed on.
I just edited the description and owner on a machine and copied/pasted in some of the characters above.
Seem's to work fine for me.
Open-AudIT is running on Debian and the client is Chrome 43 on OSX.

You should also see the line below in the sql creation script (/open-audit/other/openaudit_mysql.sql) which instructs MySQL to use UTF-8.
Code:
SET character_set_client = utf8;

What version of Open-AudIT are you running and is it on Windows or Linux?
Is it a very (very) old version that has steadily been upgraded over the years?

Attachment:
Open-AudIT_-_System_Display.png
Open-AudIT_-_System_Display.png [ 29.57 KiB | Viewed 4324 times ]

_________________
Support and Development hours available from Opmantek.
Please consider a purchase to help make Open-AudIT better for everyone.


Top
 Profile  
Reply with quote  
PostPosted: Tue Jun 09, 2015 3:06 pm 
Offline
Newbie

Joined: Thu Apr 09, 2015 4:30 pm
Posts: 25
Hi Mark,

It was first installed in January this year with the version that was current then. It is now running the latest release.

SET character_set_client = utf8; is set.

open-audit is running on Amazon AWS Linux and I'm using Chrome on Windows to view it.

Pasting those same characters into the owner field looks great until I press enter to submit my paste. Then it changes them to د�تر� ��باشد

If there is anything else I could check that would be helpful.

Thanks,
Stephen


Top
 Profile  
Reply with quote  
PostPosted: Thu Jun 11, 2015 12:46 pm 
Offline
Site Admin
User avatar

Joined: Mon Jun 07, 2004 11:48 am
Posts: 1944
Location: Brisbane, Australia
Sorry, no.
I just loaded the page on a Win7 x64 machine running Chrome and everything works as it should for me.
Unless someone else has idea's (or you're a supported customer), I think you're out of luck.
It's not a code issue, it's an fonts + Browser issue.

Maybe something like this Stack Overflow question would help?
http://stackoverflow.com/questions/10134754/why-isnt-google-chrome-showing-the-proper-encoding

One more question (just thought of it).
Go to menu -> Help -> Support.
Do you see all the required PHP extensions installed?
Specifically php_ext_mbstring?

_________________
Support and Development hours available from Opmantek.
Please consider a purchase to help make Open-AudIT better for everyone.


Top
 Profile  
Reply with quote  
PostPosted: Sat Jun 13, 2015 3:43 pm 
Offline
Newbie

Joined: Thu Apr 09, 2015 4:30 pm
Posts: 25
Thanks for your reply Mark.

In summary: following everything below, since I can directly query the database and get the correct results from a mysql prompt, from php at the command line and from a php page displayed in the browser, I'm really puzzled why I can't get the correct results from the openaudit web interface. I know Mark can't give me any more pointers, but I can't afford a support contract as we are an NGO serving the people of Afghanistan and rely entirely on donations to operate and since the foreign troops have reduced here the world is forgetting about Afghanistan and it is becoming harder and harder to get donations to support development work here, so if anyone else can give me some pointers I'd really appreciate it. Thanks!

The php extensions are all present (except LDAP and SNMP which we aren't using).

I've done a lot more digging and I've tried the various things in the article you linked to related to fonts and browser issues and in another article linked from that one. None of them made any difference. I even tried changing my own query (that works below) to use verdana (which is what openaudit is using) to force it to have the same problem as openaudit, but it doesn't. So it appears not to be a fonts and browser issue.

I've also created my own php query direct to the database and discovered that if you run the query direct from the command line using php it displays the UTF8 characters, but if you run the query using a browser it doesn't display it correctly, until you include these two commands with the select statement SET NAMES utf8; SET CHARACTER SET utf8; and then I get the UTF8 characters properly displayed in the browser (the same browser I'm using for testing with the openaudit web pages).

This led me to the /etc/my.cnf file but I haven't been able to find a set of instructions to add to this file yet that will actually make openaudit give the correct outputs.

So far I've added the following to my my.cnf file:
[mysqld]
collation-server = utf8_unicode_ci
init-connect='SET NAMES utf8'
character-set-server = utf8

[client]
default-character-set=utf8

[mysql]
default-character-set=utf8

I have checked the mysql configuration and was surprised to see that in response to both of the queries below the database is still listed as latin although everything else is in utf8. I'm not sure of the implications of that or why it would be, particularly as openaudit's install script was what set up mysql on this server (at least that is my understanding of how mysql was setup as we weren't previously using it on this server before installing openaudit).

mysql> show variables like "%character%";show variables like "%collation%";
+--------------------------+----------------------------+
| Variable_name | Value |
+--------------------------+----------------------------+
| character_set_client | utf8 |
| character_set_connection | utf8 |
| character_set_database | latin1 |
| character_set_filesystem | binary |
| character_set_results | utf8 |
| character_set_server | utf8 |
| character_set_system | utf8 |
| character_sets_dir | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

+----------------------+-------------------+
| Variable_name | Value |
+----------------------+-------------------+
| collation_connection | utf8_general_ci |
| collation_database | latin1_swedish_ci |
| collation_server | utf8_unicode_ci |
+----------------------+-------------------+
3 rows in set (0.00 sec)

So I did an ALTER DATABASE openaudit CHARACTER SET utf8 COLLATE utf8_general_ci;
and now it looks like this:

mysql> show variables like "%character%";show variables like "%collation%";
+--------------------------+----------------------------+
| Variable_name | Value |
+--------------------------+----------------------------+
| character_set_client | utf8 |
| character_set_connection | utf8 |
| character_set_database | utf8 |
| character_set_filesystem | binary |
| character_set_results | utf8 |
| character_set_server | utf8 |
| character_set_system | utf8 |
| character_sets_dir | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

+----------------------+-----------------+
| Variable_name | Value |
+----------------------+-----------------+
| collation_connection | utf8_general_ci |
| collation_database | utf8_unicode_ci |
| collation_server | utf8_unicode_ci |
+----------------------+-----------------+
3 rows in set (0.00 sec)

But it doesn't fix the problem. Reading more I learnt that mysql utf8 is incomplete and we should use utf8mb4 instead. So I changed everything to that and then altered the database and tables concerned to be utf8mb4 as well. (http://stackoverflow.com/questions/3513 ... -in-my-cnf and https://mathiasbynens.be/notes/mysql-utf8mb4) But that didn't help. I also checked that it wasn't a problem caused by utf8 truncating the data on entry (unlikely since it displays properly from php at the command prompt) by re-entering the data into the database. But that didn't help.

I also tried adding skip-character-set-client-handshake to my.cnf and that didn't help.

So since I can directly query the database and get the correct results from a mysql prompt, from php at the command line and from a php page displayed in the browser, I'm really puzzled why I can't get the correct results from the openaudit web interface.


Top
 Profile  
Reply with quote  
PostPosted: Tue Jul 21, 2015 7:43 pm 
Offline
Newbie

Joined: Thu Apr 09, 2015 4:30 pm
Posts: 25
Hi Mark,

I've got some news on this that makes me believe there may actually be a bug in the way the open-audit webpage is handling the Unicode characters. We noticed today that on SOME pages it actually does display the Unicode correctly, but on other pages it does not. So that makes me think that some pages are properly coded to handle Unicode, whereas others may need some changes.

You can see here that going into the computers list it is showing the Unicode correctly.
Attachment:
working.PNG
working.PNG [ 11.55 KiB | Viewed 4050 times ]


However, you can see in the details screen it does not show correctly.
Attachment:
failure.PNG
failure.PNG [ 23.28 KiB | Viewed 4050 times ]


All of those Unicode entries are identical - the one that works and the two that don't.

I hope this helps.
Stephen


Top
 Profile  
Reply with quote  
PostPosted: Tue Aug 18, 2015 3:27 pm 
Offline
Newbie

Joined: Mon Aug 17, 2015 11:20 pm
Posts: 1
I've had same problem with cyrillic letters and was able to figure out why this happen.
It's because of 'print_something' function used in 'theme-tango' templates.

This function itself is located in /usr/local/open-audit/code_igniter/application/views/theme-tango/v_template.php
What does it do? It applies 'htmlentities' function on input string which replaces every non-latin_alphabet_symbols to HTML sequences.
But we do not need to do this.
We just need to replace only special HTML characters.
So we should use function named 'htmlspecialchars' instead.

After replacing 'htmlentities($string)' to 'htmlspecialchars($string)', problem should be solved.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC + 10 hours


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group