After following the lead from https://stackoverflow.com/users/395384/epb I used json_decode to translate the unicode which works.
$unicode = json_decode("\u00fc\u00be\u008c\u00a3\u00a4\u00bc") ;
$column = str_replace($unicode, "'", urldecode(
columnIndex]));
U+0259 Latin Small Letter Schwa is only encoded as the byte sequence 0x02,0x59 in the UTF-16BE encoding. It is very unlikely you will be working with byte strings in the UTF-16BE encoding as it's not an ASCII-compatible encoding and almost no-one uses it.
The encoding you want to be working with (the only ASCII-superset encoding to support both Latin Schwa and Cyrillic Schwa, as it supports all Unicode characters) is UTF-8. Ensure your input is in UTF-8 format (if it is coming from form data, serve the page containing the form as UTF-8). Then, in UTF-8, the character U+0259 is represented using the byte sequence 0xC9,0x99.
str_replace("\xC9\x99", "\xD3\x99", $string);
If you make sure to save your .php file as UTF-8-no-BOM in the text editor, you can skip the escaping and just directly say:
str_replace('ə', 'ә', $string);
A couple of possible suggestions. Firstly, remember that you need to assign the new value to $string, i.e.:
$string = str_replace("\x02\x59","\x04\xd9",$string);
Secondly, verify that your byte stream occurs in the $string. I mention this because your hex string begins with a low-byte, so you'll need to make sure your $string is not UTF8 encoded.
From the PHP manual:
Single and double quoted PHP strings have special meaning of backslash. Thus if
\has to be matched with a regular expression\\, then"\\\\"or'\\\\'must be used in PHP code.
First of all, in your regular expression, you're only using one backslash (\). As explained in the PHP manual, you need to use \\\\ to match a literal backslash (with some exceptions).
Second, you are missing the capturing groups in your original expression. preg_replace() searches the given string for matches to the supplied pattern and returns the string where the contents matched by the capturing groups are replaced with the replacement string.
The updated regular expression with proper escaping and correct capturing groups would look like:
$str2 = preg_replace('/\\\\u([0-9a-f]+)/i', '&#x$1;', $str);
Output:
おはよう
Expression: \\\\u([0-9a-f]+)
\\\\- matches a literal backslashu- matches the literalucharacter(- beginning of the capturing group[0-9a-f]- character class -- matches a digit (0-9) or an alphabet (froma-f) one or more times
)- end of capturing groupimodifier - used for case-insensitive matching
Replacement: &#x$1
&- literal ampersand character (&)#- literal pound character (#)x- literal characterx$1- contents of the first capturing group -- in this case, the strings of the form304aetc.
RegExr Demo.
This page here—titled Escaping Unicode Characters to HTML Entities in PHP—seems to tackle it with this nice function:
function unicode_escape_sequences($str){
$working = json_encode($str);
$working = preg_replace('/\\\u([0-9a-z]{4})/', '&#x$1;', $working);
return json_decode($working);
}
That seems to work with json_encode and json_decode to take pure UTF-8 and convert it into Unicode. Very nice technique. But for your example, this would work.
$str = '\u304a\u306f\u3088\u3046';
echo preg_replace('/\\\u([0-9a-z]{4})/', '&#x$1;', $str);
The output is:
おはよう
Which is:
おはよう
Which translates to:
Good morning
It was a HTML entity encoded "–" :-) That's is my failure.
<?php
$str = 'Test–asd';
$old = '–';
$new = '!';
$str = str_replace ( $old, $new, $str );
echo $str;
?>
This works just fine for me:
Output:
Test!asd
Seems that you have problems with different encoding, not UTF8 character changes.
Use
$result = preg_replace('/\x{00B0}/u'," degrees ", $result );
Please see here for more information on the \x{FFFF}-syntax.
It's important to note the difference between \xB0 and \x{00B0}:
\xB0denotes a single character with hex-codeB0(176decimal) which is the degree symbol (°) in ISO-8859-1 for example\x{00B0}denotes the unicode codepointU+00B0which describes the degree symbol (°) in the unicode system. This codepoint will be encoded using two bytes\xC2\xB0when using UTF-8 encoding.
If you use the 'u' modifier, the pattern is supposed to be treated as utf-8, so why not simply write '°' instead of '\u00B0' or '\xB0'?