What you're asking for is extremely hard. If possible, getting the user to specify the encoding is the best. Preventing an attack shouldn't be much easier or harder that way.

However, you could try doing this:

iconv(mb_detect_encoding($text, mb_detect_order(), true), "UTF-8", $text);

Setting it to strict might help you get a better result.

Answer from Jeff Day on Stack Overflow
🌐
PHP
php.net › manual › en › function.utf8-encode.php
PHP: utf8_encode - Manual
#[\Deprecated] function utf8_encode(string $string): string · This function converts the string string from the ISO-8859-1 encoding to UTF-8.
Top answer
1 of 14
300

What you're asking for is extremely hard. If possible, getting the user to specify the encoding is the best. Preventing an attack shouldn't be much easier or harder that way.

However, you could try doing this:

iconv(mb_detect_encoding($text, mb_detect_order(), true), "UTF-8", $text);

Setting it to strict might help you get a better result.

2 of 14
33

In motherland Russia we have four popular encodings, so your question is in great demand here.

Only by character codes of symbols you can not detect the encoding, because code pages intersect. Some codepages in different languages have even full intersection. So, we need another approach.

The only way to work with unknown encodings is working with probabilities. So, we do not want to answer the question "what is encoding of this text?", we are trying to understand "what is most likely encoding of this text?".

One guy here in a popular Russian tech blog invented this approach:

Build the probability range of character codes in every encoding you want to support. You can build it using some big texts in your language (e.g., some fiction, use Shakespeare for English and Tolstoy for Russian, LOL). You will get something like this:

    encoding_1:
    190 => 0.095249209893009,
    222 => 0.095249209893009,
    ...
    encoding_2:
    239 => 0.095249209893009,
    207 => 0.095249209893009,
    ...
    encoding_N:
    charcode => probabilty

Next, you take text in an unknown encoding and for every encoding in your "probability dictionary" you search for the frequency of every symbol in the unknown-encoded text. Sum the probabilities of symbols. Encoding with the bigger rating is likely the winner. There are better results for bigger texts.

Btw, mb_detect_encoding certainly does not work. Yes, at all. Please, take a look of the mb_detect_encoding source code in "ext/mbstring/libmbfl/mbfl/mbfl_ident.c".

🌐
MojoAuth
mojoauth.com › character-encoding-decoding › utf-8-encoding--php
UTF-8 Encoding : PHP | Encoding Solutions Across Programming Languages
<?php // Original string in ISO-8859-1 encoding $originalString = "Hello, world!"; // Convert to UTF-8 $utf8String = mb_convert_encoding($originalString, 'UTF-8', 'ISO-8859-1'); // Output the UTF-8 encoded string echo $utf8String; // Outputs: ...
People also ask

How to insert Unicode characters in MySQL using PHP?
In order to insert Unicode characters in MySQL, you need to create a table with Unicode support, select the appropriate encoding/collation settings, and specify the charset in the MySQL connection. Then, you can proceed and employ PHP code to insert Unicode as you please.
🌐
toptal.com
toptal.com › php › a-utf-8-primer-for-php-and-mysql
A Guide to UTF-8 Encoding in PHP and MySQL | Toptal®
What is UTF-8 character set?
Defined by the Unicode standard, UTF-8 is an 8-bit character encoding capable of storing ay Unicode character. It is backwards compatible with ASCII.
🌐
toptal.com
toptal.com › php › a-utf-8-primer-for-php-and-mysql
A Guide to UTF-8 Encoding in PHP and MySQL | Toptal®
What does UTF-8 stand for?
UTF is short for Unicode Transformation Format, while the “8” suffix denotes the use of 8-bit blocks to represent characters.
🌐
toptal.com
toptal.com › php › a-utf-8-primer-for-php-and-mysql
A Guide to UTF-8 Encoding in PHP and MySQL | Toptal®
🌐
GeeksforGeeks
geeksforgeeks.org › php › php-utf8_encode-function
PHP utf8_encode() Function - GeeksforGeeks
September 27, 2024 - The utf8_encode() function in PHP converts ISO-8859-1 (Latin-1) encoded strings to UTF-8 encoding.
🌐
Delft Stack
delftstack.com › home › howto › php › php convert to utf 8
PHP UTF-8 Conversion | Delft Stack
March 11, 2025 - <?php $string = "Café"; $utf8_string = utf8_encode($string); echo $utf8_string; ?> ... In this example, the utf8_encode() function converts a string that is assumed to be ISO-8859-1 encoded into UTF-8.
🌐
Learningbox
learningbox.online › home › column
How to unify text file encoding to utf-8 in PHP| learningBOX - an e-learning system
September 11, 2020 - If you specify the source character code, mb_convert_encoding will work properly. mb_convert_encoding works correctly as long as you specify the source character code.
🌐
W3Schools
w3schools.com › php › func_xml_utf8_encode.asp
PHP utf8_encode() Function
Note: This function was deprecated in PHP 8.2.0. Unicode is a universal standard, and has been developed to describe all possible characters of all languages plus a lot of symbols with one unique number for each character/symbol. However, it is not always possible to transfer a Unicode character to another computer reliably. UTF-8 has been developed to transfer a Unicode character from one computer to another.
🌐
Toptal
toptal.com › php › a-utf-8-primer-for-php-and-mysql
A Guide to UTF-8 Encoding in PHP and MySQL | Toptal®
January 16, 2026 - A useful function for doing this (which I found here) is the following: function utf8_for_xml($string) { return preg_replace('/[^\x{0009}\x{000a}\x{000d}\x{0020}-\x{D7FF}\x{E000}-\x{FFFD}]+/u', ' ', $string); }
Find elsewhere
🌐
Honeybadger
honeybadger.io › blog › php-character-encoding-unicode-utf8-ascii
Character Encoding for PHP Developers: Unicode, UTF-8 and ASCII - Honeybadger Developer Blog
May 17, 2021 - // Auto detect encoding from JIS, eucjp-win, sjis-win, then convert str to UTF-8 $string = mb_convert_encoding($str, "UTF-8", "JIS, eucjp-win, sjis-win"); We also have another function in PHP called iconv, but since it depends on the underlying ...
🌐
TutorialsPoint
tutorialspoint.com › convert-ascii-to-utf-8-encoding-in-php
Convert ASCII TO UTF-8 Encoding in PHP?
April 7, 2020 - The iconv() function converts strings between different character encodings. It requires specifying the source and target encodings − · <?php $str = "Hello World"; echo 'Original ASCII: ' . $str .
🌐
Université Pierre et Marie Curie
www-ppti.ufr-info-p6.jussieu.fr › doc-online › PHP › php-chunked-xhtml › function.utf8-encode.html
Converts a string from ISO-8859-1 to UTF-8
<?php // Convert the string 'Zoë' from ISO 8859-1 to UTF-8 $iso8859_1_string = "\x5A\x6F\xEB"; $utf8_string = utf8_encode($iso8859_1_string); echo bin2hex($utf8_string), "\n"; ?> L'exemple ci-dessus va afficher : 5a6fc3ab · utf8_decode() - Converts a string from UTF-8 to ISO-8859-1, replacing invalid or unrepresentable characters ·
Top answer
1 of 5
6
function convert_file_to_utf8($source, $target) {
    $content=file_get_contents($source);
    # detect original encoding
    $original_encoding=mb_detect_encoding($content, "UTF-8, ISO-8859-1, ISO-8859-15", true);
    # now convert
    if ($original_encoding!='UTF-8') {
        $content=mb_convert_encoding($content, 'UTF-8', $original_encoding);

    }
    $bom=chr(239) . chr(187) . chr(191); # use BOM to be on safe side
    file_put_contents($target, $bom.$content);
}
2 of 5
3

before you can convert it to utf-8, you need to know what characterset it is. if you can't figure that out, you can't in any sane way convert it to utf8.. however, an insane way to convert it to utf-8, if the encoding cannot be determined, is to simply strip any bytes that doesn't happen to be valid in utf-8, you might be able to use that as a fallback...

warning, untested code (im suddenly in a hurry), but may look something like this:

foreach ( $datas as $data ) {
    $encoding = guess_encoding ( $data );
    if (empty ( $encoding )) {
        // encoding cannot be determined...
        // as a fallback, we simply strip any bytes that isnt valid utf-8...
        // obviously this isn't a reliable conversion scheme.
        // also this could probably be improved
        $data = iconv ( "ASCII", "UTF-8//TRANSLIT//IGNORE", $text );
    } else {
        $data = mb_convert_encoding ( $data, 'UTF-8', $encoding );
    }
    $row [] = explode ( ',', $data );
}
function guess_encoding(string $str): string {
    $blacklist = array (
            'pass',
            'auto',
            'wchar',
            'byte2be',
            'byte2le',
            'byte4be',
            'byte4le',
            'BASE64',
            'UUENCODE',
            'HTML-ENTITIES',
            '7bit',
            '8bit' 
    );
    $encodings = array_flip ( mb_list_encodings () );
    foreach ( $blacklist as $tmp ) {
        unset ( $encodings [$tmp] );
    }
    $encodings = array_keys ( $encodings );
    $detected = mb_detect_encoding ( $str, $encodings, true );
    return ( string ) $detected;
}
🌐
SitePoint
sitepoint.com › blog › patterns & practices › bringing unicode to php with portable utf-8
Bringing Unicode to PHP with Portable UTF-8 — SitePoint
November 6, 2024 - Portable UTF8 can be easily installed ... ‘voku\helper\UTF8’. For example, to convert a string to UTF8, you can use the ‘to_utf8’ method like this: ‘UTF8::to_utf8($string)’....
🌐
PHP.Watch
php.watch › versions › 8.2 › utf8_encode-utf8_decode-deprecated
`utf8_encode` and `utf8_decode` functions deprecated - PHP 8.2 • PHP.Watch
The UConverter class in the intl extension also provides a way to convert character encodings from one to another. It follows a similar function signature as mbstring counterparts as well. Using UConverter::transcode, it is possible to replicate utf8_encode functionality:
🌐
PHP Classes
phpclasses.org › package › 11063-PHP-Convert-a-CSV-file-to-have-data-in-UTF-8-encoding.html
PHP Convert CSV to UTF-8: Convert a CSV file to have data in UTF-8 encoding - PHP Classes
$string = convert_csv_to_utf8::convert_to_utf8($string, $encoding); ... No pages of applications that use this class were specified. If you know an application of this package, send a message to the author to add a link here. ... For more information send a message to info at phpclasses dot org.
🌐
OnlinePHP
onlinephp.io › utf8-encode
utf8_encode - Online Tool
PHP Functions · String Manipulation · utf8_encode · Execute utf8_encode with this online tool utf8_encode() - Converts a string from ISO-8859-1 to UTF-8 · Utf8 Encode Online Tool · Manual · Code Examples · Utf8 Encode Online Tool · Manual ...
🌐
OnlinePHP
onlinephp.io › utf8-encode › manual
utf8_encode - OnlinePHP.io Example
PHP 4, PHP 5, PHP 7, PHP 8 · utf8_encode - Converts a string from ISO-8859-1 to UTF-8 · Utf8 Encode Online Tool · Manual · Code Examples · utf8_encode( string$string ): string · This function converts the string string from the ISO-8859-1 ...