A string needs no encoding. It is simply a sequence of Unicode characters.
You need to encode when you want to turn a String into a sequence of bytes. The charset the you choose (UTF-8, cp1255, etc.) determines the Character->Byte mapping. Note that a character is not necessarily translated into a single byte. In most charsets, most Unicode characters are translated to at least two bytes.
Encoding of a String is carried out by:
String s1 = "some text";
byte[] bytes = s1.getBytes("UTF-8"); // Charset to encode into
You need to decode when you have а sequence of bytes and you want to turn them into a String. When yоu dо that you need to specify, again, the charset with which the bytеs were originally encoded (otherwise you'll end up with garblеd tеxt).
Decoding:
String s2 = new String(bytes, "UTF-8"); // Charset with which bytes were encoded
If you want to understand this better, a great text is "The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)"
Answer from Itay Maman on Stack OverflowA string needs no encoding. It is simply a sequence of Unicode characters.
You need to encode when you want to turn a String into a sequence of bytes. The charset the you choose (UTF-8, cp1255, etc.) determines the Character->Byte mapping. Note that a character is not necessarily translated into a single byte. In most charsets, most Unicode characters are translated to at least two bytes.
Encoding of a String is carried out by:
String s1 = "some text";
byte[] bytes = s1.getBytes("UTF-8"); // Charset to encode into
You need to decode when you have а sequence of bytes and you want to turn them into a String. When yоu dо that you need to specify, again, the charset with which the bytеs were originally encoded (otherwise you'll end up with garblеd tеxt).
Decoding:
String s2 = new String(bytes, "UTF-8"); // Charset with which bytes were encoded
If you want to understand this better, a great text is "The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)"
the core functions are getBytes(String charset) and new String(byte[] data). you can use these functions to do UTF-8 decoding.
UTF-8 decoding actually is a string to string conversion, the intermediate buffer is a byte array. since the target is an UTF-8 string, so the only parameter for new String() is the byte array, which calling is equal to new String(bytes, "UTF-8")
Then the key is the parameter for input encoded string to get internal byte array, which you should know beforehand. If you don't, guess the most possible one, "ISO-8859-1" is a good guess for English user.
The decoding sentence should be
String decoded = new String(encoded.getBytes("ISO-8859-1"));
When dealing with Strings, always remember: byte != char. So in your first example, you have the char c3, not the byte c3 which is a huge difference: The byte would be part of the UTF-8 sequence but the char already is Unicode. So when you convert that to UTF-8, the Unicode character c3 must become the byte sequence c3 83.
So the question is: How did you get the String? There must be a bug in that code which doesn't properly handle UTF-8 encoded byte sequences.
The reason why ISO-8859-1 usually works is that this encoding doesn't modify any char with a code point < 256 (i.e. anything between 0 and 255), so UTF-8 encoded byte sequences won't be modified.
Your last example is also wrong: The char e9 is é in ISO-8859-1 and Unicode. In UTF-8, it's not valid since it's not a byte and since it's the byte c3 prefix is missing. That said, it correctly represents the Unicode string you seek.
If you start with the Java String where "d\u00C3\u00A9jeuner".equals(stmt) then the data is already corrupt at this stage.
A Java char is not a C char. A char in Java is 16bits wide and implicitly contains UTF-16 encoded data. Trying to store any other encoded data in a Java char/String type is asking for trouble. Character data in any other encoding should be as byte data.
If you are reading the parameter using the servlet API, then it is likely that the HTTP request contains inconsistent or insufficient encoding information. Check the calling code and the HTTP headers. It is likely that the client is encoding the data as UTF-8, but the servlet is decoding it as ISO-8859-1.