string = "\x22my quote\x22"
print(string)
You don't need to decode, Python 3 does that for you, but you need the correct control character for the double quote "
If however you have a different character set, it appears you have Windows-1252, then you need to decode the byte string from that character set:
str(b"\x94my quote\x94", "windows-1252")
If your string isn't a byte string you have to encode it first, I found the latin-1 encoding to work:
string = "\x94my quote\x94"
str(string.encode("latin-1"), "windows-1252")
string = "\x22my quote\x22"
print(string)
You don't need to decode, Python 3 does that for you, but you need the correct control character for the double quote "
If however you have a different character set, it appears you have Windows-1252, then you need to decode the byte string from that character set:
str(b"\x94my quote\x94", "windows-1252")
If your string isn't a byte string you have to encode it first, I found the latin-1 encoding to work:
string = "\x94my quote\x94"
str(string.encode("latin-1"), "windows-1252")
I don't know if you mean to this, but this works:
some_binary = a = b"\x94my quote\x94"
result = some_binary.decode()
And you got the result...
If you don't know which encoding to choose, you can use chardet.detect:
import chardet
chardet.detect(some_binary)
I'm working on a project where I'm trying to get someone's old Python 2 code updated to work in Python 3, and one of the functions they used was "mystring.decode('hex'). Now I'm trying to figure out how to replicate this functionality in Python 3 and I'm having a hell of a time wrestling with it. Why did Python ditch this?
This is called "unicode-escape" encoding. Here is an example of how one would achieve this behavior in python3:
In [11]: c = b'\xe5\xb8\x90\xe6\x88\xb7'
In [12]: d = c.decode('utf8')
In [13]: print(d)
帐户
In [14]: print(d.encode('unicode-escape').decode('ascii'))
\u5e10\u6237
If you want it as bytes and not str, you can simply get rid of the .decode('ascii').
Returning the same unicode as in python2 is not possible : I have not seen unicode object like there was in python2, in python3. But it is possible to get the value of the unicode object.
To do this, you need to do several things :
- Create a byte element with value '\xe5\xb8\x90\xe6\x88\xb7'
- Transform this byte element into a string
- Gets the unicode code from the string
The first step is quite easy. To create a byte element 'c' with the same value as your c, just do :
c = b'\xe5\xb8\x90\xe6\x88\xb7'
Then, to read the element
c_string = c.decode() # default encoding is utf-8
Finally, I created a function to transform a string into its character + unicode representation
def get_unicode_code(text):
result = ""
for char in text:
ord_value = ord(char)
if ord_value < 128:
result += char
else:
hex_string = format(ord_value, "x") # turning the int into its hex value
if len(hex_string) == 2:
unicode_code = "\\x"+hex_string
elif len(hex_string) == 3:
unicode_code = "\\u0"+hex_string
else:
unicode_code = "\\u"+hex_string
result += unicode_code
return result
get_unicode_code(d) will return the same as d.encode('unicode-escape').decode('ascii'), though it is most likely less efficient.
It takes a string as an argument and returns a string with the unicode instead of the character it represents.
First find the encoding of the string and then decode it... to do this you will need to make a byte string by adding the letter 'b' to the front of the original string.
Try this:
import chardet
s = "Aur\xc3\xa9lien"
bs = b"Aur\xc3\xa9lien"
encoding = chardet.detect(bs)["encoding"]
str = s.encode(encoding).decode("utf-8")
print(str)
If you are reading the text from a file you can detect the encoding using the magic lib, see here: https://stackoverflow.com/a/16203777/1544937
You have UTF-8 decoded as latin-1, so the solution is to encode as latin-1 then decode as UTF-8.
s = "Aur\xc3\xa9lien"
s.encode('latin-1').decode('utf-8')
print(s.encode('latin-1').decode('utf-8'))
Output
Aurélien