First, obtain the integer value from the string of a, noting that a is expressed in hexadecimal:

a_int = int(a, 16)

Next, convert this int to a character. In python 2 you need to use the unichr method to do this, because the chr method can only deal with ASCII characters:

a_chr = unichr(a_int)

Whereas in python 3 you can just use the chr method for any character:

a_chr = chr(a_int)

So, in python 3, the full command is:

a_chr = chr(int(a, 16))

Try(Python 3.x):

import codecs
codecs.decode("707974686f6e2d666f72756d2e696f", "hex").decode('utf-8')

From here.

Try

line = line.decode('unicode_escape').encode('latin-1').decode('utf8')

The problem with

msg = u'\xe3\x80\x90\xe4\xb8\xad\xe5\xad\x97\xe3\x80\x91'
result = msg.decode('utf8')

is that you are trying to decode Unicode. That doesn't really make sense. You can encode from Unicode to some type of encoding, or you can decode a byte string to Unicode.

When you do

msg.decode('utf8')

Python 2 sees that msg is Unicode. It knows that it can't decode Unicode so it "helpfully" assumes that you want to encode msg with the default ASCII codec so the result of that transformation can be decoded to Unicode using the UTF-8 codec. Python 3 behaves much more sensibly: that code would simply fail with

AttributeError: 'str' object has no attribute 'decode'

The technique given in kennytm's answer:

msg.encode('latin1').decode('utf-8')

works because the Unicode codepoints less than 256 correspond directly to the characters in the Latin1 encoding (aka ISO 8859-1).

Here's some Python 2 code that illustrates this:

for i in xrange(256):
    lat = chr(i)
    uni = unichr(i)
    assert lat == uni.encode('latin1')
    assert lat.decode('latin1') == uni

And here is the equivalent Python 3 code:

for i in range(256):
    lat = bytes([i])
    uni = chr(i)
    assert lat == uni.encode('latin1')
    assert lat.decode('latin1') == uni

You may find this article helpful: Pragmatic Unicode, which was written by SO veteran Ned Batchelder.

Unless you are forced to use Python 2 I strongly advise you to switch to Python 3. It will make handling Unicode far less painful.

Use the built-in function chr() to convert the number to character, then encode that:

>>> chr(int('fd9b', 16)).encode('utf-8')
'\xef\xb6\x9b'

This is the string itself. If you want the string as ASCII hex, you'd need to walk through and convert each character c to hex, using hex(ord(c)) or similar.

Note: If you are still stuck with Python 2, you can use unichr() instead.

Your data is encoded as UTF-8, which means that you sometimes have to look at more than one byte to get one character. The easiest way to do this is probably to decode your string into a sequence of bytes, and then decode those bytes into a string. Python has built-in features for both:

value = bytes.fromhex("54 C3 BC").decode("utf-8")

Are you sure code is actually encoded in UTF-8? This works for me:

>>> b'\xee\x90\x95'.decode('utf-8')
u'\ue415'

Where ever you decoded the original string, it was likely decoded with latin-1 or a close relative. Since latin-1 is the first 256 codepoints of Unicode, this works:

>>> s = u'Gaga\xe2\x80\x99s'
>>> s.encode('latin-1').decode('utf8')
u'Gaga\u2019s'

When you do string.encode('utf-8'), it changes to hex notation.

But if you print it, you will get original unicode string.

If you want the hex notation you can get it like this with repr() function:

>>> print u'\u041a\u0418\u0421\u0410'.encode('utf-8')
КИСА
>>> print repr(u'\u041a\u0418\u0421\u0410'.encode('utf-8'))
'\xd0\x9a\xd0\x98\xd0\xa1\xd0\x90'

Python 3 does the right thing of inserting a character into a str which is string of characters, not a byte sequence.

UTF8 is the default encoding. If you need to insert a byte, a different encoding where that character is represented as a byte is needed.

$ PYTHONIOENCODING=iso-8859-1 python3 -c 'print("\x80")' | xxd
00000000: 800a

PYTHONIOENCODING

If this is set before running the interpreter, it overrides the encoding used for stdin/stdout/stderr, in the syntax encodingname:errorhandler. Both the encodingname and the :errorhandler parts are optional and have the same meaning as in str.encode().