This is why implicit conversion of byte strings to Unicode strings was removed in Python 3.

You're almost there, with the #coding line at the start of your file. Just one tiny change to turn your test character into a Unicode string:

if u'⸣' not in myvariable:
    newvariable = 100.0

You might have trouble with that particular character as I did on my system, so you can use the equivalent escape sequence instead:

if u'\u2e23' not in myvariable:
    newvariable = 100.0
Answer from Mark Ransom on Stack Overflow
🌐
Python documentation
docs.python.org › 3 › howto › unicode.html
Unicode HOWTO — Python 3.14.6 documentation
Similarly, \w matches a wide variety of Unicode characters but only [a-zA-Z0-9_] in bytes or if re.ASCII is supplied, and \s will match either Unicode whitespace characters or [ \t\n\r\f\v]. Some good alternative discussions of Python’s Unicode support are: Processing Text Files in Python 3, by Nick Coghlan.
Discussions

Python - How can I do a string find on a Unicode character that is a variable? - Stack Overflow
Or, if you already have a string in some other encoding, by converting it to Unicode (the original encoding must be specified if the string is non-ASCII). ... Or by using Python 3 where all strings are Unicode. More on stackoverflow.com
🌐 stackoverflow.com
How to check Unicode char in python - Stack Overflow
I try to find char non ASCII or ... but still find error, can somebody help me? ... This works if you have a Unicode string. If you just want to detect if the string contains a non-ASCII character it also works on a UTF-8 encoded byte string. Update for Python 3: the above ... More on stackoverflow.com
🌐 stackoverflow.com
python - Searching String for a Unicode Character - Stack Overflow
I'm using Python and I want to find a character in a string of text. The string looks like this: ☺G18M1329D3920,2511171♥7006. I'm interested in that heart. I know it's a binary 6 but how would I return it's position using a string method similar to .find()? ... When you say "binary" I think you mean "Unicode ... More on stackoverflow.com
🌐 stackoverflow.com
python - Find out the unicode script of a character - Stack Overflow
Given a unicode character what would be the simplest way to return its script (as "Latin", "Hangul" etc)? unicodedata doesn't seem to provide this kind of feature. More on stackoverflow.com
🌐 stackoverflow.com
🌐
GeeksforGeeks
geeksforgeeks.org › python › python-convert-string-to-unicode-characters
Python - Convert String to unicode characters - GeeksforGeeks
July 15, 2025 - These Unicode values are appended to the unicode list, resulting in [104, 101, 108, 108, 111] for the string "hello". In this method, each character in the string is converted to its Unicode (ASCII) value using ord(char).
🌐
DNMTechs
dnmtechs.com › finding-unicode-characters-in-python
Finding Unicode Characters in Python – DNMTechs – Sharing and Storing Technology Knowledge
# Define a string with Unicode characters text = "Hello, 你好, नमस्ते" # Find all Unicode characters in the string unicode_chars = [char for char in text if ord(char) > 127] print(unicode_chars)
Find elsewhere
🌐
Xah Lee
xahlee.info › python › unicodedata_module.html
Python: Get Unicode Name, Code Point
May 14, 2026 - from unicodedata import lookup # get character as string, from char name print(lookup("GREEK SMALL LETTER ALPHA")) # α print(lookup("BUTTERFLY")) # 🦋 print(lookup("RIGHTWARDS ARROW")) # → print(lookup("CJK UNIFIED IDEOGRAPH-5929")) # 天 ... Here's a example that prints a range of Unicode ...
Top answer
1 of 13
325

In Python 3, all strings are sequences of Unicode characters. There is a bytes type that holds raw bytes.

In Python 2, a string may be of type str or of type unicode. You can tell which using code something like this:

def whatisthis(s):
    if isinstance(s, str):
        print "ordinary string"
    elif isinstance(s, unicode):
        print "unicode string"
    else:
        print "not a string"

This does not distinguish "Unicode or ASCII"; it only distinguishes Python types. A Unicode string may consist of purely characters in the ASCII range, and a bytestring may contain ASCII, encoded Unicode, or even non-textual data.

2 of 13
136

How to tell if an object is a unicode string or a byte string

You can use type or isinstance.

In Python 2:

>>> type(u'abc')  # Python 2 unicode string literal
<type 'unicode'>
>>> type('abc')   # Python 2 byte string literal
<type 'str'>

In Python 2, str is just a sequence of bytes. Python doesn't know what its encoding is. The unicode type is the safer way to store text. If you want to understand this more, I recommend http://farmdev.com/talks/unicode/.

In Python 3:

>>> type('abc')   # Python 3 unicode string literal
<class 'str'>
>>> type(b'abc')  # Python 3 byte string literal
<class 'bytes'>

In Python 3, str is like Python 2's unicode, and is used to store text. What was called str in Python 2 is called bytes in Python 3.


How to tell if a byte string is valid utf-8 or ascii

You can call decode. If it raises a UnicodeDecodeError exception, it wasn't valid.

>>> u_umlaut = b'\xc3\x9c'   # UTF-8 representation of the letter 'Ü'
>>> u_umlaut.decode('utf-8')
u'\xdc'
>>> u_umlaut.decode('ascii')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128)
🌐
LearnByExample
learnbyexample.github.io › py_regular_expressions › unicode.html
Unicode - Understanding Python re(gex)?
# to get codepoints for ASCII characters >>> [ord(c) for c in 'fox'] [102, 111, 120] >>> [hex(ord(c)) for c in 'fox'] ['0x66', '0x6f', '0x78'] # to get codepoints for Unicode characters >>> [c.encode('unicode_escape') for c in 'αλεπού'] [b'\\u03b1', b'\\u03bb', b'\\u03b5', b'\\u03c0', b'\\u03bf', b'\\u03cd'] >>> [c.encode('unicode_escape') for c in 'İıſK'] [b'\\u0130', b'\\u0131', b'\\u017f', b'\\u212a'] # character range example using \u # English lowercase letters >>> re.findall(r'[\u0061-\u007a]+', 'fox:αλεπού,eagle:αετός') ['fox', 'eagle'] See also: codepoints.net, a site dedicated for Unicode characters. You can also specify a Unicode character using the \N{name} escape sequence. See unicode: NamesList for a full list of names. From the Python docs:
Top answer
1 of 10
175

To include Unicode characters in your Python source code, you can use Unicode escape characters in the form \u0123 in your string. In Python 2.x, you also need to prefix the string literal with 'u'.

Here's an example running in the Python 2.x interactive console:

>>> print u'\u0420\u043e\u0441\u0441\u0438\u044f'
Россия

In Python 2, prefixing a string with 'u' declares them as Unicode-type variables, as described in the Python Unicode documentation.

In Python 3, the 'u' prefix is now optional:

>>> print('\u0420\u043e\u0441\u0441\u0438\u044f')
Россия

If running the above commands doesn't display the text correctly for you, perhaps your terminal isn't capable of displaying Unicode characters.

These examples use Unicode escapes (\u...), which allows you to print Unicode characters while keeping your source code as plain ASCII. This can help when working with the same source code on different systems. You can also use Unicode characters directly in your Python source code (e.g. print u'Россия' in Python 2), if you are confident all your systems handle Unicode files properly.

For information about reading Unicode data from a file, see this answer:

Character reading from file in Python

2 of 10
54

Print a unicode character in Python:

Print a unicode character directly from python interpreter:

el@apollo:~$ python
Python 2.7.3
>>> print u'\u2713'

Unicode character u'\u2713' is a checkmark. The interpreter prints the checkmark on the screen.

Print a unicode character from a python script:

Put this in test.py:

#!/usr/bin/python
print("here is your checkmark: " + u'\u2713');

Run it like this:

el@apollo:~$ python test.py
here is your checkmark: ✓

If it doesn't show a checkmark for you, then the problem could be elsewhere, like the terminal settings or something you are doing with stream redirection.

Store unicode characters in a file:

Save this to file: foo.py:

#!/usr/bin/python -tt
# -*- coding: utf-8 -*-
import codecs
import sys 
UTF8Writer = codecs.getwriter('utf8')
sys.stdout = UTF8Writer(sys.stdout)
print(u'e with obfuscation: é')

Run it and pipe output to file:

python foo.py > tmp.txt

Open tmp.txt and look inside, you see this:

el@apollo:~$ cat tmp.txt 
e with obfuscation: é

Thus you have saved unicode e with a obfuscation mark on it to a file.

🌐
Note.nkmk.me
note.nkmk.me › home › python
Convert Between Unicode Code Point and Character: chr, ord | note.nkmk.me
January 31, 2024 - In Python, the built-in chr() and ord() functions allow you to convert between Unicode code points and characters. Additionally, in string literals, characters can be represented by their hexadecimal ...
🌐
Stanford
web.stanford.edu › class › archive › cs › cs106a › cs106a.1202 › handouts › py-string.html
Py Strings
You can write a unicode char out in a Python string with a \u followed by the 4 hex digits of its code point. Notice how each unicode char is just one more character in the string:
🌐
Reddit
reddit.com › r/learnpython › replacing literal '\u****' in string with corresponding unicode character
r/learnpython on Reddit: Replacing literal '\u****' in string with corresponding Unicode character
March 13, 2020 -

Instead of scraping entirely via xpath like I have in the past, I decided to pull the data out of a clearly visible var in a <script> element and write a function to parse it as a dictionary recursively*.

My problem is that many characters are escaped, so instead of https:// I have https:\u002F\u002F, literally -- 12 characters instead of those 2 forward slashes, the \u**** sequence doesn't stand for the frontslashes. I have this problem in body text too, and because there are lots of diacritical marks I can't get away with replacing them bluntly. I need to replace these substrings with the corresponding characters, so \u002F -> /, \u00C9 -> é, etc.

It seems like there should be an obvious and not-ridiculous solution for this, but I don't know what it is and googling turns up different problems. Is this something I need to anticipate at the scraping stage? I'm using the requests and html modules if it matters.

*Which is dumb because it was already formatted just like a dict, so I could probably just have saved the raw text in a .py and imported it as a module

🌐
Python Cheat Sheet
pythonsheets.com › notes › basic › python-unicode.html
Unicode — Python Cheat Sheet
For example, the character, é can be written as e ́ (Canonical Decomposition) or é (Canonical Composition). In this case, we may acquire unexpected results when we are comparing two strings even though they look alike. Therefore, we can normalize a Unicode form to solve the issue. # python 3 >>> u1 = 'Café' # unicode string >>> u2 = 'Cafe\u0301' >>> u1, u2 ('Café', 'Café') >>> len(u1), len(u2) (4, 5) >>> u1 == u2 False >>> u1.encode('utf-8') # get u1 byte string b'Caf\xc3\xa9' >>> u2.encode('utf-8') # get u2 byte string b'Cafe\xcc\x81' >>> from unicodedata import normalize >>> s1 = norm
🌐
Emacsos
blog.emacsos.com › unicode-in-python.html
Handling Unicode Strings in Python - Yuanle's blog
August 25, 2016 - In python, text could be presented using unicode string or bytes. Unicode is a standard for encoding character. Unicode string is a python data structure that can store zero or more unicode characters. Unicode string is designed to store text data. On the other hand, bytes are just a serial ...
🌐
Python Guides
pythonguides.com › remove-unicode-characters-in-python
How to Remove Unicode Characters in Python
September 3, 2025 - This method gives me flexibility when I want to filter out specific ranges of characters. If I want to normalize text (for example, keep “é” but convert it into “e”), I use the unicodedata module.