#!/usr/bin/env python
# -*- coding: utf-8 -*-

import unicodedata
text = u'Cześć'
print unicodedata.normalize('NFD', text).encode('ascii', 'ignore')
Answer from nosklo on Stack Overflow
🌐
Iditect
iditect.com › faq › python › replace-special-characters-with-ascii-equivalent-in-python.html
Replace special characters with ASCII equivalent in python
To replace special characters with their ASCII equivalents in Python, you can use the unicodedata module, which provides a way to normalize and replace characters with their ASCII counterparts.
Discussions

python - Turn special characters into ascii-like characters or someting else without losing readability - Stack Overflow
Trying to format data from ics calendar file to any outpu such as json or even python print(). Looking for good ways to replace special characters without losing readability and having ascii-like characters. Examples below. More on stackoverflow.com
🌐 stackoverflow.com
May 6, 2021
Replace special characters in a file with their unicode code (Python) - Stack Overflow
I have an interesting question. Suppose to have a list of chars with their matching representation in unicode: list_of_special_chars = [ ('Ą', b'\u0104'), ('ą', b'\u0105'), ('Ę', b'\u0... More on stackoverflow.com
🌐 stackoverflow.com
March 31, 2017
string - Replace special characters in python - Stack Overflow
UnicodeEncodeError: 'ascii' codec can't encode characters in position 100-101: ordinal not in range(128) – Marcos Placona Commented Jan 16, 2011 at 14:33 · You should post repr() of the string that this fails for and the particular line (i.e. does it fail on the decode?). ... Find the answer to your question by asking. Ask question ... See similar questions with ... More on stackoverflow.com
🌐 stackoverflow.com
November 8, 2011
python - encoding issue. Replace special character - Stack Overflow
And you're trying to replace it with another invalid ASCII character and that will cause the same exception. Hence why you get the "'ascii' codec can't decode byte...' message. ... This will tell Python to find a unicode string, and replace it with another unicode string. But the output data will result in the same thing in the example ... More on stackoverflow.com
🌐 stackoverflow.com
People also ask

How do I remove non-ASCII characters in Python?
Encode the string with ASCII and an explicit error policy such as ignore or replace, then decode it, understanding that information may be lost.
🌐
pythonpool.com
pythonpool.com › home › tutorials › remove unicode characters in python: ascii, symbols, and normalization
Remove Unicode Characters in Python: ASCII, Symbols, and Normalization
How do I remove only selected Unicode symbols?
Use str.translate() with a deletion mapping or a carefully defined regular expression rather than deleting every non-ASCII character.
🌐
pythonpool.com
pythonpool.com › home › tutorials › remove unicode characters in python: ascii, symbols, and normalization
Remove Unicode Characters in Python: ASCII, Symbols, and Normalization
Why should I avoid removing all Unicode?
Unicode includes valid letters, punctuation, and scripts; broad deletion can corrupt names, identifiers, or user content when the real requirement is narrower.
🌐
pythonpool.com
pythonpool.com › home › tutorials › remove unicode characters in python: ascii, symbols, and normalization
Remove Unicode Characters in Python: ASCII, Symbols, and Normalization
Top answer
1 of 2
1

The data for the .ics file should not be decoded, but passed directly to .from_ical. Use res.content instead. Then Calendar generates the data decoded correctly as UTF-8 (probably part of the .ICS spec) and print can print Unicode strings correctly. For the JSON, write with utf8 encoding and ensure_ascii=False as @JosefZ recommended to see it correctly as well:

import requests
import json
from icalendar import Calendar

url = 'http://www.formula1.com/calendar/Formula_1_Official_Calendar.ics'
res = requests.get(url)
calendar = Calendar.from_ical(res.content)
events = [
    {
        'id': event['UID'].split('@')[-1].strip(),
        'startTime': event['DTSTART'].dt.strftime('%Y-%m-%dT%H:%M:%S.%f')[:-3],
        'summary': event['SUMMARY']
    } for event in calendar.walk('VEVENT') if str(event['UID']).split('@')[0].startswith('Race')]

for event in events:
    print(event['summary'])

with open('events.json', 'w', encoding='utf8') as f:
    json.dump(events, f, ensure_ascii=False, indent=2)

print Output:

FORMULA 1 GULF AIR BAHRAIN GRAND PRIX 2021 - Race
FORMULA 1 PIRELLI GRAN PREMIO DEL MADE IN ITALY E DELL'EMILIA ROMAGNA 2021 - Race
FORMULA 1 HEINEKEN GRANDE PRÉMIO DE PORTUGAL 2021 - Race
FORMULA 1 ARAMCO GRAN PREMIO DE ESPAÑA 2021 - Race
FORMULA 1 GRAND PRIX DE MONACO 2021 - Race
FORMULA 1 AZERBAIJAN GRAND PRIX 2021 - Race
FORMULA 1 HEINEKEN GRAND PRIX DU CANADA 2021 - Race
FORMULA 1 EMIRATES GRAND PRIX DE FRANCE 2021 - Race
FORMULA 1 MYWORLD GROSSER PREIS VON ÖSTERREICH 2021 - Race
FORMULA 1 PIRELLI BRITISH GRAND PRIX 2021 - Race
FORMULA 1 MAGYAR NAGYDÍJ 2021 - Race
FORMULA 1 ROLEX BELGIAN GRAND PRIX 2021 - Race
FORMULA 1 HEINEKEN DUTCH GRAND PRIX 2021 - Race
FORMULA 1 HEINEKEN GRAN PREMIO D’ITALIA 2021 - Race
FORMULA 1 VTB RUSSIAN GRAND PRIX 2021 - Race
FORMULA 1 SINGAPORE AIRLINES SINGAPORE GRAND PRIX 2021 - Race
FORMULA 1 JAPANESE GRAND PRIX 2021 - Race
FORMULA 1 ARAMCO UNITED STATES GRAND PRIX 2021 - Race
FORMULA 1 GRAN PREMIO DE LA CIUDAD DE MÉXICO 2021 - Race
FORMULA 1 HEINEKEN GRANDE PRÊMIO DE SÃO PAULO 2021 - Race
FORMULA 1 ROLEX AUSTRALIAN GRAND PRIX 2021 - Race
FORMULA 1 SAUDI ARABIAN GRAND PRIX 2021 - Race
FORMULA 1 ETIHAD AIRWAYS ABU DHABI GRAND PRIX 2021 - Race

events.json:

[
  {
    "id": "1064",
    "startTime": "2021-03-28T16:00:00.000",
    "summary": "FORMULA 1 GULF AIR BAHRAIN GRAND PRIX 2021 - Race"
  },
  {
    "id": "1065",
    "startTime": "2021-04-18T14:00:00.000",
    "summary": "FORMULA 1 PIRELLI GRAN PREMIO DEL MADE IN ITALY E DELL'EMILIA ROMAGNA 2021 - Race"
  },
  {
    "id": "1066",
    "startTime": "2021-05-02T15:00:00.000",
    "summary": "FORMULA 1 HEINEKEN GRANDE PRÉMIO DE PORTUGAL 2021 - Race"
  },
  {
    "id": "1086",
    "startTime": "2021-05-09T14:00:00.000",
    "summary": "FORMULA 1 ARAMCO GRAN PREMIO DE ESPAÑA 2021 - Race"
  },
  {
    "id": "1067",
    "startTime": "2021-05-23T14:00:00.000",
    "summary": "FORMULA 1 GRAND PRIX DE MONACO 2021 - Race"
  },
  {
    "id": "1068",
    "startTime": "2021-06-06T13:00:00.000",
    "summary": "FORMULA 1 AZERBAIJAN GRAND PRIX 2021 - Race"
  },
  {
    "id": "1069",
    "startTime": "2021-06-13T19:00:00.000",
    "summary": "FORMULA 1 HEINEKEN GRAND PRIX DU CANADA 2021 - Race"
  },
  {
    "id": "1070",
    "startTime": "2021-06-27T14:00:00.000",
    "summary": "FORMULA 1 EMIRATES GRAND PRIX DE FRANCE 2021 - Race"
  },
  {
    "id": "1071",
    "startTime": "2021-07-04T14:00:00.000",
    "summary": "FORMULA 1 MYWORLD GROSSER PREIS VON ÖSTERREICH 2021 - Race"
  },
  {
    "id": "1072",
    "startTime": "2021-07-18T15:00:00.000",
    "summary": "FORMULA 1 PIRELLI BRITISH GRAND PRIX 2021 - Race"
  },
  {
    "id": "1073",
    "startTime": "2021-08-01T14:00:00.000",
    "summary": "FORMULA 1 MAGYAR NAGYDÍJ 2021 - Race"
  },
  {
    "id": "1074",
    "startTime": "2021-08-29T14:00:00.000",
    "summary": "FORMULA 1 ROLEX BELGIAN GRAND PRIX 2021 - Race"
  },
  {
    "id": "1075",
    "startTime": "2021-09-05T14:00:00.000",
    "summary": "FORMULA 1 HEINEKEN DUTCH GRAND PRIX 2021 - Race"
  },
  {
    "id": "1076",
    "startTime": "2021-09-12T14:00:00.000",
    "summary": "FORMULA 1 HEINEKEN GRAN PREMIO D’ITALIA 2021 - Race"
  },
  {
    "id": "1077",
    "startTime": "2021-09-26T13:00:00.000",
    "summary": "FORMULA 1 VTB RUSSIAN GRAND PRIX 2021 - Race"
  },
  {
    "id": "1078",
    "startTime": "2021-10-03T13:00:00.000",
    "summary": "FORMULA 1 SINGAPORE AIRLINES SINGAPORE GRAND PRIX 2021 - Race"
  },
  {
    "id": "1079",
    "startTime": "2021-10-10T06:00:00.000",
    "summary": "FORMULA 1 JAPANESE GRAND PRIX 2021 - Race"
  },
  {
    "id": "1080",
    "startTime": "2021-10-24T20:00:00.000",
    "summary": "FORMULA 1 ARAMCO UNITED STATES GRAND PRIX 2021 - Race"
  },
  {
    "id": "1081",
    "startTime": "2021-10-31T19:00:00.000",
    "summary": "FORMULA 1 GRAN PREMIO DE LA CIUDAD DE MÉXICO 2021 - Race"
  },
  {
    "id": "1082",
    "startTime": "2021-11-07T17:00:00.000",
    "summary": "FORMULA 1 HEINEKEN GRANDE PRÊMIO DE SÃO PAULO 2021 - Race"
  },
  {
    "id": "1083",
    "startTime": "2021-11-21T06:00:00.000",
    "summary": "FORMULA 1 ROLEX AUSTRALIAN GRAND PRIX 2021 - Race"
  },
  {
    "id": "1085",
    "startTime": "2021-12-05T16:00:00.000",
    "summary": "FORMULA 1 SAUDI ARABIAN GRAND PRIX 2021 - Race"
  },
  {
    "id": "1084",
    "startTime": "2021-12-12T13:00:00.000",
    "summary": "FORMULA 1 ETIHAD AIRWAYS ABU DHABI GRAND PRIX 2021 - Race"
  }
]
2 of 2
1
with open("events.json", mode="w", encoding="utf-8") as f:
    json.dump(events, f, indent=2, ensure_ascii=False)

From json.dump docs:

If ensure_ascii is true (the default), the output is guaranteed to have all incoming non-ASCII characters escaped. If ensure_ascii is false, these characters will be output as-is.

Used encoding="utf-8" in open as the default encoding is platform dependent (whatever locale.getpreferredencoding() returns).

🌐
Coderanch
coderanch.com › t › 709349 › languages › Replace-special-characters
Replace special characters [Solved] (Jython/Python forum at Coderanch)
Nevertheless, I tried s = s.replace(u'ç', '') but without sucess ... I assume that when you say "doesn't work" that you actually mean that the characters in question are not removed from the text. Anyway I'm surprised that the "@" character is one of those so-called "special" characters which isn't handled correctly. That means it's not a Unicode or character set issue because that's an ordinary ASCII character. However since I'm no Jython (or Python...
🌐
Delft Stack
delftstack.com › home › howto › python › python unicode to string
How to Convert Unicode Characters to ASCII String in Python | Delft Stack
February 2, 2024 - In this case, the characters ä and å are replaced with a, and the resulting string is b'Har ar ett exempel pa en svensk mening att ge dig.'. The byte literal can be further decoded to obtain a plain ASCII string if needed.
🌐
Programiz
programiz.com › python-programming › methods › built-in › ascii
Python ascii() (With Examples)
In the above example, we have used the ascii() method with a set. The method takes the individual non-printable characters in the set as arguments and replaces them with their corresponding ascii values.
Find elsewhere
🌐
py4u
py4u.org › blog › python-replace-typographical-quotes-dashes-etc-with-their-ascii-counterparts
How to Replace Typographical Quotes, Dashes, and Special Characters with ASCII Counterparts in Python (While Preserving Umlauts and Non-ASCII Text)
Instead, we’ll use Python’s built-in string manipulation tools to create a custom translation that targets only the typographical characters we want to replace. First, we need to list all the typographical characters we want to replace and their desired ASCII counterparts.
🌐
Python Pool
pythonpool.com › home › tutorials › remove unicode characters in python: ascii, symbols, and normalization
Remove Unicode Characters in Python: ASCII, Symbols, and Normalization
July 13, 2026 - text = "price " + chr(0x20AC) + " 10" clean = text.encode("ascii", "replace").decode("ascii") print(clean) This is helpful during audits because it makes removed characters visible.
🌐
Scaler
scaler.com › home › topics › remove special characters from string python
Remove Special Characters From String Python - Scaler Topics
January 6, 2024 - The output for the above example will be, The maketrans() method saves the characters as their respective ASCII values. For example, the ASCII value of h and b is 104 and 98 respectively. Let us see how to use this mapping table created using the maketrans() function to remove specific special ...
🌐
CodeRivers
coderivers.org › blog › python-replace-accented-character-with-ascii-character
Python: Replacing Accented Characters with ASCII Characters - CodeRivers
February 22, 2026 - For example, when reading a file, you can specify the encoding like open('file.txt', 'r', encoding='utf - 8'). Output Encoding: If the output needs to be in a specific encoding (e.g., for writing to a file or sending over a network in a particular format), make sure to encode the resulting ASCII-only string correctly. Replacing accented characters with ASCII characters in Python is a useful operation in various programming scenarios.
🌐
W3Schools
w3schools.com › python › ref_func_ascii.asp
Python ascii() Function
Python Examples Python Compiler ... version of any object (Strings, Tuples, Lists, etc). The ascii() function will replace any non-ascii characters with escape characters:...
🌐
Medium
medium.com › @ryan_forrester_ › python-string-replace-how-to-guide-ac6884add7b3
Python String Replace: How To Guide | by ryan | Medium
October 23, 2024 - When working with Unicode text, be aware of character encoding: def clean_unicode_text(text): # Replace common Unicode quotation marks with ASCII ones replacements = { '"': '"', # U+201C LEFT DOUBLE QUOTATION MARK '"': '"', # U+201D RIGHT DOUBLE QUOTATION MARK ''': "'", # U+2018 LEFT SINGLE QUOTATION MARK ''': "'", # U+2019 RIGHT SINGLE QUOTATION MARK } for old, new in replacements.items(): text = text.replace(old, new) return text fancy_text = "Here's some "fancy" text" plain_text = clean_unicode_text(fancy_text) print(plain_text) # Output: Here's some "fancy" text
Top answer
1 of 2
7

If, s=url['title'] makes s equal to this:

In [48]: s=u'Oscar Winners Best Pictures Box Set \xc2\xa36.49'

Then the problem is

  1. in the code that defines url,
  2. or else the content from the web is mal-formed.

If Case 1, we'd need to see the code that defines url.

If Case 2, a quick-and-dirty workaround would be to encode the unicode object s with the raw-unicode-escape codec:

In [49]: print(s)
Oscar Winners Best Pictures Box Set £6.49

In [50]: print(s.encode('raw-unicode-escape'))
Oscar Winners Best Pictures Box Set £6.49

See also this SO question.


Regarding titles like s=u'Star Trek XI £3.99': Again, it would be nice fix the problem before it gets to this stage -- perhaps by looking at how url is defined. But assuming the content from the web is mal-formed, a workaround would be:

In [86]: import re

In [87]: print(re.sub(r'&#x([a-fA-F\d]+);',lambda m: unichr(int(m.group(1),base=16)),s))
Star Trek XI £3.99

A little bit of explanation:

Note that

In [51]: x=u'£'
In [53]: x.encode('utf-8')
Out[53]: '\xc2\xa3'

So the unicode object u'£', encoded with the utf-8 codec, becomes the string object '\xc2\xa3'.

Somehow, url['title'] is getting defined to be the unicode object u'\xc2\xa3'. (The u makes a big difference!)

Thus we have u'\xc2\xa3' when we desire '\xc2\xa3'. Encoding the unicode object u'\xc2\xa3' with the raw-unicode-escape codec transforms it to '\xc2\xa3'.

2 of 2
0

Edit: you have your objects already in unicode. Seems to me there is no reason to actually use enocde/decode at all.

>>> print u'Oscar Winners Best Pictures Box Set \xc2\xa36.49'.replace(u'Â','')
Oscar Winners Best Pictures Box Set £6.49

However it seems to me that something is wrong there. The unicode objects are actually not unicode; see:

>>> print 'Oscar Winners Best Pictures Box Set \xc2\xa36.49'.decode('utf8')
Oscar Winners Best Pictures Box Set £6.49

The repr() you posted should not be unicode object. That's why I was asking where are you getting the data, there is something wrong.

🌐
Stack Overflow
stackoverflow.com › questions › 41722907 › encoding-issue-replace-special-character
python - encoding issue. Replace special character - Stack Overflow
Normally, if you print u'Ganztags ge\xf6ffnet' it will be shown as a Ö because of automatic encoding, if you need to verify if input matches that string, you have to compare them u'ö' == u'ö', if other systems depend on this data, encode it with something they understand .encode('UTF-8'). But replacing \xf6 with ö is the same thing, just that ö doesn't exist in ASCII and you need to do u'ö' - which, will result in the same data at the end. ... Character ö has code \xf6 in both unicode and 8 bits Latin1 encoding also knows as iso-8859-1. Python have support for different 8bits encoding, and Latin1 is common for west european languages including Deutsch.
🌐
Python documentation
docs.python.org › 3 › library › re.html
re — Regular expression operations — Python 3.14.7 ...
However, Unicode strings and 8-bit strings cannot be mixed: that is, you cannot match a Unicode string with a bytes pattern or vice-versa; similarly, when asking for a substitution, the replacement string must be of the same type as both the pattern and the search string. Regular expressions use the backslash character ('\') to indicate special forms or to allow special characters to be used without invoking their special meaning. This collides with Python’s usage of the same character for the same purpose in string literals; for example, to match a literal backslash, one might have to write '\\\\' as the pattern string, because the regular expression must be \\, and each backslash must be expressed as \\ inside a regular Python string literal.
🌐
GitHub
gist.github.com › tushortz › 9fbde5d023c0a0204333267840b592f9
Function to replace some annoying characters · GitHub
Function to replace some annoying characters · Raw · UNICODE to ASCII python replace · This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
🌐
Inductive Automation
forum.inductiveautomation.com › general discussion
Replacing national characters in the string with ASCII symbols - General Discussion - Inductive Automation Forum
May 3, 2023 - We want to display logged in user on machine HMI. User name is retrieved from database and includes national characters. However, HMI accepts only ASCII symbols. How can I replace national symbols in Ignition string type tag with ASCII symbols. Something like this works in online Python interpreter but not in Ignition script console (the same string containing national characters is returned for output string) # create a dictionary with national characters and their ASCII equivalents translati...