One way is to use re.sub, that's my preferred way.

import re
my_str = "hey th~!ere"
my_new_string = re.sub('[^a-zA-Z0-9 \n\.]', '', my_str)
print my_new_string

Output:

hey there

Another way is to use re.escape:

import string
import re

my_str = "hey th~!ere"

chars = re.escape(string.punctuation)
print re.sub('['+chars+']', '',my_str)

Output:

hey there

Just a small tip about parameters style in python by PEP-8 parameters should be remove_special_chars and not removeSpecialChars

Also if you want to keep the spaces just change [^a-zA-Z0-9 \n\.] to [^a-zA-Z0-9\n\.]

Put ('_**', '=') before ('*_', '(') as more characters match should be higher preference.

reps = (('_**', '='), ('_&', ' '), ('*_', '('), ('_*', ')'), ('#_', '{'), ('_#', '}'), ('_##', ';'), ('_$', '.'),
        ('_$$', ','))

Output

'AN(Gen=Air Outlet.Temp'

If, s=url['title'] makes s equal to this:

In [48]: s=u'Oscar Winners Best Pictures Box Set \xc2\xa36.49'

Then the problem is

1. in the code that defines url,
2. or else the content from the web is mal-formed.

If Case 1, we'd need to see the code that defines url.

If Case 2, a quick-and-dirty workaround would be to encode the unicode object s with the raw-unicode-escape codec:

In [49]: print(s)
Oscar Winners Best Pictures Box Set £6.49

In [50]: print(s.encode('raw-unicode-escape'))
Oscar Winners Best Pictures Box Set £6.49

See also this SO question.

Regarding titles like s=u'Star Trek XI £3.99': Again, it would be nice fix the problem before it gets to this stage -- perhaps by looking at how url is defined. But assuming the content from the web is mal-formed, a workaround would be:

In [86]: import re

In [87]: print(re.sub(r'&#x([a-fA-F\d]+);',lambda m: unichr(int(m.group(1),base=16)),s))
Star Trek XI £3.99

A little bit of explanation:

Note that

In [51]: x=u'£'
In [53]: x.encode('utf-8')
Out[53]: '\xc2\xa3'

So the unicode object u'£', encoded with the utf-8 codec, becomes the string object '\xc2\xa3'.

Somehow, url['title'] is getting defined to be the unicode object u'\xc2\xa3'. (The u makes a big difference!)

Thus we have u'\xc2\xa3' when we desire '\xc2\xa3'. Encoding the unicode object u'\xc2\xa3' with the raw-unicode-escape codec transforms it to '\xc2\xa3'.

Assuming you mean to change everything non-alphanumeric, you can do this on the command line:

cat foo.txt | sed "s/[^A-Za-z0-99]/ /g" > bar.txt

Or in Python with the re module:

import re
original_string = open('foo.txt').read()
new_string = re.sub('[^a-zA-Z0-9\n\.]', ' ', original_string)
open('bar.txt', 'w').write(new_string)

This can be done without regex:

>>> string = "Special $#! characters   spaces 888323"
>>> ''.join(e for e in string if e.isalnum())
'Specialcharactersspaces888323'

You can use str.isalnum:

S.isalnum() -> bool

Return True if all characters in S are alphanumeric
and there is at least one character in S, False otherwise.

If you insist on using regex, other solutions will do fine. However note that if it can be done without using a regular expression, that's the best way to go about it.

import re
s=re.sub("[_list of special characters_]","",_your string goes here_)
print(s)

An example for this..

str="Hello$@& Python3$"
import re
s=re.sub("[$@&]","",str)
print (s)
#Output:Hello Python3

Explanation goes here..

s=re.sub("[$@&]","",s)

• Pattern to be replaced → “[$@&]”
• [] used to indicate a set of characters
• [$@&] → will match either $ or @ or &
• The replacement string is given as an empty string
• If these characters are found in the string, they’ll be replaced with an empty string