One way is to use re.sub, that's my preferred way.

import re
my_str = "hey th~!ere"
my_new_string = re.sub('[^a-zA-Z0-9 \n\.]', '', my_str)
print my_new_string

Output:

hey there

Another way is to use re.escape:

import string
import re

my_str = "hey th~!ere"

chars = re.escape(string.punctuation)
print re.sub('['+chars+']', '',my_str)

Output:

hey there

Just a small tip about parameters style in python by PEP-8 parameters should be remove_special_chars and not removeSpecialChars

Also if you want to keep the spaces just change [^a-zA-Z0-9 \n\.] to [^a-zA-Z0-9\n\.]

import re
s=re.sub("[_list of special characters_]","",_your string goes here_)
print(s)

An example for this..

str="Hello$@& Python3$"
import re
s=re.sub("[$@&]","",str)
print (s)
#Output:Hello Python3

Explanation goes here..

s=re.sub("[$@&]","",s)

• Pattern to be replaced → “[$@&]”
• [] used to indicate a set of characters
• [$@&] → will match either $ or @ or &
• The replacement string is given as an empty string
• If these characters are found in the string, they’ll be replaced with an empty string

Put ('_**', '=') before ('*_', '(') as more characters match should be higher preference.

reps = (('_**', '='), ('_&', ' '), ('*_', '('), ('_*', ')'), ('#_', '{'), ('_#', '}'), ('_##', ';'), ('_$', '.'),
        ('_$$', ','))

Output

'AN(Gen=Air Outlet.Temp'

I agree with zigg, you'll need reg ex for that. You can find a gentle introduction here: http://www.diveintopython.net/regular_expressions/index.html#re.intro (check 'Street Address' case study, it shares some similarities with what you want to do).

EDIT:

I'm not a reg ex guru, but...

import re

pattern = "[- ]+"
re.sub(pattern, " ", your_string)

This will parse your first two examples. I'm not sure if all you need can be done in a single pattern... May some wise men come along.

Assuming you mean to change everything non-alphanumeric, you can do this on the command line:

cat foo.txt | sed "s/[^A-Za-z0-99]/ /g" > bar.txt

Or in Python with the re module:

import re
original_string = open('foo.txt').read()
new_string = re.sub('[^a-zA-Z0-9\n\.]', ' ', original_string)
open('bar.txt', 'w').write(new_string)

If, s=url['title'] makes s equal to this:

In [48]: s=u'Oscar Winners Best Pictures Box Set \xc2\xa36.49'

Then the problem is

1. in the code that defines url,
2. or else the content from the web is mal-formed.

If Case 1, we'd need to see the code that defines url.

If Case 2, a quick-and-dirty workaround would be to encode the unicode object s with the raw-unicode-escape codec:

In [49]: print(s)
Oscar Winners Best Pictures Box Set £6.49

In [50]: print(s.encode('raw-unicode-escape'))
Oscar Winners Best Pictures Box Set £6.49

See also this SO question.

Regarding titles like s=u'Star Trek XI £3.99': Again, it would be nice fix the problem before it gets to this stage -- perhaps by looking at how url is defined. But assuming the content from the web is mal-formed, a workaround would be:

In [86]: import re

In [87]: print(re.sub(r'&#x([a-fA-F\d]+);',lambda m: unichr(int(m.group(1),base=16)),s))
Star Trek XI £3.99

A little bit of explanation:

Note that

In [51]: x=u'£'
In [53]: x.encode('utf-8')
Out[53]: '\xc2\xa3'

So the unicode object u'£', encoded with the utf-8 codec, becomes the string object '\xc2\xa3'.

Somehow, url['title'] is getting defined to be the unicode object u'\xc2\xa3'. (The u makes a big difference!)

Thus we have u'\xc2\xa3' when we desire '\xc2\xa3'. Encoding the unicode object u'\xc2\xa3' with the raw-unicode-escape codec transforms it to '\xc2\xa3'.

Thanks to The fourth bird and hojkas !

I understood what was going wrong and i finally ended up with this working code (for any special characters but keep spaces) :

titre = '4K UL*tra & H~~~~D | SAMS!!!!UNG U,HD De;mo׃ LE$D T^^V'
print('original : '+ titre)
for i in titre:
    if not i.isalnum() and not i.isspace():
        titre=titre.replace(i,'')
print('modified : '+ titre)

#Output : 
#original : 4K UL*tra & H~~~~D | SAMS!!!!UNG U,HD De;mo׃ LE$D T^^V
#modified : 4K ULtra  HD  SAMSUNG UHD Demo LED TV

str.replace is not an in-place function, meaning when you call it, it returns a value that you must assign back to the original variable, otherwise the changes will not be seen. As an example, consider:

In [315]: string = 'æøå te/st-'.upper()

Now, call .replace:

In [316]: string.replace('Ø', 'OE')
Out[316]: 'ÆOEÅ TE/ST-'

In [317]: string
Out[317]: 'ÆØÅ TE/ST-'

No change. Try assigning it back now:

In [318]: string = string.replace('Ø', 'OE')

In [319]: string
Out[319]: 'ÆOEÅ TE/ST-'

As a faster alternative, consider the use of str.translate. If you're on python3, you can pass a dictionary mapping of replacements (you cannot do this on python2).

class Formatter(object):    
    @classmethod
    def string(self, strn):
        tab = dict.fromkeys(' -/', '_')
        tab.update({'Ø' : 'OE', 'Å' : 'AA', 'Æ' : 'AE'})

        return strn.upper().translate(str.maketrans(tab))

For python2, you could choose to stick with str.replace.