we dont know how your data look like:
But you can use re.sub:
import re
your_string = re.sub(r'[\\/*?:"<>|]',"","your_string")
Answer from Hackaholic on Stack Overflowwe dont know how your data look like:
But you can use re.sub:
import re
your_string = re.sub(r'[\\/*?:"<>|]',"","your_string")
The fastest way to do this is to use unicode.translate,
see unicode.translate.
In [31]: _unistr = u'sdfjkh,/.,we/.,132?.?.23490/,/' # any random string.
In [48]: remove_punctuation_map = dict((ord(char), None) for char in '\/*?:"<>|')
In [49]: _unistr.translate(remove_punctuation_map)Out[49]:
u'sdfjkh,.,we.,132..23490,'
To remove all puctuation.
In [46]: remove_punctuation_map = dict((ord(char), None) for char in string.punctuation)
In [47]: _unistr.translate(remove_punctuation_map)
Out[47]: u'sdfjkhwe13223490'
Py - Clean illegal character in filename
python - Removing characters from filename in batch - Stack Overflow
batch rename - Removing special characters from filenames in multiple subdirectories in Python - Stack Overflow
terminal - How to strip a filename of special characters? - Ask Different
You can look at the Django framework (but take their licence into account!) for how they create a "slug" from arbitrary text. A slug is URL- and filename- friendly.
The Django text utils define a function, slugify(), that's probably the gold standard for this kind of thing. Essentially, their code is the following.
import unicodedata
import re
def slugify(value, allow_unicode=False):
"""
Taken from https://github.com/django/django/blob/master/django/utils/text.py
Convert to ASCII if 'allow_unicode' is False. Convert spaces or repeated
dashes to single dashes. Remove characters that aren't alphanumerics,
underscores, or hyphens. Convert to lowercase. Also strip leading and
trailing whitespace, dashes, and underscores.
"""
value = str(value)
if allow_unicode:
value = unicodedata.normalize('NFKC', value)
else:
value = unicodedata.normalize('NFKD', value).encode('ascii', 'ignore').decode('ascii')
value = re.sub(r'[^\w\s-]', '', value.lower())
return re.sub(r'[-\s]+', '-', value).strip('-_')
And the older version:
def slugify(value):
"""
Normalizes string, converts to lowercase, removes non-alpha characters,
and converts spaces to hyphens.
"""
import unicodedata
value = unicodedata.normalize('NFKD', value).encode('ascii', 'ignore')
value = unicode(re.sub('[^\w\s-]', '', value).strip().lower())
value = unicode(re.sub('[-\s]+', '-', value))
# ...
return value
There's more, but I left it out, since it doesn't address slugification, but escaping.
You can use list comprehension together with the string methods.
>>> s
'foo-bar#baz?qux@127/\\9]'
>>> "".join(x for x in s if x.isalnum())
'foobarbazqux1279'
import os
for filename in os.listdir('dirname'):
os.rename(filename, filename.replace('_intsect_d', ''))
This code can be used to remove any particular character or set of characters recursively from all filenames within a directory and replace them with any other character, set of characters or no character.
import os
paths = (os.path.join(root, filename)
for root, _, filenames in os.walk('C:\FolderName')
for filename in filenames)
for path in paths:
# the '#' in the example below will be replaced by the '-' in the filenames in the directory
newname = path.replace('#', '-')
if newname != path:
os.rename(path, newname)
Using os.walk is a reasonable approach.
You'll need to refine your specifications somewhat, though:
Do you want to rename directories and files, or only files? (For instance, given
'('as a character-to-be-removed, what do you do with the path'this(/that'? The file name is fine, but the directory name has one of the bad characters.)What do you do if renaming a file (or directory) would result in a collision? For instance, suppose you find a file named
'this('but there is already a file named'this'(no parenthesis)?
Aside from both of these issues, the method Hackaholic just posted looks good.
you can use os.walk as you mentioned:
for dir, subdir, files in os.walk(path):
for file in files:
os.rename(os.path.join(dir,file), os.path.join(dir, "".join(filter(lambda x:x not in bad_chars, file))))
If you have a specific set of characters that you want to keep, tr works very well.
For example
tr -cd 'A-Za-z0-9_-'
Will remove any characters not in the set of characters listed. (The -d means delete, and the -c means the complement of the characters listed: in other words, any character not listed gets deleted.)
This would only replace single quotes with underscores:
for f in *; do mv "$f" "${f//'/_}"; done
This would only keep alphanumeric ASCII characters, underscores, and periods:
for f in *; do mv "$f" "$(sed 's/[^0-9A-Za-z_.]/_/g' <<< "$f")"; done
Locales like en_US.UTF-8 use the ASCII collation order on OS X, but [[:alnum:]] and \w also match characters like ä in them. If LC_CTYPE is C, multi-byte characters are replaced with multiple underscores.
Hi, basically in my python script i'm using to learn I'm trying to select the title of a web page and then save a txt file with the same name of the title of the webpage. The only problem is, if the title of the web page contains special characters such as : then the name of the txt is truncated just before this special character. How do I avoid this problem? I'm fine with completely deleting the special character from the name of the txt file.
I have tried this, during the selection
p.title = title_page['title'] + contents.replace(':','')
and also the same at the end, during the save of the file, but it doesn't work, it tells me contents is not defined in both situations. The problem is I cannot define it since the special character may be there but it may also NOT be there. How do I fix this?
Thanks
I think the safest approach here is to just replace any suspicious characters. So, I think you can just replace (or get rid of) anything that isn't alphanumeric, -, _, a space, or a period. And here's how you do that:
import re
re.sub(r'[^\w_. -]', '_', filename)
The above escapes every character that's not a letter, '_', '-', '.' or space with an '_'. So, if you're looking at an entire path, you'll want to throw os.sep in the list of approved characters as well.
Here's some sample output:
In [27]: re.sub(r'[^\w\-_\. ]', '_', r'some\*-file._n\\ame')
Out[27]: 'some__-file._n__ame'
Unfortunately, the set of acceptable characters varies by OS and by filesystem.
Windows:
- Use almost any character in the current code page for a name, including Unicode characters and characters in the extended character set (128–255), except for the following:
- The following reserved characters are not allowed:
< > : " / \ | ? * - Characters whose integer representations are in the range from zero through 31 are not allowed.
- Any other character that the target file system does not allow.
- The following reserved characters are not allowed:
The list of accepted characters can vary depending on the OS and locale of the machine that first formatted the filesystem.
.NET has GetInvalidFileNameChars and GetInvalidPathChars, but I don't know how to call those from Python.
- Use almost any character in the current code page for a name, including Unicode characters and characters in the extended character set (128–255), except for the following:
- Mac OS: NUL is always excluded, "/" is excluded from POSIX layer, ":" excluded from Apple APIs
- HFS+: any sequence of non-excluded characters that is representable by UTF-16 in the Unicode 2.0 spec
- HFS: any sequence of non-excluded characters representable in MacRoman (default) or other encodings, depending on the machine that created the filesystem
- UFS: same as HFS+
- Linux:
- native (UNIX-like) filesystems: any byte sequence excluding NUL and "/"
- FAT, NTFS, other non-native filesystems: varies
Your best bet is probably to either be overly-conservative on all platforms, or to just try creating the file name and handle errors.