The pattern rules for glob are not regular expressions. Instead, they follow standard Unix path expansion rules. There are only a few special characters: two different wild-cards, and character ranges are supported [from pymotw: glob – Filename pattern matching].
So you can exclude some files with patterns.
For example to exclude manifests files (files starting with _) with glob, you can use:
files = glob.glob('files_path/[!_]*')
The pattern rules for glob are not regular expressions. Instead, they follow standard Unix path expansion rules. There are only a few special characters: two different wild-cards, and character ranges are supported [from pymotw: glob – Filename pattern matching].
So you can exclude some files with patterns.
For example to exclude manifests files (files starting with _) with glob, you can use:
files = glob.glob('files_path/[!_]*')
You can deduct sets and cast it back as a list:
list(set(glob("*")) - set(glob("eph*")))
You can make the pattern less exclusive as some comments mention by doing something like glob.glob('*_*_*_*.json'). More details can be found here https://docs.python.org/3.5/library/glob.html#glob.glob.
This isn't ever clean and since glob isn't regular regex it isn't very expressive. Since ordering doesn't seem very important you could do something like
excludedFiles = ['merged_file.json']
includedFiles = glob.glob('*.json')
# other code here
print list(set(includedFiles) - set(excludedFile))
That answers your question however I think a better solution to your problem is separate your raw data and generated files into different directories. I think that's generally a good practice when you're doing adhoc work with data.
If you want to remove only the latest file added, then you can try this code.
import os
import glob
jsonFiles = []
jsonPattern = os.path.join('*.json')
fileList = glob.glob(jsonPattern)
for file in fileList:
jsonFiles.append(file)
print jsonFiles
latestFile = max(jsonFiles, key=os.path.getctime)
print latestFile
jsonFiles.remove(latestFile)
print jsonFiles
Output:
['man_pro_desc_nordstrom.json', 'man_pro_desc_Zalando.json', 'men_pro_desc_zalora.json', 'merged_file.json']
merged_file.json
['man_pro_desc_nordstrom.json', 'man_pro_desc_Zalando.json', 'men_pro_desc_zalora.json']
you are using the wrong character to say none of this character...
If you're looking to find any file which has log_ at the beginning, and then a load of characters where none of them are _, then you just need to do this:
log_[^_]*
You are close. The pattern you are looking for is log_[^_]*. It says it must have 'log_' followed by zero or more non-underscore characters.
Ones you have your list of files you can use list comprehension to filter and remove any files that contain 'ER'.
files = [f for f in files if 'ER' not in f]
There are other libraries that can do exclusions. For instance wcmatch (full disclosure, I'm the author of it) allows you to use exclusion patterns (when enabled via a flag). Exclusion patterns are given along with a normal patterns, and it will filter the returned list of files:
from wcmatch import glob
glob.glob(['*.txt', '!*ER*.txt'], flags=glob.N)
Here is a real world example:
from wcmatch import glob
>>> glob.glob(['*.md'], flags=glob.N)
['LICENSE.md', 'README.md']
>>> glob.glob(['*.md', '!*EA*.md'], flags=glob.N)
['LICENSE.md']
shopt -s extglob
echo rm foo.!(org)
This matches foo. followed by anything NOT org
To restore normal glob behavior afterwards, use:
shopt -u extglob
Reference: The GNU Bash reference manual
In bash, you can also use GLOBIGNORE="*.org"; rm -i foo*.
And unset GLOBIGNORE when done.
It's not really better than shopt -s extglob, but I find it easier to remember.