There are a couple of ways:
pathlib.Path().rglob()
Use pathlib.Path().rglob() from the pathlib module, which was introduced in Python 3.5.
from pathlib import Path
for path in Path('src').rglob('*.c'):
print(path.name)
glob.glob()
If you don't want to use pathlib, use glob.glob():
from glob import glob
for filename in glob('src/**/*.c', recursive=True):
print(filename)
For cases where matching files beginning with a dot (.); like files in the current directory or hidden files on Unix based system, use the os.walk() solution below.
os.walk()
For older Python versions, use os.walk() to recursively walk a directory and fnmatch.filter() to match against a simple expression:
import fnmatch
import os
matches = []
for root, dirnames, filenames in os.walk('src'):
for filename in fnmatch.filter(filenames, '*.c'):
matches.append(os.path.join(root, filename))
This version should also be faster depending on how many files you have, as the pathlib module has a bit of overhead over os.walk().
There are a couple of ways:
pathlib.Path().rglob()
Use pathlib.Path().rglob() from the pathlib module, which was introduced in Python 3.5.
from pathlib import Path
for path in Path('src').rglob('*.c'):
print(path.name)
glob.glob()
If you don't want to use pathlib, use glob.glob():
from glob import glob
for filename in glob('src/**/*.c', recursive=True):
print(filename)
For cases where matching files beginning with a dot (.); like files in the current directory or hidden files on Unix based system, use the os.walk() solution below.
os.walk()
For older Python versions, use os.walk() to recursively walk a directory and fnmatch.filter() to match against a simple expression:
import fnmatch
import os
matches = []
for root, dirnames, filenames in os.walk('src'):
for filename in fnmatch.filter(filenames, '*.c'):
matches.append(os.path.join(root, filename))
This version should also be faster depending on how many files you have, as the pathlib module has a bit of overhead over os.walk().
For python >= 3.5 you can use **, recursive=True, i.e.:
import glob
for f in glob.glob('/path/**/*.c', recursive=True):
print(f)
If recursive is
True(defaultFalse), the pattern**will match any files and zero or moredirectoriesandsubdirectories. If the pattern is followed by anos.sep, only directories andsubdirectoriesmatch.
Python 3 Demo
I think you have misunderstood the meaning of recursive=True. It is used to allow ** to mean searching subdirectories to a variable depth -- glob(folder+'\\**\\*.ext',recursive=True) will find any *.ext files in the starting folder or anywhere below it. Without recursive=True, ** would be interpreted in the normal way, with each * meaning to match any number of characters excluding \, and ** would therefore just be equivalent to * -- in other words glob(folder+'\\**\\*.ext',recursive=False) would search for any *.ext files that are exactly one level of subdirectory down from the starting folder, but not in the starting folder itself or in deeper subdirectories.
Exactly which you should use will depend on what you are trying to look for -- and it may indeed be that you need to select between different cases with an if statement -- but there is no use case for using ** with recursive=False, because it is just equivalent to * in that case.
The most likely useful options are:
glob(folder+'\\*.ext',recursive=False) # search in top folder only
glob(folder+'\\*\\*.ext',recursive=False) # search exactly one level down
glob(folder+'\\**\\*.ext',recursive=True) # search to arbitrary depth
The ,recursive=False can be omitted because it is the default.
If there were exactly two options that you wanted to select between using a boolean option, you could always write your own wrapper function, for example:
def myglob(folder, pattern, subdirs=False):
if subdirs:
return glob(f'{folder}\\**\\{pattern}', recursive=True)
else
return glob(f'{folder}\\{pattern}')
print(myglob(folder, '*.ext', subdirs=False))
print(myglob(folder, '*.ext', subdirs=True))
print(myglob(folder, '*.ext')) # same as subdirs=False in this example
(In Unix-like operating systems, read / for \ above.)
Operation of glob '**'
The glob source code can be found here. Other than comments, '**' only appears in the _isrecursive function:
def _isrecursive(pattern):
if isinstance(pattern, bytes):
return pattern == b'**'
else:
return pattern == '**'
Because the _isrecursive function checks for '' using the equality operator, '' will only operate as intended if it is the entire 'pattern'. The _isrecursive function is called 4 times in the source code, once for the entire glob and 3 times for 'basename', which is defined as:
dirname, basename = os.path.split(pathname)
This is called recursively, splitting tails from the path one at a time, and each time, basename is everything following the slash (technically os.sep,which is \ for Windows or / for POSIX). That means that in order for '**' to operate as described, it may not be adjacent to anything other than slashes.
If recursive is not explicitly set to True, '**' = '*'+'*' = '*' since '*' matches 0 or more non-slash characters.
Matching Extensions
Because of how '**' operates, it must be used to represent an entire level of the glob. Since it represents 0 or more levels, r'folder\**\*.ext' will match files in 'folder', as well as subfolders. However, without recursive set to true, this pattern will match only files in the first level of subfolders.
'**.ext' will not work because '**' isn't alone. This leaves the following options:
Use a different library (eg. os.walk, pathlib.glob, etc.)
[f for f in glob(folder+'\\**',recursive=subdirs) if f.endswith('.ext')]glob(folder+('\\**' if subdirs else '')+'\\*.ext, recursive=True)Or equivalently,
if recursive: glob(folder+'\\**\\*.ext', recursive=True) else: glob(folder+'\\*.ext')glob(folder+'\\..\\**\\*.ext', recursive=subdirs) #not recommended since it will also match files in sibling folders
Use **/* as a pattern:
>>> from pprint import pprint as pp >>> import pathlib as pl >>> >>> >>> p = pl.Path(".") >>> >>> old_way = list(p.glob("**/")) # Your way >>> pp(old_way) [WindowsPath('.'), WindowsPath('dir0'), WindowsPath('dir1'), WindowsPath('dir1/dir10')] >>> >>> new_way = list(p.glob("**/*")) >>> pp(new_way) [WindowsPath('dir0'), WindowsPath('dir1'), WindowsPath('file0.txt'), WindowsPath('dir0/file00.txt'), WindowsPath('dir1/dir10'), WindowsPath('dir1/file10.txt')] >>> >>> newer_way = [p] + list(p.glob("**/*")) # Prepend globed dir >>> pp(newer_way) [WindowsPath('.'), WindowsPath('dir0'), WindowsPath('dir1'), WindowsPath('file0.txt'), WindowsPath('dir0/file00.txt'), WindowsPath('dir1/dir10'), WindowsPath('dir1/file10.txt')]
Here's [Python.Docs]: pathlib - Path.glob(pattern) for reference.
Consider using os.walk:
import os
all_files = []
for root, subdirs, files in os.walk("/Users/user/dir"):
all_files += [os.path.join(root, file) for file in files]
print(all_files)