I had this same issue and wished glob.glob had an optional parameter to include dot files. I wanted to be able to include ALL dot files in ALL directories including directories that start with dot. Its just not possible to do this with glob.glob. However I found that Python has pathlib standard module which has a glob function which operates differently, it will include dot files. The function operates a little differently, in particular it does not return a list of strings, but instead path objects. However I used the following

files=[]
file_refs = pathlib.Path(".").glob(pattern)
for file in file_refs:
    files.append(str(file))

The other noticeable difference I found was a glob pattern ending with **. This returned nothing in the pathlib version but would return all the files in the glob.glob one. To get the same results I added a line to check if the pattern ended with ** and if so then append /* to it.

The following code is a replacement for your example that include the files in directories starting with dot

import pathlib
for fileref in pathlib.Path('/path/').glob('**/*.ext'):
    filename = str(fileref)
    print(filename)

Instead of: path.rglob('*')

Try this: path.rglob('[!.]*') if not os.path.isdir(p)

Unfortunately I can't test now, but hopefully it'll do.

You can write one yourself:

import os

def listdir_nohidden(path):
    for f in os.listdir(path):
        if not f.startswith('.'):
            yield f

Or you can use a glob:

import glob
import os

def listdir_nohidden(path):
    return glob.glob(os.path.join(path, '*'))

Either of these will ignore all filenames beginning with '.'.

You might want to try Formic.

from formic import FileSet
fileset = FileSet(directory="/some/where/interesting",
              include="*.py",
              exclude=["desktop.ini", ".*", "addition", "globs", "here"]
              )
for filename in fileset:
    # use shutil to move them

This is a Python library using Globs, but i) already understands most hidden files (list of builtins here), and ii) allows you to specify any files to exclude from the results (documentation)

Disclosure: I am the maintainer.

As per issue #26096 on the official bug tracker, this difference has been deemed not a bug and is therefore completely intended.

Credits to @vaultah for the find.

You can use the GLOBIGNORE variable to hide the . and .. directories. This does automatically also set the dotglob option, so * now matches both hidden and non-hidden files. You can again manually unset dotglob, though, this then gives the behavior you want.

See this example:

$ ls -a
.  ..  a  .a  ..a
$ GLOBIGNORE=".:.."
$ shopt -u dotglob
$ echo * # all (only non-hidden)
a
$ echo .* # all (only hidden)
.a ..a

Note the following corner cases shown by numerous users in the comments:

• . and .. are always ignored as long as GLOBIGNORE has a non-null value. GLOBIGNORE=":" ; echo .*, for instance, would work for the example above, even though it is less readable.
• If you want to match hidden directories only (and not all hidden files) and therefore glob with a trailing slash, echo .*/, you need to set GLOBIGNORE='.:..:./:../', because the glob will match ./ and ../, which must be excluded explicitly to prevent them from showing up in the result.

The pathlib version of glob doesn't filter hidden files and directories so you'll have to check them yourself. Since it returns all entries, you have to check every part (path component) separately:

[path for path in p.glob('**') if not any(part.startswith('.') for part in path.parts)]

This has several problems: it's inefficient for large hidden directories because you can't remove them just once, you have to filter out every single of their subfolders. It also won't work if any part of your starting folder is hidden, unless you cut that off in the if check.

An alternative would be to use use os.walk() which allows you to modify the descent while it happens, as described in this answer.

This all assumes you're using *nix where the hidden status is indicated by adding a dot in front of the name. On Windows you might have to resort to external tools like attrib.