Python library for traversing directory tree like unix command line tool `find`

Question

I know how to use os.walk().

I know how to implement all I need with it.

Nevertheless I think things could be simpler.

For simple stuff I use the command line tool find, since it is declarative.

If I use os.walk() I do it the imperative way.

Example: to include only files which end with '.txt' I use this in Python:

if not file.endswith('.txt'):
   continue

The find way is much easier:

find -name '*.txt'

Is there a library which provides a declarative way to traverse a directory tree with python?

Answer 1

There is the grin python library which includes grind.

• grin is a python implementation of grep using python regular expression syntax and being python it is cross platform
• grind is a python implementation of find.

Both can be used as libraries or command line utilities.

Grin command line usage:

>grind --help
usage: grind-script.py [-h] [-v] [-s] [--skip-hidden-files] [-b]
                       [--skip-backup-files] [-S] [--skip-hidden-dirs]
                       [-d SKIP_DIRS] [-D] [-e SKIP_EXTS] [-E] [--no-follow]
                       [--follow] [-0] [--dirs DIRS [DIRS ...]] [--sys-path]
                       [glob]

Find text and binary files using similar rules as grin.

positional arguments:
  glob                  the glob pattern to match; you may need to quote this
                        to prevent the shell from trying to expand it
                        [default='*']

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit
  -s, --no-skip-hidden-files
                        do not skip .hidden files
  --skip-hidden-files   do skip .hidden files
  -b, --no-skip-backup-files
                   do not skip backup~ files [deprecated; edit --skip-
                   exts]
 --skip-backup-files   do skip backup~ files [default] [deprecated; edit
                       --skip-exts]
 -S, --no-skip-hidden-dirs
                       do not skip .hidden directories
 --skip-hidden-dirs    do skip .hidden directories
 -d SKIP_DIRS, --skip-dirs SKIP_DIRS
                       comma-separated list of directory names to skip
                       [default='CVS,RCS,.svn,.hg,.bzr,build,dist']
 -D, --no-skip-dirs    do not skip any directories
 -e SKIP_EXTS, --skip-exts SKIP_EXTS
                       comma-separated list of file extensions to skip [defau
                       lt='.pyc,.pyo,.so,.o,.a,.tgz,.tar.gz,.rar,.zip,~,#,.ba
                       k,.png,.jpg,.gif,.bmp,.tif,.tiff,.pyd,.dll,.exe,.obj,.
                       lib']
 -E, --no-skip-exts    do not skip any file extensions
 --no-follow           do not follow symlinks to directories and files
                       [default]
 --follow              follow symlinks to directories and files
 -0, --null-separated  print the filenames separated by NULs
 --dirs DIRS [DIRS ...]
                       the directories to start from
 --sys-path            search the directories on sys.path

Answer 2

python-glob2 is an extention of glob that support recursive '**' globbing syntax. It can do the job if you are not looking for tons of special options

>>> import glob2
>>> all_header_files = glob2.glob('src/**/*.h')
['src/fs.h', 'src/media/mp3.h', 'src/media/mp3/frame.h', ...]

Answer 3

There is also the library python-fs which allows syntax like:

files = fs.find('*.txt')

which returns a generator or you could use:

filelists = list(fs.find('*.txt'))

Unfortunately it has a name clash with py-filesystem which I find much more useful personally.