API Overview

Top level namespace:

Any criterion can be used on its own with its dirmagic.core_criteria.Criterion.test() method, returning a dirmagic.core_criteria.CriterionResult object. This object is true if the test criterion is successful on the directory specified. It also contains a detailed report on the test result.

Generic Criteria

Module dirmagic.generic_criteria

Define the expected entries and their properties using criterion classes.

These classes are used to define project types and directory patterns.

Generic criteria can be combined using the operators &, | and negated with ~. Most generic criteria can be used within dirmagic.pattern_criteria.AnyMatchCriterion or dirmagic.pattern_criteria.AllMatchCriterion to build up complex criteria on many entries matched by name.

Project Criteria

Module dirmagic.project_types

This module contains a variety of directory criteria describing project types.

Each dirmagic.core_criteria.ProjectType provides a name and a category for the project type along with the criterion describing the expected directory structure.

By default dirmagic.identify_project() uses all project types in this module to determine matching the project types.

Please see the Development section for instructions on how to contribute to this collection.

Pattern Criteria

Module dirmagic.pattern_criteria

Detect existing files by pattern and apply a criterion for each match, based on the match filename.

Examples:

  • The generic criterion HasFilePattern(pattern, ...) is equvalent to AnyMatchCriterion(pattern, HasFile(...))

  • check whether every *.csv file follows a day-month-year.csv naming convention and the header line is time,transaction,balance.

AllMatchCriterion(
    r"^.*?([^/]*)\.csv$",
    MatchesPattern("{0[1]}", r"\d\d-\d\d-\d\d\d\d") &
    HasFile("{0[0]}", "time,transaction,balance", n=1, fixed=True)
)

  • check whether there is a *.jpg or *.png file for each *.txt file.

AllMatchCriterion(
  r"(.*)\.txt$",
  HasFile("{0[1]}.jpg") | HasFile("{0[1]}.png")
)

  • check whether there are any MacOS file properties

AnyMatchCriterion(
    r"(^|.*/)\.DS_Store$",
    HasFile("{0[0]}")
)

The criteria dirmagic.pattern_criteria.AnyMatchCriterion and dirmagic.pattern_criteria.AllMatchCriterion work as follows:

  • All entries in the test directory will be considered (breadth first), the search can be narrowed down by limiting the depth or the entry types returned.

  • The entries will be matched against the regular expression pattern using re.search() and when successful the result is returned.

  • For each match the criterion is adjusted according to this match result by using the str.format() function on one (or more) properties of the criterion (and the sub-criteria if existent).

  • For each match, the adjusted criterion is tested.

  • For dirmagic.pattern_criteria.AnyMatchCriterion, the test is successful when one test is positive, for dirmagic.pattern_criteria.AllMatchCriterion the test is unsuccessful when one test is negative.

The re.Match.group() is used to customize the criterion’s parameters depending on the match/es:

  • {0[0]} is replaced with the full match, i.e. match[0],

  • {0[1]}, … is replaced with the first, … matched group, and

  • {0[groupname]) is replaced with the named match, i.e. (?P<groupname>.*\.txt)

  • {1[groupname]}, … are used for nested match ciriteria, conting the matches from innermost (0) outwards.

The criteria dirmagic.pattern_criteria.MatchesPattern, dirmagic.pattern_criteria.FileMimeType, dirmagic.pattern_criteria.IsIn, and dirmagic.pattern_criteria.SuffixIsIn are especially useful within this context.

The criterion dirmagic.pattern_criteria.SpyCriterion prints out the parameters going into the test functions. This helps to debug complex criteria, in addition to examining the dirmagic.core_criteria.CriterionResult. The SpyCriterion is always true, so it can be added at any location using &.

Regular Expressions and Paths

The module dirmagic.pattern_criteria relies heavily on regular expressions (see re). Regular expressions have a steep learning curve at the beginning.

Some Examples:

  • use ^ to anchor the pattern at the beginning and $ at the end of the string.

  • an elaborate and versatile example:

r"^(?P<path>(?P<parent>[^/]*/)*)(?P<stem>.*?)(?P<suffix>\.[^.]+)?$""

This provides several useful elements of a path, e.g. for a/b/c/d.txt:

  • 0[path] is a/b/c/

  • 0[stem] is d

  • 0[suffix] is .txt

  • 0[parent] is c/

Todo: find extension, stem of file name, path before, use \0.

The function dirmagic.pattern_criteria.translate() converts a glob-style pattern into a regular expression, just like fnmatch.translate(), but wraps each wildcard into a match group.

Todo: Demonstrate Use…

Core Criteria

Module dirmagic.core_criteria

This module provides the abstract criteria and operations on those, like &, | and ~. The test result object is defined here as well.