New in v2.1: file globbing

PhotoStructure’s directory walker was rewritten in v2.1 to added glob support.

What’s this?

PhotoStructure comes built-in with several hundred exclusion patterns to avoid walking into system and application directories.

If any of these cause your files to be ignored, or you’re already using some other naming convention to exclude directories (instead of, for example, NoMedia), globs should give you the flexibility to teach PhotoStructure to only import the files you want it to.

One list for both accept and reject paths

  • The globs setting includes file path patterns to both force-accept or force-reject imported files.

  • Negative matches start with a !.

  • Positive matches don’t start with a !.

  • Order matters: the first pattern than matches will “win”.

  • If any of your globs match, the built-in exclusion patterns will be ignored for that file or directory.

Other filters may still apply

Note that globs only overrides PhotoStructure’s built-in file and directory exclusion patterns. All of the other filters, including

  • requireMakeModel,
  • rejectRatingsLessThan,
  • keywordBlocklist,
  • minImageDimension,
  • minVideoDimension,
  • minVideoDurationSec,
  • maxVideoDurationSec,
  • minAssetFileSizeBytes,
  • maxAssetFileSizeBytes

(note this list is current as of v2.1, and should be expected to grow in future releases)

Why is it called “glob”?

Globs” are patterns that may include asterisks, like /a/*/c (which matches any sub-directory within /a that includes a file or sub-directory named c), and other “magic” characters to match desired patterns.

Magic glob characters

  • * Matches 0 or more characters in a single path portion
  • ? Matches 1 character
  • [...] Matches a range of characters, similar to a RegExp range. If the first character of the range is ! or ^ then it matches any character not in the range.
  • !(pattern|pattern|pattern) Matches anything that does not match any of the patterns provided.
  • ?(pattern|pattern|pattern) Matches zero or one occurrence of the patterns provided.
  • +(pattern|pattern|pattern) Matches one or more occurrences of the patterns provided.
  • *(a|b|c) Matches zero or more occurrences of the patterns provided
  • ** If a “globstar” is alone in a path portion, then it matches zero or more directories and subdirectories searching for matches. It does not crawl symlinked directories.

This list is courtesy of glob. FWIW, PhotoStructure uses picomatch with default options.

Always use forward slashes

Even on Windows: back-slashes are for escaping characters.

Example #1: exclude custom directories

If you’ve already been using a naming convention to organize files, globs might help you teach PhotoStructure about these naming conventions.

For example, if you always put random screenshots that you don’t want in your library in a “screenshots” directory, add this pattern:

!**/screenshots/**

the exclamation at the beginning of the pattern tells PhotoStructure that this is a “negative” or “exclusion” match: things that match should not be imported.

Note that globs is a standard PhotoStructure setting, so you can either set it with environment variables, or via the library settings.toml.

Example #2: include custom directories

PhotoStructure has a (ton!) of file patterns it uses to avoid “ignorable” directories. If any of these patterns turn out to skip a directory you wanted PhotoStructure to import, please tell me (either reply here, or ping me on Discord), but you can also tell PhotoStructure to omit that “ignorable” pattern.

As an example, Linux systems traditionally put files that have been recovered by fsck into a directory called lost+found. These files are frequently truncated or otherwise corrupt, though, so PhotoStructure “ignores” that directory by default.

If you coincidentally have photos in other directories that are called “lost+found” that you want to import, add this pattern:

**/lost+found/**

How this works with scanPaths

The scanPaths setting is what this section of the settings page edits:

Note that scanPaths still drives what directories that are scanned.

In other words, if you add a glob pattern for /photos/**, but scanPaths is set to MANUAL and doesn’t include /photos (or any subdirectory), that glob pattern will be ignored.

See also

Pardon my ignorance here, but could you please provide a few concrete examples? Does this need to go to the settings.toml file?

Good idea: I added one example, with links to the settings docs. If you describe what you’d like to specifically include or exclude, tell me, and I can add that as an additional example.

Your example is already very good. Additionally, one example with a specific folder/path?

Done: holler if you have any other questions!

1 Like

Ok, so I could exclude by adding the following entry?

!*X:/abc/WhatsApp/ *

Almost: I’d go with an even simpler pattern: !**/WhatsApp/** will skip over any file in a WhatsApp directory or subdirectory (so X:\Photos\WhatsApp\2022\image.jpg would get excluded, for example)

1 Like