Best settings to performance, large deduplicate photo/video library with wrong Metadata files

Welcome to the PhotoStructure forum, @paqmac ! Thanks for the kind words.

In general, the more CPU cores and the faster the disk you can put the PhotoStructure library (at least $library/.photostructure/models!), the better.

The Intel® Core™ i5-9500T doesn’t have hyperthreading, so PhotoStructure will only schedule work on 4 of those cores, and sync takes a core, so you should only see 3 concurrent file imports. Your system may be OK with setting PS_CPU_BUSY_PERCENT=85 or even 90. Check your sync report for timeouts to validate you aren’t overscheduling and getting hammered by iowait.

I scanned through your settings, and those seem, for the most part, just fine, except the following:

Note that this will omit photos that have had metadata stripped from them. Ideally, they will be in a directory that encodes the day (or month): ideally, .../YYYY-mm-dd/..., but there are a ton of different patterns I try to match against.

If sync is stuck like that, there should be some clue as to what’s going wrong in the sync reports and the logfiles.

This is tricky, given that the current PhotoStructure schema doesn’t really track photo bursts correctly–they are lumped together as a single set of variations. Ideally, all the assets assigned to a “burst” would be given a common “burst ID”, but I haven’t found a metadata tag that is consistent between cameras (so I’ll have to resort to some sort of “set of synonyms” like I have had for so many other things).

If processing takes about 10 seconds per asset (which may be on average about right, especially if you have HEVC and videos that need transcoding), that pencils out:

require("./dist/core/date/DurationFormat")
  .fmtFullDuration(25_000 * 10_000)
'2d21h26m40s'

The next release improves the function timers, so setting PS_EMIT_TIMINGS_ON_EXIT=true will enumerate all higher-level functions, how often they are being called, and what is taking up most of the time. Running sync against a single file or a handful of problematic files can be enlightening.