Got a (really) large library? Here are some tune-up tips

PhotoStructure’s default setting values should work reasonably well with 500,000 file libraries. If your library is larger than that, or your library is on slow storage, like a remotely-mounted disk or 5400 RPM external drive, the following tips may help:

Tip #1: Make setup more patient

The setupTimeoutMs system setting defaults to 35 seconds. That may not be enough time for PhotoStructure to set up a local replica of your library database (if your network or disk is slow), and then do any preliminary maintenance. Increasing this value to 1 minute (60000) or 2 minutes (120000) is advised to make sure startups aren’t flaky.

Update #1: I updated the startup process in version 1.0 to delay most of this maintenance until after the webservice is up and running, so startups should be faster (and hopefully this won’t be an issue).

Update #2:: I just taught PhotoStructure to automatically extend the timeout for larger libraries (specifically, to max(setupTimeoutMs, ([size of library database in Kb] / 10)), so things “just work.” The setting will remain, but no one should have to manually adjust it after v1.0 is out (hopefully).

Why / 10? A test model database is 160Mb that tracks 100k assets, and loading the database takes ~1.6 seconds on a slow external drive on a fast Linux workstation. I normally 10x timeouts to account for slower hardware and slower storage, which would be ~16 seconds, or 16,000ms for 160,000,000 bytes.

Tip #2: Give more RAM to the DB

The dbCacheSizeMb library setting defaults to 192. Try bumping this up to 512.

Update: I just taught PhotoStructure to automatically change this setting based on the size of the database (specifically, at least dbCacheSizeMb, and at most, 1.1 * [size of db.sqlite3 in MB]). As with the setupTimeoutMs, this setting will remain, but no one should have to manually adjust it after v1.0 is out (hopefully).

Tip #3: Make the DB more patient

The dbTimeoutMs library setting defaults to 1000 milliseconds (1 second). Try bumping this up to 5000 (5 seconds).

And when v1.0 comes around…

Version 1.0 introduces a new originalsDir setting. You may want to put your original asset files on a large disk, and store your database and previews on a fast local SSD.

1 Like