Performance tweaks for lots of photos?

Are there any performance tweaks to enable? I notice that now that I have finished importing my photos (250K+), it takes about 35 seconds to load up the preview images on my Where/timeline page and 20+ seconds to load some of my larger year folders (the month preview views). It’s not unusable, but would be nice if I could speed that up. The server I’m running on should have plenty of memory/CPU, but doesn’t have an SSD.

Sorry that performance is lacking. I consider slow page loading to be a bug: we’ll get this fixed.

One thing you can do right off the bat is give the SQLite cache more memory. Set the dbCacheSizeMb library setting to something bigger than the default of 192MB. I’d use the very scientific method of looking at the size of your library database and multiplying by 1.5 or 2.

The db queries for most page loads should be hitting indexes: if you open the developer tools and click the network tab, XHR requests shouldn’t take longer than 50-100ms.

The other issue you’re fighting is image loading, and there are a couple of known issues that you’re probably hitting:

  1. etags don’t seem to be respected by some browsers, so thumbnails aren’t cached properly. This is on my bug todo list to fix.

  2. The current lazy-loading implementation is way too aggressive, loading images outside of the current viewport. It also doesn’t cancel prefetches when you scroll, so it requests (way) more images than it needs. This is also on my todo list to address.

  3. The previews are almost never cached by the filesystem (as the random-shuffle really is a pretty good random!), so loading those are I/O bound. With the next version of PhotoStructure, I’m hoping worker threads will be supported across all editions, which may help load the previews faster.

Once these issues are addressed, really the only other performance enhancement is getting faster I/O. SSDs are now cheap (a 500MB/s m2 1TB drive is now $70!), and my larger test libraries on SSD load the home page essentially instantly. The test libraries that are on slow spinning rust, mounted over a NAS, load in a couple seconds.

Putting previews and your library on an SSD is doable in v0.9.1 with the PS_PREVIEWS_DIR, but the next version’s PS_ORIGINALS_DIR gives a bit more flexibility.

I’ll work on the bugfixes for the next release. If you have time to play with increasing the db cache, please tell me if that helps XHR latencies.

Cheers and happy holidays!

My db is 298MB, so I increased the cache to 600MB and uncommented and quit and restarted the app. It seems faster, but still not instantaneous.
dbCacheSizeMb = 600

The xhr requests seem to be taking 1.23-1.82s, one came back at 766ms. The jpegs came back in 8-16ms but then a handful were 1.23s-1.5s and then many of them got progressively slower 87-187ms (images it loaded towards the end).

1 Like

Thanks for the update!

Is it still slow if you switch to large thumbnails? That should put much less load on the lazy-loader.

It seems pretty fast with large thumbnails since as you point out, it’s barely loading any images, so it’s less load.