Are there any performance tweaks to enable? I notice that now that I have finished importing my photos (250K+), it takes about 35 seconds to load up the preview images on my Where/timeline page and 20+ seconds to load some of my larger year folders (the month preview views). It’s not unusable, but would be nice if I could speed that up. The server I’m running on should have plenty of memory/CPU, but doesn’t have an SSD.
Sorry that performance is lacking. I consider slow page loading to be a bug: we’ll get this fixed.
One thing you can do right off the bat is give the SQLite cache more memory. Set the dbCacheSizeMb library setting to something bigger than the default of 192MB. I’d use the very scientific method of looking at the size of your library database and multiplying by 1.5 or 2.
The db queries for most page loads should be hitting indexes: if you open the developer tools and click the network tab, XHR requests shouldn’t take longer than 50-100ms.
The other issue you’re fighting is image loading, and there are a couple of known issues that you’re probably hitting:
-
etags don’t seem to be respected by some browsers, so thumbnails aren’t cached properly. This is on my bug todo list to fix.
-
The current lazy-loading implementation is way too aggressive, loading images outside of the current viewport. It also doesn’t cancel prefetches when you scroll, so it requests (way) more images than it needs. This is also on my todo list to address.
-
The previews are almost never cached by the filesystem (as the random-shuffle really is a pretty good random!), so loading those are I/O bound. With the next version of PhotoStructure, I’m hoping worker threads will be supported across all editions, which may help load the previews faster.
Once these issues are addressed, really the only other performance enhancement is getting faster I/O. SSDs are now cheap (a 500MB/s m2 1TB drive is now $70!), and my larger test libraries on SSD load the home page essentially instantly. The test libraries that are on slow spinning rust, mounted over a NAS, load in a couple seconds.
Putting previews and your library on an SSD is doable in v0.9.1 with the PS_PREVIEWS_DIR, but the next version’s PS_ORIGINALS_DIR gives a bit more flexibility.
I’ll work on the bugfixes for the next release. If you have time to play with increasing the db cache, please tell me if that helps XHR latencies.
Cheers and happy holidays!
My db is 298MB, so I increased the cache to 600MB and uncommented and quit and restarted the app. It seems faster, but still not instantaneous.
dbCacheSizeMb = 600
The xhr requests seem to be taking 1.23-1.82s, one came back at 766ms. The jpegs came back in 8-16ms but then a handful were 1.23s-1.5s and then many of them got progressively slower 87-187ms (images it loaded towards the end).
1 Like
Thanks for the update!
Is it still slow if you switch to large thumbnails? That should put much less load on the lazy-loader.
It seems pretty fast with large thumbnails since as you point out, it’s barely loading any images, so it’s less load.
In case anyone else views this topic:
I noticed in recent testing that Chrome is 2-10x faster at rendering thumbnails than Firefox, at least on Linux. It pains me to say this, but you might want to try Chrome.
I’ve also made some front-end performance improvements to image loading in the latest v1.0.0-beta builds (we’re now using the browser-native lazy image loader if we’re not on safari), which seems to have helped as well.
There’s also a new advanced system setting to host your .photostructure/previews directory on a separate SSD: I just wrote this up here:
Just a note for others. I started with an Unraid NAS setup running as a docker, with the library hosted on my large and slow HDD over a network. The performance felt relatively slow with this setup. I recently installed a 1TB SSD for my Unraid setup and moved the library to this drive. This dramatically affected the performance and now everything feels almost instantaneous even with the smallest thumbnails option.
In addition to using an SSD, etc… Would just adding http/2 or http/3 support to sync-web maybe help out? Right now, it looks like it’s still using http 1.1. I know it can be a bit of a hassle since TLS cert configuration and associated headaches are required, but http2 would replace bunches of individual http GET requests for thumbnails with far fewer requests.
Unfortunately, PhotoStructure currently uses Express which doesn’t have http/2 support. I’ve looked at switching to fastify (which has experimental http/2 support)
I’ve also toyed with supporting N web processes bound into a cluster.
(Also, I mentioned this previously in this thread: but if you’re on Firefox (like I am), know that Chrome renders the PhotoStructure home page 2-10x faster than Firefox, at least on Ubuntu 20).