Sync never completely finishes

Currently running 2024.3.3-beta via an unraid docker. Was running 1.1.0 for the longest time without realizing lots of new useful functionality in these new betas. To keep things clean, I created a new library and enabled the plus feature to copy all my photos to the library. This seems to be working well, but I’ve noted that not all my photos are importing. During the sync process, the progress bar will often go away completely and sometimes I can still see the sync logs updating along with worker logs doing the same other times they both stop and CPU usage drops to 0 indicating nothing is happening. I can restart the sync and more stuff is picked up but then it craps out some time later. I’ve been through this a few times of the last couple of days, but i’m not sure what to look for in the logs. I cannot clearly see where and/or why its stopping. I tried turning on debug logging and while I can see more, still no clear (to me) reasons for this behavior.

I have seen it validate the existing library successfully tons of times now, I’ve also seen at least twice claim to be done with the sync on the import, but its clearly missing hundreds or more photos (for example, none of my 2024 photos imported at all. They loaded up just fine in 1.1.0, so not sure whats going on.

What is the recommended path for determining why this happening? Otherwise loving this version, but clearly its not hitting everything as expected. I’m trying to dig through the sync log, but its quite large and will take some time to review. Just looking for any other guidance as to what may be the cause.

Welcome to the forum, @Hooverjc !

Apologies that sync isn’t running smoothly on your box. Unfortunately, there’s a litany of issues that PhotoStructure may be wrestling with that may cause sync to skip files or get “stuck,” which is why I wrote the health check system, and added the sync reports (rather than just relying on pouring through debug logs that can be an overwhelming needle-in-the-haystack exercise).

I recommend you check your latest sync report and search for errors. Those errors may be symptomatic of the actual issue (like timeouts due to an unhealthy disk), or durable issues, (like from file permission issues), or due to a bug in PhotoStructure (like from an unexpected photo or video format that causes the current processing pipeline to crash).

just wanted to share I found the issue. I had an adobe lightroom folder in this photos folder (with the DB, previews, and other stuff in there. 1.1.0 didn’t care or do anything with this folder, but the latest release was choking on it. I moved the folder out and that seems to have corrected the issue.

Thanks!

1 Like

Thanks for the followup!

I have some exclusion patterns in PhotoStructure to handle lightroom directories, but I wonder if those could be improved–could you DM me the result of

du -sh $path_to_your_lightroom_directory/*

This will give me the name and aggregate size of the subdirectories in your lightroom directory