My sync got "stuck"!

If your sync seems to not be making progress, or is processing the same file for a long time, read on:

Possible issues

  1. Your library or cache directory may not have sufficient free space. PhotoStructure automatically pauses imports if there’s less than minDiskFreeGb free disk space (this is a system setting, and defaults to 6gb). The about page will highlight any volumes that are almost full.

  2. Is it a long video that isn’t formatted as an .mp4? PhotoStructure will try to transcode it so it will play through your browser, but on slower computers, transcoding can take 4-10x longer than the duration of the video.

  3. Is it an image? How big is the directory that the image lives in? Version 0.9.1 and earlier versions may have a hard time processing large directories (those with 10,000+ files), especially if the folder is remotely mounted.

Current solutions

To solve issue 1: Normally PhotoStructure will not import videos that are > 500MB, but this may still be “too big” in some cases. Adjust this with the maxAssetFileSizeBytes library setting.

To solve issue 2: For now, either move assets into smaller directories, or mark these gigantic directories with a NoMedia folder to have those assets be skipped.

Future solutions

For issue 1: PhotoStructure will have a new “maxVideoTranscodeDuration” that will only transcode the first N minutes of the video.

For issue 2: The next version has several changes to how directories are iterated, and when “tag inference” is used, so assets found in gigantic directories can be imported properly.

Related:

Edit 20210304: added disk-full check

Hi,

I’m there currently. Stuck.

I just installed PhotoStructure on a Proxmox server, under a Turnkey container (turnkey-nodejs); I followed the instructions, and was able to load the settings page.
I added the path to the folder where my photos are stored (“Manual” option, and “No thanks” ;-)), everything seems correct.
After saving the settings, I’m stuck on the page saying that “Your library is currently empty” with the animation “/Photos processing” and “Scanning photos”. It’s been hours and there is still nothing progress.
When I check the “About” page, I see “0 assets / 5 tags”.
My library is not that big, only 10GB and about 7000 photos (in 114 folders/sub-folders).
I’m sure I’m missing something, somewhere, but I don’t know where to look at.

Any tip ?

Hey @GoustiFruit welcome to PhotoStructure!

I haven’t tried PhotoStructure under proxmox before: can you share the volumes table in your about page?

You can also shell into your docker container and manually start the sync, with --verbose, and it should tell you what’s going on.

If the logs don’t make sense, email them to suuport@photostructure.com and I can take a look.

PhotoStructure is installed in a LXC container, not Docker. I followed the instructions for Node under Ubuntu; Turnkey being based on Debian, the instructions are just spot on :slight_smile:

I bind mounted my folder from my storage in Proxmox, “/Stockage/Photos”, to a “/Photos” folder in my container, with the command:

pct set 101 -mp0 /Stockage/Photos,mp=/Photos

I can access and read that folder from the container (see screen capture).

Hmm, yes, I will send you my .log file, I really don’t have the knowledge to parse and understand it !

Sorry, I re-sent my log file, I hope you received it this time.

I resend my question here, because I’m not sure if my mails went through (I always forget to attach something, or send it to the wrong address, etc.).

As I mentionned:

  • I’m using Proxmox, I have a ZFS storage (named “Stockage”) where I created a “Photos” folder and I put my photos into subfolders (each user will have its own folder).

  • I installed PS in a Turnkey LXC container (unprivileged), and bind mounted Stockage/Photos from the host to /Photos in the container (ID 100):
    pct set 100 mp0 /Stockage/Photos,mp=/Photos

  • as PhotoStructure UID is 1000 in the container, I set the permissions to the /Stockage/Photos from the host to 101000. When I start the container, I can list the content of my /Photos
    chown -R 101000:101000 Photos

I can access PS web UI, set the options, but then I’m stuck on PhotoStructure processing page, with an animation of PS scanning my /Photos folder, but nothing else happens.

Here are some screenshots that may help (or not ?). Did anyone have a similar problem ? Did you solve it ? And most importantly… how ?



Hey there, sorry for the latency! PhotoStructure looks like it’s paused because there isn’t enough free disk space on both your library and cache volumes.

I added this check because I’ve had both Windows and macOS crash hard when their system disk filled (and a system update tried to apply or something else mucked with system files). I just edited the top post to include this issue.

I’ll add more obvious “hey, I’m stuck here” notifications in the main UI soon.

Holler if this doesn’t turn out to be the problem, but know that proxmox isn’t a platform I can test against (or give reasonable support for: I’ve never run it).

Ah, got it. I read somewhere that the minimum space for the cache should be 6GB, and I currently only have 5.59GB free on that system ! Is the required space relative to the weight (number and size) of the medias stored, or is that a minimum absolute that will be enough whatever how many files are stored later ? (on my early tests, I only have a handful of pictures, totalizing about 23 MB, but I will then add about 50GB more)

It’s not fancy: it just looks at free disk space, and if it’s less than 6 GB (not GiB), it pauses the sync process completely (so you don’t wake up to PhotoStructure having borked your box, which may have happened to me a couple times before I made that health-check)

I resized the disk of my container (from 8GB to 10GB):
pct resize 100 rootfs 10G
and restarted it.

PhotoStructure is finally working :slight_smile:
Thanks !

1 Like