Is the Library Writable?

The Setup:
I am using a Photo Structure Plus docker on Unraid. The destination is an NFS mounted directory on my Qnap NAS (/mnt/remotes/Photos/Photo_Doris_Structure/). The source is also an NFS directory Qnap NAS (/mnt/remotes/Photos/Photos_Doris/). All else is standard Unraid config (e.g. /mnt/user/appdata/photostructure/config/).

The Issue:
Using Beta 12, there were zero problems with transferring photos from the source to the new structure. However, with Beta 13, I am consistently getting the error below. Out of 35k photos and movies it has happened 9 times. Upon restart, it will continue for awhile, then hit the same error.

{“fatal”:true,“exit”:true,“status”:12,“pid”:25,“ppid”:18,“error”:“ChildService(web).onStdout(): Error: ChildService(web).onStdout()Cannot write to /ps/library: LibraryHealthChecks: Failed to copy sample file to library. Is /ps/library writable?¹⁶”}

Attempted diagnosis:
I saw somewhere else in the forums a list of commands to ‘touch’ the directories from within the container. I ran those commands with NO errors.

Thank you for your time and patience.

Welcome to PhotoStructure @Slash_Sk8tr : apologies that PhotoStructure is misbehaving for you!

Checking to see if the library directory is writable is one of the “health checks” that PhotoStructure runs periodically. I realized (this morning, actually) that only the main process really needed to do this check: beta.14 will have this change.

You can also completely disable the check by setting healthCheckLibraryIsWritable=false (either via environment variables or the library settings.toml. Note that all of the health checks can be disabled via settings: the defaults.env has a list of them here.

The underlying issue, though, may be due to PhotoStructure overwhelming the I/O on your remote NAS (especially if your Unraid box has a bunch of cores).

If disk operations take “too long” (defined by the commandTimeoutMs setting, which defaults to 25 seconds, so it’s pretty patient), things will start to timeout, and asset imports will automatically retry, but PhotoStructure isn’t smart enough (currently) to know that it need to throttle down the sync to prevent excessive iowait.

One way to see if this is the issue is to run top or iostat on your NAS during an import. If it is, indeed thrashing your disks, you can tell PhotoStructure to run slower via the cpuLoadPercent or maxSyncFileJobs settings: open the About page to see what PhotoStructure is currently using for the defaults on your system, and maybe set maxSyncFileJobs=2 (or 3).

If you have time to try this out, it’d be great if you gave an update (either positive or negative!).

Cheers!

(Note: if anyone knows how to check iowait on a remote filesystem, I’d love to have PhotoStructure throttle workloads to prevent this sort of thing automagically)

Ok. I have been playing around with the CPU usage to see if there are any changes. The images below are my Unraid Server CPU usage after making the change to Photo Structure. Notice, All Unraid CPU cores are maxed, but the about page states 25%. If I stop the container, CPU usage falls to 5% to 10% on all cores.


If you run top I suspect ffmpeg may be hitting all cores: I tell ffmpeg to use a fraction of available CPUs, but that seems to be ignored in certain os/version configurations.

I suspect ffmpeg may be hitting all cores

You are absolutely correct. Also, it seems limiting the CPU has stopped the errors. Thank you for your help.

The next question is a separate issue. if you would prefer for me to create a new post, please let me know.

My wife’s phone has recreated a (new) photo directory within the source photo’s folder. A Photo Structure resync does not seem to ‘find’ the images. Does this mean that each time a new folder is created a “rebuild” must take place?

Source directory:
/mnt/remotes/Photos/Photos_Doris/

New folder in directory:
/mnt/remotes/Photos/Photos_Doris/2021 Backup/

In the future, opening a new topic is preferable (just to help people with the same question find it later).

No, a resync should find the directory and pull in the new images. Is the new folder “ignorable”?

This also may help: