Assets on NFS share always receive error: AssetPreviewBuilder none of the files are valid

I’m experimenting with Photostructure for the first time and I’ve hit a strange issue that I can’t resolve.

I’m using Docker. I started with the stable tag but then I hit the “unexpected missing asset file id” on all files and nothing showed up. I have now moved to the beta tag and am running 2.0.0-beta.1.

My current problem is that any photos from an NFS-mounted network share fail with a “” error. Any photos on a local path import just fine. The library and cache directories are local to the docker host and look like they are populated for files that work. The config and log directories are on an NFS share I use for all containers.

The NFS share is mounted ro and I’ve tested opening a shell in the docker container and copying those files around, so the container can definitely read them.

Bind mounts for photo sources:

      - type: bind
        source: "/media/truenas/photos/2022/2022 01 01/" # this is a Truenas NFS share
        target: "/media/photos/2022/2022 01 01/"

      - type: bind
        source: "/media/bulkstorage/photostructure/samples" #this is local on the docker host
        target: "/media/photos/samples"

A working local AssetPreviewBuilder from the locally hosted samples directory (I’ve obfuscated some of the path and file names):

{"ts":1647511861664,"l":"info","ctx":"AssetPreviewBuilder(144)","msg":"build_(): asset file candidates:","meta":["psfile://3dRZK1qJ8/Professional%20portraits/C%20P/2009%2008%2016%20XXXX%20at%206%20months/dms%2012.jpg","psfile://3dRZK1qJ8/Professional%20portraits/C%20P/2009%2008%2016%20XXXX%20at%206%20months/dms%2013.jpg"]}
{"ts":1647511861682,"l":"info","ctx":"AssetPreviewBuilder(144)","msg":"_build(/media/photos/samples/Professional portraits/C P/2009 08 16 XXXX at 6 months/dms 12.jpg)","meta":{"uri":"psfile://3dRZK1qJ8/Professional%20portraits/C%20P/2009%2008%2016%20XXXX%20at%206%20months/dms%2012.jpg"}}
{"ts":1647511861693,"l":"info","ctx":"UpdateAsset(144)","msg":"previews built","meta":{"assetPreviewInfo":{"assetFileId":167,"assetId":144,"uri":"psfile://3dRZK1qJ8/Professional%20portraits/C%20P/2009%2008%2016%20XXXX%20at%206%20months/dms%2012.jpg","width":2165,"height":1578,"mimetype":"image/jpeg","sha":"8q4xip+sKoB7CCt/074/TirvD9G5cjs3","rotation":0,"path":"/media/photos/samples/Professional portraits/C P/2009 08 16 XXXX at 6 months/dms 12.jpg","mtime":1647510860003,"filesize":1927990,"fitSizes":"hd,qvga"}}}

A failing AssetPreviewBuilder from the NFS-mounted photos share (I obfuscated the IP address):

{"ts":1647510215948,"l":"info","ctx":"SyncFileService","msg":"stdinReceiver","meta":{"stdin":"{\"path\":\"/media/photos/2022/2022 01 01/D_IMG_4332.JPG\"}"}}
{"ts":1647510216144,"l":"info","ctx":"AssetFileImporter(/media/photos/2022/2022 01 01/D_IMG_4332.JPG)","msg":"Setting timeout to 1 minute"}
{"ts":1647510216409,"l":"info","ctx":"AssetFileFinder(2022/2022 01 01/D_IMG_4332.JPG)","msg":"No existing asset yet. Fetching advisory locks."}
{"ts":1647510216411,"l":"info","ctx":"AdvisoryLock","msg":"withLocks()","meta":{"lockNames":["bname:d_img_4332","at:202201011334.072","sha:Ga1IiFBFeBOfemIoMIttvdwEz0Ehmg8M"],"acquired":true}}
{"ts":1647510216445,"l":"info","ctx":"AssetFileRepository","msg":"shouldCopy()","meta":{"result":false,"file":"/media/photos/2022/2022 01 01/D_IMG_4332.JPG","whyNot":"copyAssetsToLibrary is false"}}
{"ts":1647510216445,"l":"info","ctx":"AssetFileRepository","msg":"importAssetFile(psnet://192.168.xx.x/mnt/MainPool/MainPoolDataset/PhotosDataset/2022/2022%2001%2001/D_IMG_4332.JPG): importFile was a no-op"}
{"ts":1647510216454,"l":"info","ctx":"AdvisoryLock","msg":"withLocks()","meta":{"lockNames":["assetId:101"],"acquired":true}}
{"ts":1647510216454,"l":"info","ctx":"UpdateAsset(101)","msg":"starting"}
{"ts":1647510216458,"l":"info","ctx":"AssetPreviewBuilder(101)","msg":"build_(): asset file candidates:","meta":["psnet://192.168.x.x/mnt/MainPool/MainPoolDataset/PhotosDataset/2022/2022%2001%2001/D_IMG_4332.JPG"]}
{"ts":1647510216460,"l":"error","ctx":"AssetPreviewBuilder(101)","msg":"throw","meta":{"cause":{},"message":"AssetPreviewBuilder(101) none of the files are valid {\"doNotSend\":true}","fatal":false,"retriable":false,"doNotSend":true,"meta":{"doNotSend":true}}}
{"ts":1647510216463,"l":"warn","ctx":"UpdateAsset(101)","msg":"Failed to update previews. Un-showing asset 101","meta":{"cause":{}}}

Any suggestions for what I can try?

Welcome to PhotoStructure, @devon! Apologies for the glitch (both in v1.1 and this one!)

I wrote the “info” tool to help debug these sorts of issues. I’d run it with --info or --debug logging. Details on how to run it is here:

https://photostructure.com/server/tools/#file-information

It also responds to --help.

Also, the logtail and logcat tools will convert the JSON logfiles into something much easier for humans to parse: https://photostructure.com/server/tools/#logging

It looks like PhotoStructure wasn’t able to drop a volume UUID file into the mountpoint of your NFS drive, so it had to resort to the (less reliable) psnet URI scheme.

Assuming the NFS mountpoint is /mnt/MainPool/, (and you don’t mind adding this file!) run uuid > /mnt/MainPool/.uuid to give that volume a unique id that PhotoStructure can use in future runs: you should see a psfile: URI instead of a psnet: URI.

What are these shenanigans, you ask? Here’s the explainer:

Holler if this doesn’t let your drive get imported!

Also: if you’re willing, can you share more details about your setup (via DM) so I can try to reproduce this issue on my systems? I have an NFS mount that I test with, so it should work, but there are a ton of variables that I may not have thought of.

That looks like the solution, thanks. I added a .uuid and now I can see assets appearing as the scan progresses.

As an aside (and not worth a whole new thread), sometimes entering the container shell doesn’t give interactive output. I’m not sure if that’s because the scan is very busy or something else. I’ll send you a DM about the NFS share.

Edit: I don’t seem to be able to create a DM, maybe my account is too new. Feel free to DM me with what you’d like to know.

I bumped up your account level, in case you want to retry. You can also send an email to hello@photostructure.com.

Thanks again for the assist in debugging the issue!