Version 1.0.0-alpha.6 is out!

mrm · April 13, 2021, 5:24am

Alpha? What’s that?

Remember: alpha builds should be considered extremely experimental, and may not even launch on all platforms.

More information here.

Did you sign up for a subscription?

Thank you for your help in testing! I’ve extended everyone’s free trial period through to the end of April.

Minor general features

Added meta headers to support iOS homescreen.
Have scanned images of older photos? A new datesBeforeAreEstimated setting automatically considers all captured-at times before 1999 to be an “estimated” time, which requires files to have a tighter image correlation to be considered a duplicate of an existing asset variant. This addresses issues like this.

PhotoStructure for Desktops changes

Fixed the stripe checkout background
The stripe.com link in the plans page is now clickable

PhotoStructure for Docker changes

Fixed a couple bogus “PS_LIBRARY_PATH must be set” errors in /settings
If you are seeing file permission problems, temporarily set the environment variable PS_FIX_PERMISSIONS=1 and run docker-compose up (without --detach) or docker run -it photostructure/server:alpha. This will run chmod -R $UID:$GID /ps as root from within docker, so make sure UID and GID are set appropriately. This chmod should address any permission issues if you previously ran a PhotoStructure container as the root user.
Set the environment variable UID=0 if you want to run PhotoStructure as the root user within your docker container, as it has in prior versions.
/ If either the /ps/tmp or /ps/cache directories are bind-mounted, either will be used for the cache directory. This should solve spurious EACCES errors that some alpha testers saw.

Installation instructions

See this for info about subscriptions (including a thank-you discount for alpha testers):

https://forum.photostructure.com/t/v1-0-0-alpha-1-is-out-alpha-testers-assemble/369#say-hello-to-photostructure-plus

What’s next?

There will probably be a couple more alpha builds tomorrow as I switch from dockerhub to Github Actions to build the docker images. Github Actions gives two things:

multi-arch images, (x64 + arm!)
better tagging (so I can tag all builds with the actual specific version number: helpful when people want to revert back to a prior build if an alpha or beta has a show-stopping bug)

After this is working, I’ll make a beta build, and if there aren’t any showstopping bugs found, finally release v1.0.0.

(I’m committing the mortal devops sin of shipping and then going offline shortly afterwards, but the prior docker build was solidly borked, so things can only improve… I’ll be back online tomorrow morning).

Chat!

Wooskey · April 13, 2021, 11:21am

I’m still getting an error.

From Docker log:

{"fatal":true,"exit":true,"status":12,"pid":21,"ppid":14,"error":"ChildService(web).onStdout(): Error: ChildService(web).onStdout()setup() failed: Error: Can't read /ps/app/public. Please visit <https://forum.photostructure.com/> for help.¹⁶"}

Shutting down PhotoStructure...

From PS log:

{"ts":1618311921749,"l":"info","ctx":"Service(main)","msg":"setup()"}
{"ts":1618311921812,"l":"info","ctx":"SettingsIO.importFileSettings(/ps/config/PhotoStructure/settings.toml)","msg":"loaded","meta":{"tomlMap":{"scanMyPictures":false,"scanPaths":["/ps/library","/root/Pictures"]},"imported":[{"name":"scanMyPictures","value":true,"persist":false},{"name":"scanPaths","value":["/ps/library"],"persist":false}]}}
{"ts":1618311921813,"l":"info","ctx":"SettingsIO","msg":"_libraryHasSettings","meta":{"result":false}}
{"ts":1618311921814,"l":"info","ctx":"Service(main)","msg":"setupErrorHandling(): not adding stdin/stdout/stderr close handlers: we're daemonized or in docker."}
{"ts":1618311921814,"l":"warn","ctx":"Sentry","msg":"Failed to set up sentry: TypeError: Cannot read property 'init' of undefined"}
{"ts":1618311921815,"l":"info","ctx":"Service(main)","msg":"setup()","meta":{"version":"1.0.0-alpha.6","start":1618311921237,"argv":["/usr/local/bin/node","/ps/app/bin/main.js"],"arch":"x64","platform":"linux","isDocker":true,"isPacked":true,"isElectron":false,"versions":{"node":"14.16.1","v8":"8.4.371.19-node.18","uv":"1.40.0","zlib":"1.2.11","brotli":"1.0.9","ares":"1.16.1","modules":"83","nghttp2":"1.41.0","napi":"7","llhttp":"2.1.3","openssl":"1.1.1k","cldr":"37.0","icu":"67.1","tz":"2020a","unicode":"13.0"},"settings":{"logLevel":"info","httpPort":1787,"rpcPort":1807,"libraryPath":"/ps/library"},"NODE_ENV":"production","PS_LOG_COLOR":"true","PS_LOG_LEVEL":"info","PS_REQUIRE_MAKE_MODEL":"0","PS_SCAN_ALL_DRIVES":"1","PS_SCAN_MY_PICTURES":"1","PS_SCAN_PATHS":"/ps/library","PS_SYNC_INTERVAL_HOURS":"2"}}
{"ts":1618311921832,"l":"info","ctx":"RpcServer","msg":"Setting up RPC..."}
{"ts":1618311921836,"l":"info","ctx":"rpc.Server","msg":"listening on 1807"}
{"ts":1618311921836,"l":"info","ctx":"RpcServer","msg":"RPC service serving port 1807"}
{"ts":1618311921838,"l":"info","ctx":"ChildService","msg":"pathToService()","meta":{"result":{},"cmd":"web.js","isPacked":true,"dirs":["/ps/app/bin/web.js","/ps/app/app.asar/web.js","/ps/app/dist/app/web.js"]}}
{"ts":1618311921841,"l":"info","ctx":"WatchedChild(web)","msg":"_restart()","meta":{"stopped":false,"ended":false}}
{"ts":1618311921841,"l":"info","ctx":"WatchedChild(web)","msg":"_stop()","meta":{"stopped":false,"ended":false}}
{"ts":1618311921841,"l":"info","ctx":"WatchedChild(web)","msg":"restart()","meta":{"startRate":{"ttlMs":120000,"_eventCount":0,"_lastEventTime":0,"eventDeltas":{"ttlMs":120000,"times":[],"a":[]}},"maxErrorsPerMinute":3}}
{"ts":1618311921842,"l":"info","ctx":"WatchedChild(web)","msg":"_start()","meta":{"stopped":false,"ended":false}}
{"ts":1618311921848,"l":"info","ctx":"WatchedChild(web:34)","msg":"_start(): spawned pid 34"}
{"ts":1618311922809,"l":"info","ctx":"rpc.Server","msg":"Connection from IPv6:::ffff:127.0.0.1:58346"}
{"ts":1618311921832,"l":"debug","ctx":"Service(main)","msg":"setup done.","meta":{"reject":false}}
{"ts":1618311921833,"l":"debug","ctx":"rpc.Server","msg":"start(1807)"}
{"ts":1618311921841,"l":"debug","ctx":"WatchedChild(web)","msg":"isErrorRateExceeded()","meta":{"result":false,"maxErrorsPerMin":3}}
{"ts":1618311921844,"l":"debug","ctx":"ChildProcess","msg":"spawn()","meta":{"command":"/usr/local/bin/node","args":["/ps/app/bin/web.js"],"maxAgeMs":-1,"opts":{"env":{"PS_SCAN_MY_PICTURES":"false","NODE_VERSION":"14.16.1","HOSTNAME":"86aab5c9cacf","YARN_VERSION":"1.22.5","SHLVL":"2","HOME":"/root","PS_IS_DOCKER":"1","PS_SCAN_ALL_DRIVES":"true","PS_SCAN_PATHS":"[\"/ps/library\",\"/root/Pictures\"]","PS_LOG_LEVEL":"info","PS_SYNC_INTERVAL_HOURS":"2","PID":"1000","PATH":"/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/lib/libraw:/opt/local/sbin:/opt/local/bin:/usr/sbin:/sbin","PS_LOG_COMPRESS":"1","GID":"1000","UID":"1000","PS_MAX_CPU_PERCENT":"100","PS_FIX_PERMISSION":"1","PS_HOST_SCAN_PATHS":"/ps/library","PWD":"/ps/app","PS_REQUIRE_MAKE_MODEL":"false","NODE_ENV":"production","VIPS_MIN_STACK_SIZE":"2m","VIPSHOME":"/target","PS_LOG_COLOR":"true"},"detached":false,"shell":false}}}
{"ts":1618311922103,"l":"debug","ctx":"Pids","msg":"addPid() wrote /ps/config/PhotoStructure/pids/34.json","meta":{"pid":34,"cmd":"node","maxAgeMs":-1,"ppid":21,"startTime":1618311921848}}
{"ts":1618311922818,"l":"error","ctx":"WatchedChild(web:34)","msg":"onError()","meta":{"src":"ChildService(web).onStdout()","fatal":true,"ignorable":false,"errToS":"setup() failed: Error: Can't read /ps/app/public. Please visit <https://forum.photostructure.com/> for help.¹"}}
{"ts":1618311922820,"l":"info","ctx":"Error","msg":"isFatalErrorAllowed()","meta":{"result":false,"mainService":true,"postProbation":false,"lowErrorRate":false,"fatalErrorRatePerMinuteSetting":10}}
{"ts":1618311922821,"l":"error","ctx":"Error","msg":"onError(): Error: ChildService(web).onStdout()setup() failed: Error: Can't read /ps/app/public. Please visit <https://forum.photostructure.com/> for help.¹⁶\nError: ChildService(web).onStdout()setup() failed: Error: Can't read /ps/app/public. Please visit <https://forum.photostructure.com/> for help.¹⁶\n    at k.onError (/ps/app/bin/main.js:9:134369)\n    at L.onStdout (/ps/app/bin/main.js:9:131279)\n    at s.onData (/ps/app/bin/main.js:9:138359)\n    at /ps/app/bin/main.js:9:245567\n    at Array.forEach (<anonymous>)","meta":{"event":"fatal","message":"ChildService(web).onStdout()"}}
{"ts":1618311922826,"l":"warn","ctx":"Service(main)","msg":"exit()","meta":{"status":12,"reason":"ChildService(web).onStdout(): Error: ChildService(web).onStdout()setup() failed: Error: Can't read /ps/app/public. Please visit <https://forum.photostructure.com/> for help.¹⁶","waitForJobs":false,"ending":false}}
{"ts":1618311922826,"l":"info","ctx":"WatchedChild(web:34)","msg":"_stop()","meta":{"stopped":false,"ended":true}}
{"ts":1618311922830,"l":"info","ctx":"\u001b[35mEndable\u001b[39m","msg":"endEndables()","meta":{"isTest":false,"isSingleSpecTests":false}}
{"ts":1618311922834,"l":"info","ctx":"PromiseTimer","msg":"timings:\n","meta":{"fs.readFile":{"sumSec":0.047,"mean":47,"k":1}}}
{"ts":1618311922842,"l":"info","ctx":"rpc.Server","msg":"Closing connection from IPv6:::ffff:127.0.0.1:58346"}
{"ts":1618311922842,"l":"warn","ctx":"DirectoryEntry","msg":"children() failed to readdir(/ps/config/PhotoStructure/pids)","meta":{"errno":-13,"code":"EACCES","syscall":"mkdir","path":"/ps/tmp/readdircache","stack":["Error: EACCES: permission denied, mkdir '/ps/tmp/readdircache'"]}}
{"ts":1618311923136,"l":"info","ctx":"WatchedChild(web:34)","msg":"onExit","meta":{"code":0,"stopped":false,"ended":true}}
{"ts":1618311923136,"l":"info","ctx":"WatchedChild(web:34)","msg":"_restart()","meta":{"stopped":false,"ended":true}}
{"ts":1618311923136,"l":"info","ctx":"WatchedChild(web:34)","msg":"_stop()","meta":{"stopped":false,"ended":true}}
{"ts":1618311923136,"l":"info","ctx":"WatchedChild(web:34)","msg":"onExit(): finished setting up new child undefined"}

I’m guessing it’s due to permissions but haven’t checked yet. I have to get to work and can look into it more tonight.

GavinCampbell · April 13, 2021, 1:25pm

I’m up and running. Did a fresh install and its scanning in my library now.

While scanning though I can’t get rid of the “Your library is currently empty” message even though there are now things in the library.

mrm · April 13, 2021, 6:14pm

Did you pick up alpha.5, or alpha.6? I believe this is fixed in alpha.6.

Also: you may want to start using the UID/GID support I just added in alpha.4. Here’s my run-docker.sh:

#!/bin/sh -x

PSLIBRARY="$HOME/Pictures" # < CHANGE THIS LINE
mkdir -p $PSLIBRARY

# This must be a fast, local disk with many gigabytes free:
PSTMP="/tmp/ps-$USER"
mkdir -p "$PSTMP"

# This directory stores your "system settings". This directory must not be
# the same as the one used by PhotoStructure for Desktops.
PSCONFIG="$HOME/.config/PhotoStructure-docker"
mkdir -p "$PSCONFIG"

# This directory stores PhotoStructure logfiles.
PSLOGS="$HOME/.config/PhotoStructure/logs"
mkdir -p "$PSLOGS"

docker run \
  --stop-timeout 120 `# < gives PhotoStructure 2 minutes to shut down cleanly.` \
  --publish 1787:1787 \
  -e TZ="$(cat /etc/timezone)" \
  -e UID="$(id -u)" `# < makes PhotoStructure write library files as the current user (instead of using --user)` \
  -e GID="$(id -g)" \
  -e PS_LOG_LEVEL=debug \
  -v "$PSLIBRARY":/ps/library \
  -v "$PSTMP":/ps/tmp \
  -v "$PSCONFIG":/ps/config \
  -v "$PSLOGS":/ps/logs \
  photostructure/server:alpha

Wooskey · April 13, 2021, 10:24pm

I don’t know how to tell which alpha I got. I deleted the image and container and re-pulled but get the same results. I can confirm I’m pulling from the :alpha channel and can tell you it was created 15 hours ago.

I still get the same error. Now I’m guessing it has nothing to to with Photostructure, but instead something to do with my environment.

Const · April 14, 2021, 2:19pm

./photostructure --version
1.0.0-alpha.6

or click on UI “burger” menu on top left corner and scroll to third section “about…”

Lars_Noschinski · April 14, 2021, 5:21pm

I just tried out the “View by folder” function: Is it intentional that I see a tag “fs / Library”? This path seems to refer to the PhotoStructure library itself?

My docker bind mounts containing photos are:

      - type: bind
        source: /volume2/phstr-library/
        target: /ps/library

      - type: bind
        source: "/volume1/homes/lars/Fotos/"
        target: /media/fotos-lars-backup

      - type: bind
        source: "/volume1/photo/"
        target: /media/fotos-photostation

mrm · April 14, 2021, 5:30pm

Yes. The library can be mounted at different directories, so the tag is relative to the library root (as are the AssetFile.URI references).

If you can think of a better or less surprising way to handle these paths, please suggest!

Lars_Noschinski · April 14, 2021, 7:08pm

I think I was confused because I didn’t expect the library itself to be present in the fs hierarchy, But since you can move stuff around there, it makes sense.

One thought I had which might or might not make sense on closer consideration: Depending on how (dis)organized the source libraries are, the user might prefer not to see the moint points of the source libraries in the tags:

For example:

I have my private photos mounted under /media/photos and business related photos under /media/business. In this case, I’d probably like to see fs/photos/... and fs/business/..., as it is currently implemented.
If I like to organize my photos by year/month, but am a bit messy and have my library spread around various disks, I have maybe mountpoints /media/disk1 to /media/diskn, each with a year/month structure below the mountpoint. Then I’d probably like to see fs/year/month/....

So, a library option (or maybe both tags in parallel) might be an interesting feature.

mrm · April 14, 2021, 7:37pm

Assets will be tagged with all the “asset file variants” associated to that asset:

So if you’ve got duplicates in /media/photos, /media/backups1, /media/backups2, … the asset will show up for all those tags (thanks to hierarchical inheritance).

You can see these tags in the asset info panel:

Lars_Noschinski · April 14, 2021, 7:55pm

I was not thinking about duplicates but rather the mount point being an uninteresting part of the path.

Maybe a bit more concrete. In a hierarchy like

/media/disk1/2018/birthday-lars
/media/disk1/2018/vacation-vegas
/media/disk1/2019/walk-in-the-park
/media/disk2/2019/birthday-matthew
/media/disk2/2021/corona-vaccination

the mount point would make browsing more cumbersome (since I’m probably not interested where the image was originally stored). One could image a separate top-level tag (folder?) for that.

That being said, this should probably be a (low prio) feature request instead of clogging up this thread.

mrm · April 14, 2021, 8:00pm

Agreed: it’s stored as a “volume UUID SHA” (which why your URL looks like “tag/fs/2NMQsMVCK/home”). PhotoStructure stores the last-seen mountpoint for that volume as the “display name” for the mounpoint.

I included the mountpoint/volsha in the fs tag because it was helpful for me to be able to browse only the assets on a specific drive. Do you want to perhaps collapse all the mountpoints into a single virtual “drive”?

I could add a setting that says “include volsha in filesystem tags” (which defaults to true, and you could set it to false). Would that address this? If so, feel free to add a new feature request!

mrm · April 15, 2021, 8:26am

Did you figure out what was going on?

The home page is supposed to detect progress and automatically re-fetch assets and display them as they are imported. Did you need to hit refresh on your browser?

mrm · April 15, 2021, 8:29am

I’m hoping to release alpha.7 tomorrow:

https://photostructure.com/about/2021-release-notes/#v100-alpha7

GavinCampbell · April 15, 2021, 11:23am

The message continued to appear even after the initial scan was done. I did everything from trying different browsers to refreshing the page.

Once I restarted the container though it went away and all is good. So not sure why it initially was stuck but after a restart it cleared itself. So I wouldn’t worry about it too much.

It was on a fresh install though.

mrm · April 15, 2021, 4:26pm

Ok, I’ll try to reproduce that today.

Is your library stored on a local volume or a remotely mounted filesystem?

GavinCampbell · April 15, 2021, 9:26pm

It was a docker, so I had it mounted via a path. The docker runs on the unraid server that hosts the files too.