V1.0.0-alpha.1 is out! Alpha testers: assemble

mrm · March 15, 2021, 1:55am

Alpha? What’s that?

Remember: alpha builds should be considered extremely experimental, and may not even launch on all platforms.

Read more about alpha, beta, and stable releases here.

Questions? Comments?

Feel free to reply here or to the new discord server: PhotoStructure

What’s in 1.0.0-alpha.1?

A lot:

New .heif support
New expanded RAW image support
New curators: person tag extraction and parsing
Browse-by-filesystem
40+ minor features and bugfixes
PhotoStructure’s infrastructure was updated, which pulled in a bunch of security updates, bug fixes, and some performance improvements

Details here: https://photostructure.com/about/2021-release-notes/#v100-alpha0

Say hello to PhotoStructure Plus!

Version 1.0.0 is the first version that has a paid tier of functionality. As discussed previously, there is a free tier, “PhotoStructure lite”, and a paid tier, “PhotoStructure plus.”

There is a new welcome-page flow, (if PhotoStructure is running on your computer, visit http://localhost:1787/welcome) that takes you to the new PhotoStructure account billing management site. You’ll then pick a plan, and if you choose “plus”, you’ll need to verify your email to log in (a verification code will be emailed to you), and then be returned to your library (along with a license that is activated for your library). You can also activate a “plus” plan from the nav menu or via the About page.

Note that PhotoStructure partners with Stripe for billing. Subscription details are here.

As a thank-you to beta testers, here’s a $10 off coupon: IHAVEBACKUPS10. This will give you

2 months for free, if you pick the $5/month plan, or
25% off the already discounted $42/year annual price

There are also discounts available for

PhotoStructure testers that have shared feedback or reported on bugs and issues
Educators and students
PhotoStructure contributors

Contact me: I’ll get you set up!

Installation instructions

Please take a backup of your system before continuing!

PhotoStructure for Desktop users

(alpha.2 has working macOS and Windows builds)

If you have HEIF/HEIC files, follow these instructions: PhotoStructure | HEIF/HEIC support with PhotoStructure
Download and install: https://photostructure.com/go/alpha

PhotoStructure for Docker users

Shut down PhotoStructure
Run docker pull photostructure/server:alpha in a terminal
change your docker run command to use photostructure/server:alpha

PhotoStructure for Docker Compose users

docker-compose down
Edit your docker-compose.yml to use image: photostructure/server:alpha
docker-compose pull ; docker-compose up -detach

PhotoStructure for Node users

Note that we’ve renamed our primary branch to main. If you have an older git clone, you’ll need to

$ git branch -m master main
$ git fetch
$ git branch --unset-upstream
$ git branch -u origin/main

Make sure you’ve got Node.js 14.16.0 or later installed. If you’re using nvm, run nvm install 14 to get the latest.
Install the new dependencies. If you’re on Ubuntu:

$ sudo apt install libraw-bin libheif-examples

Switch to the alpha branch by shutting down, and then running

$ git checkout alpha ; start.sh

Important changes

New welcome → plans → settings page flow

If you go to your library, and change the URL path to /welcome (so something like http://localhost:1787/welcome), you’ll see the new user page flow. Feedback is welcome!

Reverse proxy users

There’s a new trustProxy networking setting. If you run PhotoStructure behind a reverse proxy, you’ll need to set the PS_TRUST_PROXY environment variable to loopback, 1, or your local subnet. See http://expressjs.com/en/guide/behind-proxies.html for details.

If you don’t set this, PhotoStructure won’t know what the requested URL is, and when you click “return to your library”, it’ll go to the wrong URL.

bdillahu · March 15, 2021, 2:56am

I know what I’m doing during those boring meetings tomorrow

shrippen · March 15, 2021, 8:34am

I’m getting the following error when starting the new docker image:

{"fatal":true,"exit":true,"status":12,"pid":13,"ppid":6,"error":"Library.setup() failed¹: Error: Library is already opened by 61ea7e556a55 (sync:21079) (library setup)¹⁶²"}

Error: Library is already opened by 61ea7e556a55 (sync:21079) (library setup)¹⁶²

Error: Library is already opened by 61ea7e556a55 (sync:21079) (library setup)¹⁶²

at kh.throwIfUnavailable (/OpenedBy.ts:321:13)
at fb.thunk (/Library.ts:146:7)
{"fatal":true,"exit":true,"status":14,"pid":13,"ppid":6,"error":"main setup failed: Error: Library is already opened by 61ea7e556a55 (sync:21079) (library setup)¹⁶²"}

Running with docker on unraid. Had the beta installed before and just changed the repository to photostructure/server:alpha

bdillahu · March 15, 2021, 2:33pm

If you are on Unraid:

Change the “repository” to be:

photostructure/server:alpha

And it should work I believe.

shrippen · March 15, 2021, 3:05pm

Did exactly that and I got the mentioned error

tkohhh · March 15, 2021, 3:06pm

I’m getting the same error on Unraid:

{"fatal":true,"exit":true,"status":12,"pid":13,"ppid":6,"error":"Library.setup() failed¹: Error: Library is already opened by f3230df956b2 (sync:2172) (library setup)¹⁶²"}

bdillahu · March 15, 2021, 3:32pm

Where are you seeing this error?

Mine is busy rebuilding the library and “just seems to be working”… did nothing to it except change that flag.

I did have it shut down when I made the change… only thing I can think of.

tkohhh · March 15, 2021, 3:39pm

It’s in the container log. Restarting the container has no impact.

My process was:

Stop the container
Update the tag to “alpha” and watch the packages install
Start the container
Observe that the UI was not loading, check the log, see error.

mrm · March 15, 2021, 3:42pm

“library is already open” style errors could happen if:

you force-kill PhotoStructure. It has stuff out needs to do on shutdown, and if you just kill the container, it’ll leave the db in a bad state.
there’s a bug in the library lock code. If it can’t reliably determine the current hostname, it’ll think some other box has the library open.

If you wait 5 mintutes, the prior open lock will be considered stale and it shouldn’t show you that error anymore, but it’s probably indicative of a forced shutdown. The docker arg --stop-timeout 120 should fix it.

tkohhh · March 15, 2021, 3:47pm

I’m still getting the error… it’s been over 40 minutes. What else can we try?

tkohhh · March 15, 2021, 3:52pm

ok, I did get it running… not sure why this helped with the error, but adding the argument you suggested seemed to jar something loose.

Const · March 15, 2021, 3:53pm

I am on Ubuntu running Node version. Updated to new dependencies, but it looks like it still requires old dcraw?

~/photostructure-for-servers$ ./start.sh
Please install the system prerequisites. (missing commands: dcraw_emu)
See https://photostructure.com/server/photostructure-for-node/ or send an email to support@photostructure.com for help.

I am on correct branch, am I?

~/photostructure-for-servers$ git status
On branch alpha
Your branch is up to date with ‘origin/alpha’.
nothing to commit, working tree clean

shrippen · March 15, 2021, 4:03pm

Hm, I added the --stop-timeout 120 arg and watched the log when I stopped the container with the old beta version. The log showed me a successful verification of the database and backed it up. So that doesn’t seem to be a forceful shutdown. So I suspect that maybe the hostname changes?

Unraid has the habit of changing the name (the internal name I guess) of docker containers when settings are updated. Which you can see here, this is the log of starting up the 0.9 beta and then shutting it down and changing the repository to the alpha.

PhotoStructure is ready: <http://localhost:1787/>

Shutting down PhotoStructure...

Shutting down PhotoStructure...
Verifying & backing up /ps/tmp/local-db/models/db.sqlite3...
e":"No such container: e03772b74f62"}
e":"No such container: e03772b74f62"}
e":"No such container: e03772b74f62"}
e":"No such container: e03772b74f62"}
e":"No such container: e03772b74f62"}
e":"No such container: e03772b74f62"}

(The e":"No such container errors happen when the log window tries to show the log for a not existing docker.)

This is the log of the “new” container with the alpha:

{"fatal":true,"exit":true,"status":12,"pid":13,"ppid":6,"error":"Library.setup() failed¹: Error: Library is already opened by e03772b74f62 (sync:35) (library setup)¹⁶²"}

{"fatal":true,"exit":true,"status":12,"pid":13,"ppid":6,"error":"Library.setup() failed¹: Error: Library is already opened by e03772b74f62 (sync:35) (library setup)¹⁶²"}

Error: Library is already opened by e03772b74f62 (sync:35) (library setup)¹⁶²

Error: Library is already opened by e03772b74f62 (sync:35) (library setup)¹⁶²

at kh.throwIfUnavailable (/OpenedBy.ts:321:13)
at fb.thunk (/Library.ts:146:7)
{"fatal":true,"exit":true,"status":14,"pid":13,"ppid":6,"error":"main setup failed: Error: Library is already opened by e03772b74f62 (sync:35) (library setup)¹⁶²"}

The container name has changed to b3493b5e7a3c - so I guess that’s why this db lock happens?

I will see if waiting for 5 minutes changes anything.

Edit:
I “fixed” it by manually deleting the jsons insinde the opened-by folder. Could start up the docker successfully.

John · March 15, 2021, 4:04pm

Worked fine here on Unraid.

It is telling me it going to take 6 days to rebuild my library

64k “assets” but its hardly taxing the box at all in terms or RAM or CPU (set at the 75% default resource limit).

mrm · March 15, 2021, 5:44pm

dcraw_emu is actually provided by libraw-bin. Did you run

sudo apt install libraw-bin libheif-examples

?

(edit: previously said libraw, but we want libraw-bin)

mrm · March 15, 2021, 5:47pm

OOF

How many CPUs did you give your VM? PhotoStructure will only schedule ~75% utilization of the visible resources, so if it’s light on CPU or RAM, it’ll only run a single import thread.

You can see what it’s thinking by opening the about page via the nav menu. Here’s my workstation:

Const · March 15, 2021, 5:50pm

Thanks, yeah. I figured this actually failed and still failing for me.

sudo apt install libraw libheif-examples
Reading package lists… Done
Building dependency tree
Reading state information… Done
E: Unable to locate package libraw

mrm · March 15, 2021, 5:56pm

Ugh, that’s no good: and, frustratingly, I didn’t touch the library locking code between v0.9.1 and v1.0.0, so I don’t know what’s going on there.

For now, just as a bodge, there’s a setting to ignore library locks on startup: PS_FORCE_OPEN:

DANGEROUS: if set, all previously-existing library locks will be removed. This should only be necessary if the prior PhotoStructure process was not shut down gracefully.

I’d like to know why it’s having this issue, though: if you can set PS_LOG_LEVEL=debug, run a couple times to show the error, and send me those logs, that’d be a big help.

mrm · March 15, 2021, 5:59pm

Crap, I meant libraw-bin, I’ll fix my post now.

$ apt-file search dcraw_emu
libraw-bin: /usr/lib/libraw/dcraw_emu

And you’re running ubuntu 20.04 LTS, right?

John · March 15, 2021, 6:00pm

snap on the processor. I’m not using any docker CPU pinning so it should have free reign over the 75%.