Version 2.1.0-alpha is ready for testing!

Alpha? What’s that?

Please note: alpha builds should be considered extremely experimental, and may not even launch on all platforms.

More details about alpha builds are here: Alpha, beta, stable, and latest; What should you use?

Questions? Comments?

Feel free to reply here or on our “alphabeta” room in Discord : PhotoStructure

What’s in v2.1?

Version 2.1 brings almost 100 improvements and bugfixes: lines of code and files changed isn’t a great metric, but it’s directionally correct:

sync improvements

Improvements and bugfixes:

Release notes

The comprehensive release notes are here:

Installation instructions

Please take a backup of your system before continuing!

PhotoStructure for Desktop users

  1. Shut down PhotoStructure
  2. Download and install: https://photostructure.com/install/?alpha

Running the alpha installer will replace the currently installed version. PhotoStructure for Desktops will detect that it’s an alpha build, and automatically upgrade you to newer alpha or beta builds, and then “stick” to stable builds (once v2.1-stable is released).

PhotoStructure for Docker users

  1. Shut down PhotoStructure
  2. Run docker pull photostructure/server:alpha in a terminal
  3. Change your docker run command to use photostructure/server:alpha

PhotoStructure for Docker Compose users

  1. docker-compose down
  2. Edit your docker-compose.yml to use image: photostructure/server:alpha
  3. docker-compose pull ; docker-compose up -detach

PhotoStructure for Node users

  1. Make sure you’ve got either Node.js 14.19.2 or Node.js 16.15.0 installed. If you’re using nvm, run nvm install 16 ; nvm alias default node to get the latest.

  2. Shut down PhotoStructure

  3. cd into your photostructure-for-servers directory, and run git checkout alpha ; ./start.sh

OMG I upgraded and my screen is blank!

If you’re accessing your library via http, know that newer versions of Chrome and Firefox will include a new Upgrade-Insecure-Requests header. If you open your browser’s developer console, you’ll see something like this:

Version 2.1 has a bunch of settings to tweak your CSP and CORS headers.

  • First try setting upgradeInsecureRequests=false. This should fix it.

  • Then try setting cspReportOnly=true. This disables all CSP protections (although they don’t really matter if you’re just using http)

  • If all else fails, enableWebSecurity=false is the “nuclear” option, and disables all security headers.

4 Likes

Bugs found in v2.1.0-alpha

I’ll get these sorted as soon as I can. If you see other bugs, please post here or in discord!

  • papaparse not found (fixed in alpha.1)

  • Docker build takes 40 minutes for GitHub Actions to release (the fix requires extracting the prebuild step into a new published base-tools image) (fixed in next build)

  • The docker v2.1.0-alpha.1 build doesn’t seem to have been tagged with :alpha (fixed in next build)

  • Windows fails to launch with “SYSTEMROOT not set” (issue with batch-cluster and new Node 16 windows fork behavior, fix soon)

  • look into @avdp’s SQLite corruption issue (can you start with a new library, and if it recurs, send me debug logs, please?)

  • Remove assets button results in “lost network connection” (via @adamf)

  • Requesting a new sync or rebuild from the web doesn’t seem to work (via @Nighthawk)

  • About page has a caution icon by the library directory on docker

  • About page sync status incorrectly states all directories are stale

  • Progress panel doesn’t show parade of newly-imported thumbnails

Features that need testing

Docker/RPi support

Deletion/Trash can support

  • Do “deleted” files on local disks on macOS end up properly in the trash can? Does “Put Back” work?

  • Do “deleted” files on local disks on Windows end up properly in the trash can? Does “Undo Delete” work?

  • Do “deleted” files on local disks on Ubuntu/gnome end up properly in the trash can? Does “Restore from trash” work?

Sync Reports

  • Can people open up their sync report with their favorite spreadsheet app?

  • Does it help answer the “why is my file not in my library” question?

De-duplication

New full-screen mode

  • please click the full screen button in the upper right header bar, and then continue to browse your library. If it doesn’t behave as expected, please report OS, browser, and what went wrong.

I got a sqlite corruption while navigating through the UI. Restarting the container (on UNRAID) cleared out the issue for now. Nevertheless it seems the sqllite issues are not behind us with 2.1.0-alpha

{"toast":{"text":"Well, darn, that didn't work","uuid":"SqliteError: SQLITE_CORRUPT: database disk image is malformed","details":"SqliteError: SQLITE_CORRUPT: database disk image is malformed","type":"warning","button1Text":"Return home","button1Url":"/tag#dismiss","button1Class":"info","onClickUrl":"#dismiss"}}

Same problem here

@avdp is your library on local disk? I want to try to reproduce that on my UnRAID test box.

Yes, physical - nvme to be precise. And I did the lazy docket setup (one mapped library folder)

Testing on Ubuntu 22.04 LTS - Desktop Version.

Bug, I think: Sync => Pause processing/Resume & Restart Sync is not doing anything for me. This is an issue when you have a big library and the process got interrupted. Even after a restart (app and system) it doesn’t continue with the import/scan process. A “forced” restart sync from the menu doesn’t help either.

It seems “exposeNetworkwithoutAuth” is also not working…it generates an URL, but the page stays white. Tested on iPad, Android, Win 10 with Chrome, Safari, Firefox and Edge.

1 Like

I’ve just added that to the buglist, and I’ll look into it asap.

Ah! Holler if this doesn’t fix the issue: https://forum.photostructure.com/t/version-2-1-0-alpha-is-ready-for-testing/1456#omg-i-upgraded-and-my-screen-is-blank-11

1 Like

I did place this in the settings.toml and it worked.
RTFM, I guess. My sincere apologies.

I did the exact same thing.

No worries at all: I appreciate your help in banging on the alpha build, and I suspect this will trip up more people.

Is there a safe way I can default to making things “work by default”, I wonder?

For reference, v2.1 now has the following CSP/CORS/ “web security” settings:

  • upgradeInsecureRequests, if true, adds the upgrade-insecure-requests CSP directive. If false, the directive is omitted. CSP linters suggest that it be included.

  • cspReportOnly, if true, uses the Content-Security-Policy-Report-Only header rather than the standard Content-Security-Policy. CSP violations are only reported. This should only be set to true temporarily for debugging CSP errors.

  • enableWebSecurity: this defaults to true if you use --expose or exposeNetworkWithoutAuth=true. If explicitly set to false, it changes the default of cspReportOnly to true and upgradeInsecureRequests to false. If you’re accessing your PhotoStructure library via the web interface, you want this set to true.

So, with that out of the way–perhaps I default upgradeInsecureRequests to false?

After explaining this, I think enableWebSecurity is too confusing and complicated: I’m going to delete it.

Related: did people know that there’s a trustProxy setting? It currently defaults to false because I’m chicken, but I think defaulting it to loopback might be almost as safe of a default. More defaults about trustProxy are here.

Upgrade insecure requests assumes that you are capable of serving over https. How many of your users do you think are using https? I would leave it out until such a time as Photostructure supports https out of the box.

1 Like

@mrm - by the way, this was with a brand new library (I always wipe everything out between version upgrades). When I restarted the container it continued where it left off and it has since complete successfully. I will retry from scratch again this weekend, just to see if I reproduce.

@mrm - I wiped the library out and recreated the container. The sqllite error re-occured. I will DM a link to the logs off my google drive.

I take that back. After some testing it works for me, but I changed now from CIFS to NFS, which might or might not have something to do with it.

Is it just me or is the import process much slower than in other versions? It’s running now for 2 hours or so and it feels significantly slower.

Update: I might understand now why it feels slow. It seems that (obviously) videos take much more time to import. Once that is done, the progress on the photos is a lot faster…so it’s a perception thing…

I did a full rebuild with the new version and it seemed significantly faster… but that was just perceived.

1 Like

Hello @mrm

I have now imported my library with 36k images and ~1k videos. The sync-report is quite helpful, thank you!

Some observations:

  • After ~26 hours and 19k processed pictures the process just stopped and I had to force a re-sync, after another 12 hours the remaining pictures got processed
  • I have quite a few pictures that got rejected with “dimensions too small” or “file too small”
  • Seems like “wmv” is not supported => “not supported mime type, dimensions too small”
  • Seems lime “bmp” is not supported => “not photo or video extension”
  • I have quite a few pictures that failed with “Cannot build previews: none of the files are valid: VipsJpeg: Invalid SOS parameters for sequential JPEG²⁶” (happy to provide an example file per PM message)

Building on my observations a few questions/comments:

  • Many videos got rejected because they are too big or too small. In fact I would actually prefer to not process videos at all, but “link” to the original source instead and displaying a thumbnail only. This would save a lot of processing time and keep the overall footprint small. I understand that not all users would like to have it like that, but as an option I would appreciate that
  • It would be helpful too understand what parameters to tweak if you have a lot of CPU power, if you have a lot of RAM, or even both
  • Adding folders to exclude (also ex-post) as per my PM would be awesome

Overall it runs so far very stable and does what it should. :+1:

Many videos got rejected because they are too big or too small. In fact I would actually prefer to not process videos at all, but “link” to the original source instead and displaying a thumbnail only.

You can tell photostructure not to transcode videos. Either set transcodeVideos=false in the settings file, or set the PS_TRANSCODE_VIDEOS environment variable to false. (depending on how you’re running photostructure)

Please do send me any photos or videos whose dimensions are not being extracted correctly!

As @avdp stated, there’s a PS_TRANSCODE_VIDEOS which is the big switch to disable all transcoding, but you can configure which video types get transcoded, and at what bitrate, via the following settings:

  • transcodeBitrateQVGA
  • transcodeBitrateUHD
  • doNotTranscodeMimeTypes
  • doNotTranscodeVideoCodecs
  • doNotTranscodeAudioCodecs

I added it! See New in v2.1: file globbing