Version 2.1.0-alpha.1 is ready for testing!

@mrm - I wiped the library out and recreated the container. The sqllite error re-occured. I will DM a link to the logs off my google drive.

I take that back. After some testing it works for me, but I changed now from CIFS to NFS, which might or might not have something to do with it.

Is it just me or is the import process much slower than in other versions? It’s running now for 2 hours or so and it feels significantly slower.

Update: I might understand now why it feels slow. It seems that (obviously) videos take much more time to import. Once that is done, the progress on the photos is a lot faster…so it’s a perception thing…

I did a full rebuild with the new version and it seemed significantly faster… but that was just perceived.

1 Like

Hello @mrm

I have now imported my library with 36k images and ~1k videos. The sync-report is quite helpful, thank you!

Some observations:

  • After ~26 hours and 19k processed pictures the process just stopped and I had to force a re-sync, after another 12 hours the remaining pictures got processed
  • I have quite a few pictures that got rejected with “dimensions too small” or “file too small”
  • Seems like “wmv” is not supported => “not supported mime type, dimensions too small”
  • Seems lime “bmp” is not supported => “not photo or video extension”
  • I have quite a few pictures that failed with “Cannot build previews: none of the files are valid: VipsJpeg: Invalid SOS parameters for sequential JPEG²⁶” (happy to provide an example file per PM message)

Building on my observations a few questions/comments:

  • Many videos got rejected because they are too big or too small. In fact I would actually prefer to not process videos at all, but “link” to the original source instead and displaying a thumbnail only. This would save a lot of processing time and keep the overall footprint small. I understand that not all users would like to have it like that, but as an option I would appreciate that
  • It would be helpful too understand what parameters to tweak if you have a lot of CPU power, if you have a lot of RAM, or even both
  • Adding folders to exclude (also ex-post) as per my PM would be awesome

Overall it runs so far very stable and does what it should. :+1:

Many videos got rejected because they are too big or too small. In fact I would actually prefer to not process videos at all, but “link” to the original source instead and displaying a thumbnail only.

You can tell photostructure not to transcode videos. Either set transcodeVideos=false in the settings file, or set the PS_TRANSCODE_VIDEOS environment variable to false. (depending on how you’re running photostructure)

Please do send me any photos or videos whose dimensions are not being extracted correctly!

As @avdp stated, there’s a PS_TRANSCODE_VIDEOS which is the big switch to disable all transcoding, but you can configure which video types get transcoded, and at what bitrate, via the following settings:

  • transcodeBitrateQVGA
  • transcodeBitrateUHD
  • doNotTranscodeMimeTypes
  • doNotTranscodeVideoCodecs
  • doNotTranscodeAudioCodecs

I added it! See New in v2.1: exclude files with globs

Done per PM.

but they will still be visible in the library with a thumbnail or not at all?

Oh, I see, I need to check it out. Thank you!

I was running for a while (Thanks @mrm for help with the DB restore!), and PS crashed. Now trying to start it with --versbose flag seeing:

2022-05-09T11:36:38.148Z main-2626615 error Error onError() { event: 'fatal', message: "Can't restart sync, failure rate is too high.¹" }
2022-05-09T11:36:38.149Z main-2626615 error exit() exit() { reason: "Can't restart sync, failure rate is too high.¹",
  status: 12,
  ending: false }

Any way to quickly fix t?

Thanks!

If you want to send DM/email me the prior log lines, that’d be great.

Glob support is in the release I hope to get out tomorrow, BTW, not the current alpha. 1 release.

Emailed.

upgradeInsecureRequests=false

Where to set this up


It’s not working after the update to the alpha version.
Using the node variant

You need to place this into the settings.toml file of your library.

Just as a random mention, the ‘what’s new’ link points to version 1.0

That’s were I placed them. But doesn’t help and I get no connection to server after some time. Nothing interesting if I logcat

2 Likes

Welcome to PhotoStructure, @mexusbg!

The site.webmanifest is served as a static file: you should see it in your photostructure-for-servers/public directory.

It’s odd that the webservice is returning a 401: PhotoStructure’s static file server router returns 404s for unknown routes, so I believe that status is coming from your reverse proxy. What do you have in front of PhotoStructure?

1 Like

I guess I’m blind, but where are the sync reports stored?

They live in .photostructure/sync-reports in your PhotoStructure library.

I will add a proper link and web view for that directory hierarchy to the UI after user auth is done.

BMP support can work with PhotoStructure for Node, but only via GraphicsMagick and recompiling libVips. I’d suggest converting them (with GraphicsMagic) to PNG: that would be a lossless conversion, and should safe disk space. Native support would require pulling in a new library.

Can you email one of these .wmv files to support@photostructure.com so I can take a look? Those should be supported, but as WMV is a container format, there are a ton of different codec combinations, and ffmpeg might not be happy with whatever you’ve got (given the settings I currently feed it).

I don’t know if this is related to the new version or not, but I report this in case anyone else can correlate.

I was investigating a problem where the sync seemed to die, so I was logtailing inside the container. I was seeing lots of permissions errors with the app trying to delete items from both the imgcache and database copy in /ps/tmp.

I checked the permissions on the real filesystem and a bunch of directories were owned by root, instead of the container owner. I’ve been running the previous version for a while with no issue and the locations are the same (same docker-compose.yml with the UID set). I’ve changed the ownership of the whole mount for /ps/tmp back to my container user again and things seem happier.