1.0.0-beta.9 is ready

mrm · June 18, 2021, 5:57am

What’s in v1.0.0-beta.9?

/ Fixed a race condition in child process handling affecting videos and raw images that could cause these files to not be imported.

…and everything in v1.0.0-beta.8:

Added new disableAllFilters setting, that forces all filter settings to their most permissive value.
Added new ffmpegHwaccel setting enabling FFmpeg hardware-accelerated transcoding.
Transcoded videos now support high gamut output.
Added several new date parsers, new extraDateTimeFormats defaults, and CreationTime to capturedAtTags.

Fixed paging for tags with large numbers of the exact same captured-at time
/ Updated the is-this-file-in-sync test to handle remote filesystem timestamp skew. This should also help speed up re-syncs of existing volumes.
Full-text search indexes and asset tag counts are now only rebuilt when changed. This should remove the high I/O issue and help with slow rebuilds.
“Send recent logs” was removed from the nav menu (context was frequently insufficient, and new versions of sentry broke the prior implementation)
Upgraded all dependencies, including electron 13.

Installation instructions and other details

Setup instructions are the same as prior beta releases. See:

https://forum.photostructure.com/t/version-1-0-0-beta-3-reporting-for-duty/615

bdillahu · June 18, 2021, 1:19pm

Just as some initial feedback… I had needed to do a library rebuild and started with beta.07… stopped when there seemed to be issues there and waited for this.

It’s been running for an hour or so and done as much as beta.07 did in a day - so it seems MUCH faster.

And it’s keeping my CPU loaded (about 80%) - previous versions never kept things “busy”.

Hopefully it’s actually working

avdp · June 18, 2021, 1:26pm

Same here… full library rebuilt completed in just a few hours. Previously this was a multi-day thing.

Philip_Nickels · June 18, 2021, 4:21pm

I did the update and now it wont launch at all. Anyone else have this issue? Tried running as admin, running from directory, reboots, etc. nothing works.

what did seem to resolve the issue is going into appdata/local/photostructure_updater and getting the exe for the latest beta and copying to desktop. Then uninstalling photostructure and reinstalling it using the new exe from the desktop/ . This sucessfully let me use the new version.

Anyways this is a real PIA any advice on how to avoid this jank method going forward.

mrm · June 18, 2021, 5:02pm

That’s no good! What OS are you on? (it seems that you’re on Windows: what build?)

What version of PhotoStructure are you using? (PhotoStructure for Desktop, Docker, or Node?)

I’d recommend not running (any software, really) as an admin. PhotoStructure should absolutely run as a non-admin user.

Agreed, upgrades should be seamless!

In the future, if you have issues, please set your logging to info or debug, relaunch, and then send me logs so I can see what’s going on. https://photostructure.com/faq/error-reports/#how-to-manually-send-your-logs

mrm · June 18, 2021, 5:10pm

I’m seeing this too, and will have a fix for it shortly.

Philip_Nickels · June 18, 2021, 5:24pm

Windows 10 Pro for Workstation build 2004

Using Photstructure for Desktop

I don’t run the app as admin just ran the the installer as admin as part of troubleshooting, once it is installed and working i DO NOT have to run as admin.

If i remember right the same issue happened when i updated to build 7 as well, but so much going on i cant remember very well.

I see you mentioned a fix incoming soon so once the next beta comes out ill set to debug and send you whatever you need that could be useful!

mrm · June 18, 2021, 5:29pm

By “admin” do you mean agreeing to the UAC prompt?

Or do you have to right-click the installer and run-as-administrator?

If that’s the case, we have to play the “one of these is not like the other” game between my windows test boxes and your windows box. (They’re set up as “vanilla” as possible with no third-party software installed other than Firefox, Chrome, and possibly Git for Windows).

bdillahu · June 18, 2021, 5:38pm

Hmmm… I kind of assumed that it was a good sign, as it was keeping CPU loaded instead of waiting.

But whatever

I do note that after the first 20000 items or so it seems to have dropped CPU load down a lot. Not sure if it has slowed down or not, hard to judge.

Philip_Nickels · June 18, 2021, 5:48pm

So yes i do click the UAC prompt as part of the install process. When i mentioned running as admin i meant right clicking and selecting run as admin, but again doing that MAY NOT be required, it is just a habit i have as part of troubleshooting installs on software as ive seen some software require install as admin via rightclick menu or they don’t work. (not unique to just this machine i have seen it on many others too)

Again probably not necessary for this scenario as the fix seems to have been uninstall and reinstall with the update exe from the appdata folder by double clicking it and accepting uac prompt. The cumbersome part is only around the fact that each update id have to uninstall and get that update exe vs just restarting which should be smooth process.

mrm · June 18, 2021, 5:49pm

So, at least on my test machine, the busy process is sync.

There are two things banging around in sync that kept it busy:

I had a permission issue in my cache dir (some directories were owned by root). I’ll make the cache dir cleanup function give up on permission issues (currently it retries fruitlessly).
It looks like main is poking sync for health checks continually. I’m fixing that now.

If tail your logs and it doesn’t seem like that’s what’s keeping your system busy, please send me your logs and I can take a look!

mrm · June 18, 2021, 5:57pm

Ah: if you could uninstall PhotoStructure, and then re-install the beta as the (non-admin) user, I suspect the automatic upgrade (for beta.10, which I’m working on now) may work.

I suspect that by installing as admin it makes the installed file ACLs to be owned by admin, and the windows upgrader expects the same user that installed the app will be doing the upgrade. (fwiw, I’ve found that successful in-place upgrades on Windows require all the stars to be aligned and billg to be in a good mood.)

Philip_Nickels · June 18, 2021, 6:12pm

awesome i will try your suggestion in beta 10 and see if that works then Thanks for the Help!

tkohhh · June 19, 2021, 5:49pm

Just want to report that I updated from beta 4 to beta 9 and everything looks good!

adept · July 3, 2021, 12:08am

Beta 9 in docker. Is it expected that after it finished syncing the library, the “PhotoStructure sync” process will still be running, consuming 1-3% CPU, and seemingly doing little besides reading from /proc/self/stat in a tight loop?

This is how it looks in strace:

[pid 18721] open("/proc/self/stat", O_RDONLY|O_LARGEFILE) = 26
[pid 18721] read(26, "48 (PhotoStructure ) R 27 20 20 "..., 1023) = 386
[pid 18721] close(26)                   = 0
[pid 18721] open("/proc/self/stat", O_RDONLY|O_LARGEFILE) = 26
[pid 18721] read(26, "48 (PhotoStructure ) R 27 20 20 "..., 1023) = 386
[pid 18721] close(26)                   = 0
[pid 18721] open("/proc/self/stat", O_RDONLY|O_LARGEFILE) = 26
[pid 18721] read(26, "48 (PhotoStructure ) R 27 20 20 "..., 1023) = 386
[pid 18721] close(26)                   = 0
[pid 18721] open("/proc/self/stat", O_RDONLY|O_LARGEFILE) = 26
[pid 18721] read(26, "48 (PhotoStructure ) R 27 20 20 "..., 1023) = 386
[pid 18721] close(26)                   = 0
[pid 18721] epoll_pwait(13, [], 1024, 0, NULL, 8) = 0
[pid 18721] epoll_pwait(13, [], 1024, 1, NULL, 8) = 0
[pid 18721] open("/proc/self/stat", O_RDONLY|O_LARGEFILE) = 26
[pid 18721] read(26, "48 (PhotoStructure ) R 27 20 20 "..., 1023) = 386
[pid 18721] close(26)                   = 0
[pid 18721] open("/proc/self/stat", O_RDONLY|O_LARGEFILE) = 26
[pid 18721] read(26, "48 (PhotoStructure ) R 27 20 20 "..., 1023) = 386
[pid 18721] close(26)                   = 0
[pid 18721] open("/proc/self/stat", O_RDONLY|O_LARGEFILE) = 26
[pid 18721] read(26, "48 (PhotoStructure ) R 27 20 20 "..., 1023) = 386
[pid 18721] close(26)                   = 0
[pid 18721] open("/proc/self/stat", O_RDONLY|O_LARGEFILE) = 26
[pid 18721] read(26, "48 (PhotoStructure ) R 27 20 20 "..., 1023) = 386

Stats over about 10 seconds of realime:

# strace -f -p 18721 -c -e open,close,read
strace: Process 18721 attached with 11 threads
strace: Process 4741 attached
^Cstrace: Process 18721 detached
strace: Process 18722 detached
strace: Process 18723 detached
strace: Process 18724 detached
strace: Process 18725 detached
strace: Process 18726 detached
strace: Process 18727 detached
strace: Process 18733 detached
strace: Process 18734 detached
strace: Process 18735 detached
strace: Process 18736 detached
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 55.02    0.000806           0      4835         5 open
 29.42    0.000431           0      4871           read
 15.56    0.000228           0      4846           close
------ ----------- ----------- --------- --------- ----------------
100.00    0.001465                 14552         5 total

So it seems to be doing this about 400-500 times per second

mrm · July 3, 2021, 12:49am

Yikes, thanks for reporting! What’s your docker host OS?

Update: I see this on my docker instance, I’m poking at it now.

Ah crap: beta.9 didn’t skip over /proc mount points. This should be fixed in beta.10.

adept · July 3, 2021, 10:09am

Host OS is linux. Beta.10 will be the next one out, right? I will give it a try when it is available