Extremely slow rebuilding - V1.0.0

I have 89,935 items to go. 24 hours ago, it was 90,963. At this rate, it will take three more months to rebuild. Ideas? Windows 10 desktop version, running on an i3 processor with 8GB RAM.

Thanks.
Earl
Screenshot 2021-08-15 132612

Oh man, that’s no good. Can you set your log level to info, restart, let the sync run for 10 minutes, and then send me logs?

Thanks for sharing those logs. (I deleted them just to keep them private).

System operations on that machine seem to be taking a really long time to complete: like, seconds instead of milliseconds.

Are you using antivirus other than Microsoft Windows Defender?

Yes. I have McAfee installed and running.

For what it’s worth, I’ve found no end of problems on Windows from other anti-malware and anti-virus software (but especially from McAfee and Malwarebytes).

Microsoft’s Windows Defender is free, ranks highly for malware and virus detection, and plays nicely with PhotoStructure.

Can you try switching to Windows Defender temporarily and see if that fixes the slow sync issue? (Don’t just disable McAfee, though: I’d hate for your system to be vulnerable during this test).

I talked more about this here:

It’s been a while since I used McAfee (and I agree, it’s a terrible program), but there should be able to exclude paths and applications from the scans. Might help.

The issue I saw in his logs was system readdir calls were taking 3.5-4.5 seconds (it should be, max, 10ms). I don’t expect that excluding c:\Program Files\PhotoStructure would help with slow system calls (but who knows, maybe they’re monkeypatching system calls?).

I also am hesitant to suggest to people that they exclude PhotoStructure from malware detection. I’m the “IT guy” for my family, and if I read those instructions for some other software anywhere else, I’d find that advice to be super sketchy.

Probably exclude the source directories and library folder. If the OP doesn’t want to switch to defender, might be worth a shot. At a minimum disable mcafee for a short while to troubleshoot / confirm that’s indeed the problem.

I haven’t yet totally disabled or removed McAfee, but I did turn off “real-time scanning” for a period of time, with no real effect. I will switch to Microsoft Defender today and update you on the result. Again, thanks.

McAfee has been uninstalled, but the speed of re-building is not measurably improved following a restart. Next steps in diagnosing?

If you kept the LOG_LEVEL at debug, can you send me logs since you switched to Windows Defender?

(Please DM or email me, just to keep them private)

Sent via DM. McAfee has been completely removed.

I took a look at your logs and found that the rebuild work queue wasn’t working as expected on your machine: sync-file-34384, for example, was being submitted 3 tasks simultaneously.

There’s also, currently, an advisory lock that’s acquired for asset rebuilds, but I just realized that’s unnecessary.

Thanks for sharing your logs, and apologies for the hassle: I’ll try to have a solution for this in the next build.

Good morning! I know you are probably swamped, but I’m wondering when the next build might be scheduled that addresses my slow rebuild issue.

I still have over 36,000 assets to go…

Earl

Apologies for the latency here! I have a hypothesis for what’s going on, and will code up a proof and solution in a day or two as soon as I push out a pre-release of v1.2. @mnaoumov is also setting similar poor performance, so I’m motivated to get this fixed.

1 Like

Hi! Any ETA on V1.2?

Earl

Apologies again for the delay!

I’m hoping to work on this as soon as I ship the first alpha build of v2.0.0.: I’m pretty sure I’ve worked out all the changes I need to make in my head already.