Installed but not everything is being synced

Hi there, I have installed PS on a VM running on a Docker in Proxmox, everything should be smooth, but for some reason I dont see the syncing progressing. I did have to reinstall at some point, not sure if that made things bad or broke something, but I dont know who to check this.
To add on top of everything I am new to Linux, been using ChatGPT 4 Turbo help to guide me on what to do, but now its lost with the problem I seem to be having.

Adding a screenshot of the import folder I added PS to import, you can see the number of files, in 8 hours there was no progress so something is going on for sure, but I dont know where or how to look.

Thank you for your help!

It is odd that the sync info only lists /ps/library
I am guess you were expecting /photos_import_temp_8tb_ssd to be in list list as well?

What did you set in the settings under “Where else are your photos and videos?”

Hi there, sharing a screenshot:

Ok, just change your scanned folder to /photos_import_temp_8tb_ssd
That’s the mount point from inside the container

also, what’s you container configuration (all the parameters, environment variables and mounts) when you created the docker container? If it was done through a compose file, just paste that in here.

Hi there,

Thank you for the tip of the mountpoint, it would be way more easy to understand if the GUI would display what are the available mount points by exposing them in a dropdown. Otherwise is very confusing. Especially for non Linux experts.

I try to share more details below, thanks for your help!


docker-compose.yml

version: ‘3.3’
services:
photostructure:
image: photostructure/server:latest
ports:
- “1787:1787”
volumes:
- /mnt/nas/photostructure_library:/ps/library
- /mnt/local_ssd_2tb/photostructure/tmp:/ps/tmp
- /home/fernando/photostructure/config:/ps/config
- /home/fernando/photostructure/logs:/ps/logs
- /mnt/photos_import_temp_8tb_ssd:/photos_import_temp_8tb_ssd:ro

environment:
  - UID=1000
  - GID=1000
restart: unless-stopped

root@docker:~/docker-compose-files# docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6a2b884adfb3 photostructure/server:latest “/sbin/tini – /ps/a…” 37 hours ago Up 37 hours (healthy) 0.0.0.0:1787->1787/tcp, :::1787->1787/tcp docker-compose-files-photostructure-1
fc22e360bda9 teslamate/grafana:latest “/run.sh” 11 days ago Up 37 hours 0.0.0.0:3000->3000/tcp, :::3000->3000/tcp teslamate-grafana-1
0226308ec97f eclipse-mosquitto:2 “/docker-entrypoint.…” 11 days ago Up 37 hours 1883/tcp teslamate-mosquitto-1
2062d9c05f91 teslamate/teslamate:latest “tini – /bin/sh /en…” 11 days ago Up 37 hours 0.0.0.0:4000->4000/tcp, :::4000->4000/tcp teslamate-teslamate-1
d4aaec9ab75a postgres:13 “docker-entrypoint.s…” 11 days ago Up 37 hours 5432/tcp teslamate-database-1
5cb61bd47e91 portainer/portainer-ce “/portainer” 11 days ago Up 37 hours 0.0.0.0:8000->8000/tcp, :::8000->8000/tcp, 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp, 9443/tcp portainer

root@docker:~/docker-compose-files# docker inspect 6a2b884adfb3
[
{
“Id”: “6a2b884adfb36b1f8ce3e522a4c7ec8850f4c8b6085f126814508a83fc2f1da6”,
“Created”: “2023-11-23T22:50:48.601463328Z”,
“Path”: “/sbin/tini”,
“Args”: [
“–”,
“/ps/app/docker-entrypoint.sh”
],
“State”: {
“Status”: “running”,
“Running”: true,
“Paused”: false,
“Restarting”: false,
“OOMKilled”: false,
“Dead”: false,
“Pid”: 6986,
“ExitCode”: 0,
“Error”: “”,
“StartedAt”: “2023-11-23T22:50:48.918505994Z”,
“FinishedAt”: “0001-01-01T00:00:00Z”,
“Health”: {
“Status”: “healthy”,
“FailingStreak”: 0,
“Log”: [
{
“Start”: “2023-11-25T13:00:05.036994568+01:00”,
“End”: “2023-11-25T13:00:05.324014511+01:00”,
“ExitCode”: 0,
“Output”: “OK”
},
{
“Start”: “2023-11-25T13:00:35.335772975+01:00”,
“End”: “2023-11-25T13:00:35.380901945+01:00”,
“ExitCode”: 0,
“Output”: “OK”
},
{
“Start”: “2023-11-25T13:01:05.393862683+01:00”,
“End”: “2023-11-25T13:01:05.595547782+01:00”,
“ExitCode”: 0,
“Output”: “OK”
},
{
“Start”: “2023-11-25T13:01:35.61505403+01:00”,
“End”: “2023-11-25T13:01:35.663242665+01:00”,
“ExitCode”: 0,
“Output”: “OK”
},
{
“Start”: “2023-11-25T13:02:05.675136584+01:00”,
“End”: “2023-11-25T13:02:05.865345952+01:00”,
“ExitCode”: 0,
“Output”: “OK”
}
]
}
},
“Image”: “sha256:7f932e83f9d4af93452028e2a8c92bdda84720f31e6db4ccc9ce4478238a103c”,
“ResolvConfPath”: “/var/lib/docker/containers/6a2b884adfb36b1f8ce3e522a4c7ec8850f4c8b6085f126814508a83fc2f1da6/resolv.conf”,
“HostnamePath”: “/var/lib/docker/containers/6a2b884adfb36b1f8ce3e522a4c7ec8850f4c8b6085f126814508a83fc2f1da6/hostname”,
“HostsPath”: “/var/lib/docker/containers/6a2b884adfb36b1f8ce3e522a4c7ec8850f4c8b6085f126814508a83fc2f1da6/hosts”,
“LogPath”: “/var/lib/docker/containers/6a2b884adfb36b1f8ce3e522a4c7ec8850f4c8b6085f126814508a83fc2f1da6/6a2b884adfb36b1f8ce3e522a4c7ec8850f4c8b6085f126814508a83fc2f1da6-json.log”,
“Name”: “/docker-compose-files-photostructure-1”,
“RestartCount”: 0,
“Driver”: “overlay2”,
“Platform”: “linux”,
“MountLabel”: “”,
“ProcessLabel”: “”,
“AppArmorProfile”: “docker-default”,
“ExecIDs”: null,
“HostConfig”: {
“Binds”: [
“/home/fernando/photostructure/config:/ps/config:rw”,
“/home/fernando/photostructure/logs:/ps/logs:rw”,
“/mnt/photos_import_temp_8tb_ssd:/photos_import_temp_8tb_ssd:ro”,
“/mnt/nas/photostructure_library:/ps/library:rw”,
“/mnt/local_ssd_2tb/photostructure/tmp:/ps/tmp:rw”
],
“ContainerIDFile”: “”,
“LogConfig”: {
“Type”: “json-file”,
“Config”: {}
},
“NetworkMode”: “docker-compose-files_default”,
“PortBindings”: {
“1787/tcp”: [
{
“HostIp”: “”,
“HostPort”: “1787”
}
]
},
“RestartPolicy”: {
“Name”: “unless-stopped”,
“MaximumRetryCount”: 0
},
“AutoRemove”: false,
“VolumeDriver”: “”,
“VolumesFrom”: null,
“ConsoleSize”: [
0,
0
],
“CapAdd”: null,
“CapDrop”: null,
“CgroupnsMode”: “private”,
“Dns”: null,
“DnsOptions”: null,
“DnsSearch”: null,
“ExtraHosts”: ,
“GroupAdd”: null,
“IpcMode”: “private”,
“Cgroup”: “”,
“Links”: null,
“OomScoreAdj”: 0,
“PidMode”: “”,
“Privileged”: false,
“PublishAllPorts”: false,
“ReadonlyRootfs”: false,
“SecurityOpt”: null,
“UTSMode”: “”,
“UsernsMode”: “”,
“ShmSize”: 67108864,
“Runtime”: “runc”,
“Isolation”: “”,
“CpuShares”: 0,
“Memory”: 0,
“NanoCpus”: 0,
“CgroupParent”: “”,
“BlkioWeight”: 0,
“BlkioWeightDevice”: null,
“BlkioDeviceReadBps”: null,
“BlkioDeviceWriteBps”: null,
“BlkioDeviceReadIOps”: null,
“BlkioDeviceWriteIOps”: null,
“CpuPeriod”: 0,
“CpuQuota”: 0,
“CpuRealtimePeriod”: 0,
“CpuRealtimeRuntime”: 0,
“CpusetCpus”: “”,
“CpusetMems”: “”,
“Devices”: null,
“DeviceCgroupRules”: null,
“DeviceRequests”: null,
“MemoryReservation”: 0,
“MemorySwap”: 0,
“MemorySwappiness”: null,
“OomKillDisable”: null,
“PidsLimit”: null,
“Ulimits”: null,
“CpuCount”: 0,
“CpuPercent”: 0,
“IOMaximumIOps”: 0,
“IOMaximumBandwidth”: 0,
“MaskedPaths”: [
“/proc/asound”,
“/proc/acpi”,
“/proc/kcore”,
“/proc/keys”,
“/proc/latency_stats”,
“/proc/timer_list”,
“/proc/timer_stats”,
“/proc/sched_debug”,
“/proc/scsi”,
“/sys/firmware”,
“/sys/devices/virtual/powercap”
],
“ReadonlyPaths”: [
“/proc/bus”,
“/proc/fs”,
“/proc/irq”,
“/proc/sys”,
“/proc/sysrq-trigger”
]
},
“GraphDriver”: {
“Data”: {
“LowerDir”: “/var/lib/docker/overlay2/48573d8ceee240943e8569ce0d611915bb8d0efe409aca664270fcfe8867445b-init/diff:/var/lib/docker/overlay2/bec50730c0149ce415f544d3caa6ee9b90cf146895f4480fd89e91aca04ce01e/diff:/var/lib/docker/overlay2/9734947aadada9131f3c096c7284cf08220358985a86e756703e0f3489bfafef/diff:/var/lib/docker/overlay2/a6184d07449f5f7e8f6f0a79b22af87d564626c415d93641e02d6535ae88f912/diff:/var/lib/docker/overlay2/62d51cad4d0a5d0f1786501bad5f231d07df5543481937275e7dbce1508647e2/diff:/var/lib/docker/overlay2/26c5beb54159c682dd786bbdd4558cde9de2e4ccb7b3fc588d6bb6a1d78d0105/diff:/var/lib/docker/overlay2/73b82a0968789b41a4d50e1ba6531c376285328f2eac154476a77b195ed7093c/diff:/var/lib/docker/overlay2/5c1e8d50afcf5ce9ea35ae38862dcccd2a983d6461d0ba268514bdcee966b761/diff:/var/lib/docker/overlay2/871887f90b9cde492eff2bd3796ca703a488f2e5165472026cb57d47887e7eae/diff”,
“MergedDir”: “/var/lib/docker/overlay2/48573d8ceee240943e8569ce0d611915bb8d0efe409aca664270fcfe8867445b/merged”,
“UpperDir”: “/var/lib/docker/overlay2/48573d8ceee240943e8569ce0d611915bb8d0efe409aca664270fcfe8867445b/diff”,
“WorkDir”: “/var/lib/docker/overlay2/48573d8ceee240943e8569ce0d611915bb8d0efe409aca664270fcfe8867445b/work”
},
“Name”: “overlay2”
},
“Mounts”: [
{
“Type”: “bind”,
“Source”: “/mnt/nas/photostructure_library”,
“Destination”: “/ps/library”,
“Mode”: “rw”,
“RW”: true,
“Propagation”: “rprivate”
},
{
“Type”: “bind”,
“Source”: “/mnt/local_ssd_2tb/photostructure/tmp”,
“Destination”: “/ps/tmp”,
“Mode”: “rw”,
“RW”: true,
“Propagation”: “rprivate”
},
{
“Type”: “bind”,
“Source”: “/home/fernando/photostructure/config”,
“Destination”: “/ps/config”,
“Mode”: “rw”,
“RW”: true,
“Propagation”: “rprivate”
},
{
“Type”: “bind”,
“Source”: “/home/fernando/photostructure/logs”,
“Destination”: “/ps/logs”,
“Mode”: “rw”,
“RW”: true,
“Propagation”: “rprivate”
},
{
“Type”: “bind”,
“Source”: “/mnt/photos_import_temp_8tb_ssd”,
“Destination”: “/photos_import_temp_8tb_ssd”,
“Mode”: “ro”,
“RW”: false,
“Propagation”: “rprivate”
}
],
“Config”: {
“Hostname”: “6a2b884adfb3”,
“Domainname”: “”,
“User”: “”,
“AttachStdin”: false,
“AttachStdout”: true,
“AttachStderr”: true,
“ExposedPorts”: {
“1787/tcp”: {}
},
“Tty”: false,
“OpenStdin”: false,
“StdinOnce”: false,
“Env”: [
“GID=1000”,
“UID=1000”,
“PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin”,
“NODE_VERSION=16.16.0”,
“YARN_VERSION=1.22.19”,
“PS_IS_DOCKER=true”,
“NODE_ENV=production”
],
“Cmd”: null,
“Healthcheck”: {
“Test”: [
“CMD-SHELL”,
“wget --quiet --output-document - http://localhost:1787/ping”
]
},
“Image”: “photostructure/server:latest”,
“Volumes”: null,
“WorkingDir”: “/ps/app”,
“Entrypoint”: [
“/sbin/tini”,
“–”,
“/ps/app/docker-entrypoint.sh”
],
“OnBuild”: null,
“Labels”: {
“com.docker.compose.config-hash”: “ec5b2db75980dc298b703d2696fdff8b3f38fa023f41a28f54a7826c63bac1b3”,
“com.docker.compose.container-number”: “1”,
“com.docker.compose.depends_on”: “”,
“com.docker.compose.image”: “sha256:7f932e83f9d4af93452028e2a8c92bdda84720f31e6db4ccc9ce4478238a103c”,
“com.docker.compose.oneoff”: “False”,
“com.docker.compose.project”: “docker-compose-files”,
“com.docker.compose.project.config_files”: “/root/docker-compose-files/docker-compose.yml”,
“com.docker.compose.project.working_dir”: “/root/docker-compose-files”,
“com.docker.compose.service”: “photostructure”,
“com.docker.compose.version”: “2.23.0”
}
},
“NetworkSettings”: {
“Bridge”: “”,
“SandboxID”: “1b231f1c3d741dcd1fc9e95805e15ddf4bd47e432d4a14579e5a2ddab0f77025”,
“HairpinMode”: false,
“LinkLocalIPv6Address”: “”,
“LinkLocalIPv6PrefixLen”: 0,
“Ports”: {
“1787/tcp”: [
{
“HostIp”: “0.0.0.0”,
“HostPort”: “1787”
},
{
“HostIp”: “::”,
“HostPort”: “1787”
}
]
},
“SandboxKey”: “/var/run/docker/netns/1b231f1c3d74”,
“SecondaryIPAddresses”: null,
“SecondaryIPv6Addresses”: null,
“EndpointID”: “”,
“Gateway”: “”,
“GlobalIPv6Address”: “”,
“GlobalIPv6PrefixLen”: 0,
“IPAddress”: “”,
“IPPrefixLen”: 0,
“IPv6Gateway”: “”,
“MacAddress”: “”,
“Networks”: {
“docker-compose-files_default”: {
“IPAMConfig”: null,
“Links”: null,
“Aliases”: [
“docker-compose-files-photostructure-1”,
“photostructure”,
“6a2b884adfb3”
],
“NetworkID”: “e7ede9387c3ef3cda190fe03927ab8b6e2f105b74840cc1e28f62b0516ae08b5”,
“EndpointID”: “e5bc0ac35cbc6b6e6200c0d463da011bcd91fae79bfd5b33eaf45c6745b525eb”,
“Gateway”: “172.19.0.1”,
“IPAddress”: “172.19.0.2”,
“IPPrefixLen”: 16,
“IPv6Gateway”: “”,
“GlobalIPv6Address”: “”,
“GlobalIPv6PrefixLen”: 0,
“MacAddress”: “02:42:ac:13:00:02”,
“DriverOpts”: null
}
}
}
}
]

This all looks ok. So did it work when you changed the mount point?

Honestly, especially on docker you could just use “automatic”. The beauty (and curse) of containers is that it can only see what you bind mounted into it, so it’s not going to find folders on the host you didn’t intend to include in the scan.

Also, since you’re using docker you would be much better off using photostructure/server:alpha rather than photostructure/server:latest - the “latest” tag is in reality more than a year old. (feel free to give @mrm a hard time about this, he’s used to it at this point)

Thanks for your feedback, what is the best practice to update it? I pause syncing, then update to alpha on docker compose ?
Then reload Docker compose, then restart sync?

Some additional questions you can maybe guide me. I unfortunatelly found out that PS does not support editing or tagging pictures yet, something I honestly thought it had because it is adding the tags from a Google Takout :confused:
So how should I edit these to organize pictures? I want to organize stuff once, and then even if I need to move away from PS, I want too be sure that all the time spent re-organizing, tagging is not wasted. What is your suggestion for me to do this? Also to create albums of specific events and such. Exif popular tool doesnt seem to have a GUI, I dont feel comfortable with comand line tools, I like using the GUI for as much as possible. This be it on the Mac or on Windows.

Another question, how should my workflow be once I have everything organized and imported into PS? For example when I use PhotoSync to sync pics from my devices, where should I sync them too? Like to a transfer folder that I should add to PS to monitor so it can import from there? Would PS be able to remove the files from there once they are imported I need to run a manual cleanup?

I want to avoid having to maintain 2 locations with all my originals, due to the fact that I have more than 1 million files between pictures, videos, etc.

Thank you for your tips and suggestions

Fernando

Sorted it out with the help of ChatGPT4 and now I am running on Alpha release. ChatGPT has been my trainer for several days now with all this Linux world😃

I am uploading the history, so you can see how I talk with “it” :smiley:

I am also awaiting the tag editing feature. I know it’s high on the enhancement list. Meanwhile I use digiKam to manage all the metadata (tags, people, GPS locations, etc) and just Photostructure do what it does best, which is being the excellent photo browser/viewer (photostructure is also the best - possibly only - tool that does hierarchical tags well, which is a game changer for me!). Digikam will save all the tags inside the pictures and/or in sidecar (xmp). This is as standard / portable as it gets, and photostructure understands all that perfectly. There are other tools besides digiKam. If you search for “DAM” (Digital Asset Manager) on this forum, you’ll find plenty of discussions on this.

On your other questions, there is no right or wrong answer but this is how I do it:

  • I use hierachical tags heavily to organize photos. I found that method as good, if not better than “albums”. But also know that an album feature is coming.

  • I only have my photos in a single place. I do not let photostructure make a copy of them in the library (I don’t have that setting turned on, eventhough I am a “plus” member) especially since it organizes the photos is a similar structure than I already had them. If you DO let photostrure copy them, that’s fine, but don’t delete your source folders, keep them as a backup basically forever.

  • To your question about new files from your devices, please remember PS does not ever delete files from any paths it scans (in fact, as you did yourself, it’s best to keep those paths “read-only”). I know this use case comes up frequently in the forum and discord, and people have solved it in various ways. Hopefully somone else can respond with suggestions on that.

Thanks for sharing your comments.

Once I have everything organized, I will consider not having the option of copying to the PS library.

But I want to avoid having to maintain 2 sets of large amount of data, while at the same time benefiting from deduplication, etc.

If I start doing everything outside PS, I am starting to believe I got here at a too early stage :confused: (also a Plus user here because I like to support Matt).

  • If I have to edit the tags manually outside until the feature comes.
  • If I have to organize the libraries on my own because there might not be the flexibility to do it in a different way.
  • If I cannot create custom albums.
  • If I cannot share with anyone outside my house.
  • If I cannot let people collaborate on my libraries.

I feel there are too many if´s. Dont get me wrong, I get it, Matt seems to be working alone on this project, and he might not have statted it at all with a comercial point if view at all. As we all read he had a specific need and he might have focused on that first.
It would be awesome if we could help Matt with resources, if more people could help him develop the functionalities everyone is expecting, faster.

Today I came across https://immich.app/ which mimics Google Photos but on your own infra. imagine if Matt could join efforts with them and also with the team in Germany that works on PhotoSync app which is amazing.

I think there is a clear need in this space to make so many things, I just think a lot of people are working on their own, when they could be joining efforts.

I have no clue what PS is doing now, is it scanning the rest of the folders or not? I have a lot of stuff there…but there isnt enough information…

Seems to be syncing because I have the option of “Pausing” but there is no status information about what the app is doing now. I wouldnt want to have to go and start writting terminal commands to check what is going on.

image

immich is open source, photostructure is not, so not likely that the two apps and developers would join force. (Although ironically, immich uses exif-vendored which @mrm wrote and is open source, so he occasionally get bug reports from the competition).

Immich has a lot of potential, and it is evolving a lot faster than PS. The beauty of open source. I am watching it… I install it every once in a while to see how it is progressing, it is still missing one key feature for me, and that’s browsing through hierarchical tags.

As far as the features you’re looking for… Yes, you’re too early. Make sure to vote for the feature you want here

You don’t see the progress bar at the very top of the screen on the main page of photostructure?
Also, you can check on the “about” screen if you asset counts are increasing, and if the sync is running.

There are very detailed logs to show all the picture it found and its progress scanning them. But not through the GUI.

Its really a pity, I really wanted this to work :frowning:

This is what I see on the NAS (copied by PS)

These are the amount of assets and image files reported on the about page…something is very off.

In the .photostructure folder from your screenshot above there should be a sync-reports folder. Take a look at the files in there - they’re just csv files (you can open them either with excel, or any text editor) and see if you seen anything weird that may point you to the problem.