Files in /ps/tmp owned by root

Bug
, , , ,
1

Opening this to add more information to the “file owned by root” curious issue described here:

I have been experimenting with triggering a sync through ./photostructure sync inside the container.

$ sudo docker exec -ti photostructure sh
/ps/app # ./photostructure sync /media/photos/2022/2022\ 07\ 01/*
{"dbReady":true}

As soon as I do this, I see a photostructure command and the exiftool processes owned by root in top:

 823591 root      20   0  640284 388216  46576 S  24.6   4.8   1:06.44 photostructure
 824878 root      29   9   18164  16240   2852 R   8.3   0.2   0:00.25 exiftool
 823344 root      20   0  307916  62008  36896 S   1.0   0.8   0:06.02 node
   1156 root      20   0 2195316  50212  19860 S   0.7   0.6  11:54.05 dockerd
 823636 root      29   9  396504 145200  45076 S   0.7   1.8   1:52.95 photostructure
    102 root      20   0       0      0      0 S   0.3   0.0   0:37.31 kswapd0
 823227 realuser  20   0  347544  83188  44948 S   0.3   1.0   0:04.52 photostructure
 823289 realuser    20   0  370480 115356  46672 S   0.3   1.4   0:12.34 photostructure

I then see files owned by root:

$ ll /media/bulkstorage/photostructure/cache/imgcache/
total 0
drwxr-xr-x 24 realuser realuser 226 Jul 12 09:49 ./
drwxrwxr-x  6 realuser realuser 135 Jun 15 03:49 ../
drwxr-xr-x  3 root       root        16 Jul 12 09:44 1b/
drwxr-xr-x  3 root       root        16 Jul 12 09:44 1u/
drwxr-xr-x  3 root       root        16 Jul 12 09:47 28/
drwxr-xr-x  3 root       root        16 Jul 12 09:44 44/
drwxr-xr-x  3 root       root        16 Jul 12 09:48 4w/
drwxr-xr-x  3 root       root        16 Jul 12 09:46 5e/
drwxr-xr-x  3 root       root        16 Jul 12 09:49 63/
drwxr-xr-x  3 root       root        16 Jul 12 09:47 6h/
drwxr-xr-x  3 root       root        16 Jul 12 09:47 7w/
drwxr-xr-x  3 root       root        16 Jul 12 09:44 8c/
drwxr-xr-x  3 root       root        16 Jul 12 09:48 9m/
drwxr-xr-x  3 root       root        16 Jul 12 09:48 9r/
drwxr-xr-x  3 root       root        16 Jul 12 09:44 cc/
drwxr-xr-x  3 root       root        16 Jul 12 09:48 d3/
drwxr-xr-x  3 root       root        16 Jul 12 09:49 dy/
drwxr-xr-x  3 root       root        16 Jul 12 09:45 e1/
drwxr-xr-x  3 root       root        16 Jul 12 09:49 er/
drwxr-xr-x  3 root       root        16 Jul 12 09:47 es/
drwxr-xr-x  3 root       root        16 Jul 12 09:48 fd/
drwxr-xr-x  3 root       root        16 Jul 12 09:46 g0/
drwxr-xr-x  3 root       root        16 Jul 12 09:46 gj/
drwxr-xr-x  3 root       root        16 Jul 12 09:45 w4/

System information

Version 2.1.0-alpha.3
Edition PhotoStructure for Servers
Plan lite
OS Docker Alpine Linux v3.16 on x64
Node.js 16.15.1
ExifTool 12.41
Video support FFmpeg 5.0.1
HEIF support /usr/bin/heif-convert
2

Thanks for reporting this, @devon!

TL;DR: Please change your docker exec command to run either

docker exec -u node -it photostructure sh

or

docker exec -it photostructure su -p node

These commands are almost equivalent, but the result is the same: you’ll have a terminal running as node, rather than root, whose UID and GID is whatever you’ve configured $PUID and $PGID to be in your container.

Longer background

I’d written the tooling and sync documentation before I’d added support for custom UID/GID–so,
(un)fortunately, that command is operating as (un)expected: docker tells Alpine to spawn a shell, and the default userid is root, so photostructure will be running as root as well.

You need to su to the user within the docker container before you kick off sync.

docker-entrypoint.sh does this su stuff for the initial main daemon, but (I don’t know if I can) prevent root logins from subsequent docker execs. (Docker experts: if there is a way, I’m all ears!)

There’s a bit more going on here, as well: your host computer’s realuser won’t exist within the docker container. Rather than creating a new realuser within the container, the docker-entrypoint.sh changes the existing node user to match the UID of $UID or $PUID. The end result is that you want to run photostructure as node, so replace sh with su -p node (the -p is short for --preserve-environment, and may not be necessary), or tell docker to run the shell as node by using docker exec -u node.

In summary, please use the following for your docker exec command:

sudo docker exec -u node -it photostructure sh

update: I’ll fix the existing documentation, and PhotoStructure v2.1.0-alpha.4 will emit to stderr if there’s a mismatch between $UID, $PUID, $GID, or $PGID and the current effective user id or group id:

$ docker exec -it photostructure sh
# ./photostructure --help

WARNING: current process UID=0 but $PUID=1001.

This may result in file permission issues.
See <https://forum.photostructure.com/t/1597/2> for details.

See also:

2 Likes
3

Thanks! That works a treat.

Would you expect to see a difference between these two? The first doesn’t seem to do much and the second whirs into action:

/ps/app # ./photostructure sync /media/photos/2022/2022\ 07\ 01/

and

/ps/app # ./photostructure sync /media/photos/2022/2022\ 07\ 01/*

4

PhotoStructure’s sync tries to be “clever” and not re-do directories it thinks has already been recently synchronized.

Sometimes that cleverness is too clever by half.

I suspect the first command “no-op”-ed (did nothing) because there was a record of that directory having been recently sync’ed.

The second command ran because sync wasn’t sure it had recently completed those child directories.

You can currently disable this cleverness by adding the --force flag, but that makes a bunch of other bits more thorough than you need–it will drop all prior caches and revisit every file it finds to make sure all is present and accounted for. You really shouldn’t need to use --force normally–it’s more of a workaround.

This issue has hit me too, and the more I think about this situation, the less I can defend the current operation.

If the user is manually running sync and specifies a path, they obviously want to re-scan that directory.

sync shouldn’t require a --force here, it should just re-scan the directory like you asked it to.

This is an easy one-line addition to the start of sync. This will be in the next alpha build.

Here’s the $PUID/$PGID documentation: Add linuxserver-style PUID and PGID support to the PhotoStructure docker image

5

I tried this in 2.1.0-alpha.7 and it worked, thank you!