New "easy mode" for Docker coming in v2.1

[geeohgeegeeoh]

That Discord channel reference bounces every time I try to use it. Do you have it in another form?

(btw “soon” is the agonising message I know we all want to hear, but the reality is much harder: it’s ready when it’s ready. I have wanted to ask “when” about 100 times since you cashed my (e)cheque but the reality is, we’re here for the long haul: get it right, is better than ship it now)

[adamf]

Echoing the above, the discord link also doesn’t load anything for me. Is there a general server invite URL?

[mrm]

There’s a discord link in the footer that should work: once you’re in, the link to the discord comment should work …

[adamf]

I’m in! Thanks.

[mrm]

Sorry, I didn’t reply to this comment: I meant to when I first read it.

@geeohgeegeeoh

we’re here for the long haul: get it right, is better than ship it now

I can’t tell you how much I appreciate that.

Also, in case you missed it, I was able to ship the first alpha builds of v2.1 this morning. It’s rough still, but the stable release is nigh!

[madmari]

Are there updated instructions for that alpha easy docker mode? I have a Synology that I could test that on.

[tkohhh]

Here’s how to migrate (please wait for @mrm to confirm that this is correct!)

1. Stop your Photostructure container
2. Move the contents of your mapped config folder into /ps/library/.photostructure/docker-config
3. Move the contents of your mapped logs folder into /ps/library/.photostructure/logs
4. In the container configuration, remove the mapped directories for tmp, config, and logs. You only need the library mapped directory now (and any source photo directories you may want mapped)
5. Restart your Photostructure container

[madmari]

Thanks - but I do not have an existing docker installation. I have PS running on the desktop now and do not want to touch that with an alpha release. Wanted to try docker which is my ultimate goal to move to.
So, I was looking for a guide for an initial install using the easy docker option.

[tkohhh]

Gotcha!

1. Set up a new container using photostructure/server:alpha as the repository
2. Add a mapped directory for the library. The container directory should be ‘/ps/library’. The host directory can be whatever directory you want on your host machine.
3. Start the container

There may be some particulars depending on your host, but this bare-minimum should get you pretty close, if not all the way there!

[mrm]

@madmari : @tkohhh is correct: the result of “docker easy mode” is that instead of 4 required bind mounts, you just give the container a single bind-mount to /ps/library, and PhotoStructure will put the other stuff (like cache and logs) into a subdirectory within that bind mount.

So, in other words, the docker run command becomes just

docker run \
  --name photostructure \
  --detach \
  --restart on-failure \
  --stop-timeout 120 \
  --publish 1787:1787 \
  -e TZ="$TZ" \
  -e UID=$(id -u) \
  -e GID=$(id -g) \
  -v /path/on/your/nas/to/your/photostructure/library:/ps/library \
  -v /path/on/your/nas/to/your/photos:/photos \
  photostructure/server:v2.1.0-alpha.1

Be sure to change /path/on/your/nas/to/your/photostructure/library and /path/on/your/nas/to/your/photos to suit your setup.

Note that there was a docker tagging bug found yesterday, so :alpha isn’t pulling the correct image. After I get that fixed (probably today), please change photostructure/server:v2.1.0-alpha.1 to photostructure/server:alpha.

Once v2.1:stable is released, I will update the docker and docker-compose instructions.

[Daniel]

@mrm

PhotoStructure still needs a local disk!

PhotoStructure still needs to have a cache dir that supports memory mapping, so if you want to store your library on mergefs or a union filesystem, you can bind-mount that volume to /ps/library, but you also need to provide a standard, local (fast!) filesystem bind-mounted to /ps/tmp.

Read more about why on this forum post .

Do you have an estimate as to how large the cache dir needs to be?

Is there a reason that it can’t be an NFSv4 mount?

[tkohhh]

Just to clarify here, when @mrm is referring to a cache directory, he’s talking about the “temp” directory for photostucture, not the Unraid cache disk. If you’re using Unraid in a standard way*, you’re not going to have any problems, even if you don’t have a cache disk (although it might run a little slow if you don’t use a cache disk).

As for the sizing of the library directory, for my collection of 21,000 photos, my library is 12.2GB. That’s about 630KB of library size per photo. Your mileage may vary!

*if you use the mergerfs or unionfs plugin, or if you use Unassigned Devices to attach a network drive, then you’ll have to be careful about using those directories in Photostructure.

edit: sorry, I got my wires crossed and was thinking this was an Unraid question!

[mrm]

@Daniel

Do you have an estimate as to how large the cache dir needs to be?

Great question: I moved my answer to this new topic (so it’d be easier to find later):

Everything You Ever Wanted To Know About PhotoStructure's Cache Directory

What’s the “cache” or “tmp” directory? This is where PhotoStructure can temporarily stores larger files. Some other applications call this a “scratch” directory. Where is this cache directory stored? It’s different on every platform: PhotoStructure tries to be a “good citizen” for every OS and follow the standard for that OS. On macOS, it lives in $HOME/Library/Caches/PhotoStructure. Note that the Library directory is hidden in the Finder: hit ⌘-shift-. (command-shift-period) to see it. …

[adamf]

What is the intent behind having the new subfolders for cache, docker-config, and all the other folders within .photostructure be a hidden folder? If we’re already specifying a location for /ps/library, why would the only subdirectory content (other than the .uuid file) in that location need to default to hidden?

It just adds an extra flag requirement of -a on an ls from my perspective where if I’m looking for something.

Going further, couldn’t all of the content that is already living within .photostructure just be moved to up to live adjacent to the /ps/library/.uuid file within /ps/library - else we’re just adding one layer of directory structure.

If /ps/tmp, /ps/config, or /ps/logs are defined, used those. Else, fall back to (respectively) /ps/library/cache-$UID, /ps/library/docker-config, and /ps/library/logs.

[mrm]

@adamf

intent behind having the new subfolders for cache, docker-config, and all the other folders within .photostructure be a hidden folder

I’m hoping that for most users these folders are totally ignorable “implementation details”.

Are you thinking I could move all the directories in .photostructure up one level and have them hidden?

I’m a bit hesitant to have them not be hidden, just to avoid customer support issues when people delete their database or previews directory because “it’s taking up disk space” and they don’t know the provenance of those files.

(But I agree with you, there may be better defaults than just cramming everything into .photostructure/)

[adamf]

Moreso I was hoping things could be either unhidden, or unhidden and moved up a level. That said, perhaps sensitive things could be moved up a level and remain hidden, but others could be moved up and also unhidden?

Eg: Hide docker config. Don’t hide logs. You’d need to make the call for each directory/folder.

It does feel like it could be a win though, because right now my directory structure (similar to others, I imagine) is ~/docker/photostructure/.photostructure/<content>

A bunch of other services I have running dump stuff directly into the equivalent of /ps/config or /ps/library at the top level, and so are only two levels deep from my parent docker folder.

Edit: I could also bind mount into the .photostructure subfolder directly, but not sure of the consequences within the app if I do that as I’d be pointing both /ps/library and /ps/library/.photostructure to the same location.

[tkohhh]

@mrm

I’m a bit hesitant to have them not be hidden, just to avoid customer support issues when people delete their database or previews directory because “it’s taking up disk space” and they don’t know the provenance of those files.

I suspect the people that would do this are running Photostructure on Windows or Mac, not docker.

The hidden folder doesn’t bother me at all. But it’s absence wouldn’t bother me either. shrug

[mrm]

OK:

for i in logs docker-config ; do 
  mkdir -p ~/docker/photostructure/$i
done

The next build will use those directories if they exist.

Will that work for you?

[adamf]

That was moreso an example based on me running an ls -ahl on the folder and choosing two sample directories. Personally, while troubleshooting or otherwise I’d often jump into the models folder to dig out the database.

I guess what I’m saying is, don’t move only 1 or 2 folders up to check if they exist directly at the root level of /ps/library's bind mount. That would just add a lot of conditionals to the code. Just leave things as they are right now, or if you’re so inclined, move everything up a level but hide things you think are too sensitive to not be hidden.

To @tkohhh’s point though, I doubt the people setting this up on docker-compose are the ones who would be accidentally deleting folder structure to free up disk space. It feels like the equivalent of the Media, Metadata, Plug-ins, Plug-in Support, Codecs, etc. folders within the Plex docker image. None of them are hidden, and there is vibrant community discussion on the forums as to what is safe to delete, if any, when migrating installations of Plex.

(Admittedly this is not the best overall example however, since Plex buries everything 3 folders down in Library/Application Support/Plex Media Server, but they aren’t going to take my suggestions to fix it on the forums anyway.)

[mrm]

@adamf

there is vibrant community discussion on the forums as to what is safe to delete

Hmm, that seems to indicate the stash-everything-away approach might result in less drama/confusion?

@adamf

That would just add a lot of conditionals to the code.

Meh, I think it’s OK?

export function defaultLogDir() {
 if (isDocker()) {
    return firstDirOrFail(
      [
        "/ps/logs",
        "/ps/library/logs", // < added line
        {
          dir: "/ps/library/.photostructure/logs",
          parentRequiredToExist: "/ps/library"
        }
      ],
      "logs"
    )
  }