Disk I/O Error on db.sqlite3 with /ps/tmp on MergeFS

Support
,
1

Just spun up PhotoStructure server and I’m getting the error:
Failed to set up /ps/tmp/local-db/models/db.sqlite3: SqliteError: code SQLITE_IOERR_SHMMAP: disk I/O error¹⁶

Here’s my docker-compose.yml:

version: "3.7"
services:
  photostructure:
    container_name: photostructure
    image: photostructure/server
    restart: on-failure
    stop_grace_period: 2m

    user: 1000:1000 # < If you need a different user id, see below

    volumes:
      # This is where your PhotoStructure Library will be stored.
      # It must be readable, writable, and have sufficient free space.
      # If it is a remote volume, uncomment the PS_FORCE_LOCAL_DB_REPLICA
      # environment line below.

      - type: bind
        source: "/mnt/storage/DockerContainerStorage/photostructure/library" # < CHANGE THIS LINE
        target: /ps/library

      # This must be fast, local disk with many gigabytes free.
      # PhotoStructure will use this directory for file caching
      # and for storing a temporary database replica when your
      # library is on a remote volume.

      - type: bind
        source: "/mnt/storage/DockerContainerStorage/photostructure/temp"
        target: /ps/tmp

      # This directory stores your "system settings"

      - type: bind
        source: "/mnt/storage/DockerContainerStorage/photostructure/config"
        target: /ps/config

      # This directory stores PhotoStructure logfiles.

      - type: bind
        source: "/mnt/storage/DockerContainerStorage/photostructure/logs"
        target: /ps/logs

      # Example additional directories to import into your library.
      # Add as many as you'd like, or remove one or both of these examples.

      # Set the target to /media/... or /mnt/...
      # (the name doesn't matter, as long as it is unique)

      - type: bind
        source: "/mnt/storage/photos/googlephotos05092021" # < CHANGE THIS LINE
        target: /var/googlephotos05092021

    ports:
      - 1787:1787/tcp

    environment:
      # PhotoStructure has _tons_ of settings. See
      # <https://photostructure.com/faq/environment-variables/>

      # This tells PhotoStructure to only log errors, which is the default:
      - "PS_LOG_LEVEL=error"

    labels:
      # See https://containrrr.dev/watchtower/container-selection/
      - "com.centurylinklabs.watchtower.enable=true"

    networks:
      - caddy_net

  watchtower:
    image: containrrr/watchtower
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    # Check for updates every couple hours: (3h * 60m * 60s)
    command: --interval 10800
    environment:
      - "WATCHTOWER_LABEL_ENABLE=true"

networks:
  caddy_net:
    external: true

Running ls -l on /mnt/storage/DockerComposeStorage/photostructure/temp/local-db-models returns:

-rw-r--r-- 1 localuser localuser 16384 May 11 11:21 db.sqlite3
-rw-r--r-- 1 localuser localuser 32768 May 11 11:21 db.sqlite3-shm
-rw-r--r-- 1 localuser localuser     0 May 11 11:21 db.sqlite3-wal

where localuser is 1000:1000

I’ve also run sudo chown -R 1000:1000 on /mnt/storage/DockerContainerStorage/photostructure, but that did not solve the issue

Any advice?

2

Welcome to PhotoStructure @TimidNail !

When I’ve seen this, it’s due to SQLite trying to write to a non-local volume.

PhotoStructure runs SQLite in WAL mode, which requires a file that’s mem-mapped for multi-process coordination.

I suspect /mnt/storage/DockerContainerStorage/photostructure/temp is not a local disk.

If it is a local disk, can you tell me more about your docker host (OS, volume metadata about /mnt/storage/DockerContainerStorage/photostructure, …)?

Also: if you want to run PhotoStructure as a custom user, I’d suggest running the v1.0.0 beta instead of v0.9.1 (v0.9 relies on Docker’s --user arg which is a bit knotty).

  1. Change to image: photostructure/server:beta,
  2. delete the user: line in your docker-compose, and
  3. add a linuxserver-style UID and GID environment values:
    environment:
      - UID=1000
      - GID=1000
3

Thanks for getting back so quickly!

/mnt/storage/ is a MergerFS drive created from 5 local harddrives. Although they’re local, I’m assuming that it being a pooled drive is causing the errors.
If there’s no way to solve this, I wouldn’t mind leaving this on my internal drive since I don’t plan on using remote volumes anyway,

Regarding the G/UID, that sounds perfect. I was initially going to give it custom IDs, but decided not to after reading up on Docker’s requirements hahaha

4

Yup.

You don’t have to store your whole library on a local, non-mergefs volume: just the /ps/tmp directory must be a plain-vanilla local disk.

yuuuup

Holler if PhotoStructure gives you any more attitude.

5

Moved temp to /home/localuser/Documents/PhotoStructure/temp and it’s all good now, thanks!

6

Sorry for bumping an old thred but i have i similar problem but with the library folder on a Union Filesystem “virtual” drive that is consist of three physical local drives. I get SQL lite error and that the drive has worng permissions. After puting the library folder directly on one of the disk Photostructure runs fine. Altso this works it would be better if i could run the library folder on the Union Filesystem volume. Is there any settings i can try to change to get it to work? Running it on an OMV machine.

7

Welcome to PhotoStructure, @Andreas !

Your library directory should be fine to run off a mergefs-type drive, as long as PS_FORCE_LOCAL_DB_REPLICA is true, and /ps/tmp is bind-mounted to a local standard disk.

In other words, /ps/tmp needs to support memory mapping/SHM for SQLite to work properly (which composite filesystems may not support properly). tmpfs is fine, too (and will be the fastest option).

Holler if you have any other issues!