even if I do a resync via the page menu some pictures do not get imported.
In some folders 1 image is missing in others 5-10 etc. Why so?
If I do it manually by using the terminal and us the “photostructure sync -v” command and specify it imports the missing images according to the log. but when I restart the PS the images are still not there displayed. (even stranger, for one folder it worked that way and the missing images were displaying after restarting PS)
I haven’t seen SQLite "revert"or ignore changes: that’s no good.
For the missing file, try running just ./photostructure sync-file --debug /path/to/file.jpg
.
Feel free to send me those logs over email if you can’t decipher what’s going on.
hmm nope, it did not import the image…
2021-09-25T15:53:22.767Z sync-file-572 info SettingsIO.importFileSettings(/ps/config/settings.toml) loaded { imported:
{ copyAssetsToLibrary: false,
libraryDir: '/ps/library',
scanAllDrives: false,
scanPaths: [ '/photos' ] } }
2021-09-25T15:53:22.789Z sync-file-572 info SettingsIO _libraryHasSettings { result: true,
settings: '/ps/library',
librarySettingsFile: '/ps/library/.photostructure/settings.toml' }
2021-09-25T15:53:22.809Z sync-file-572 info SettingsIO.importFileSettings(/ps/library/.photostructure/settings.toml) loaded { imported: { reportErrors: true } }
2021-09-25T15:53:22.817Z sync-file-572 info Sentry Sentry initialized { isElectron: false }
2021-09-25T15:53:22.822Z sync-file-572 info Service(sync-file) setup() { version: '1.1.0',
start: 1632585201793,
argv:
[ '/usr/local/bin/node',
'/ps/app/bin/sync-file.js',
'--debug',
'/photo/2021/Micha',
'1.',
'Kindergarten/bild',
'01.JPG' ],
arch: 'x64',
platform: 'linux',
isDocker: true,
os: 'Alpine Linux v3.13 on x64',
isPacked: true,
isElectron: false,
versions:
{ node: '16.2.0',
v8: '9.0.257.25-node.16',
uv: '1.41.0',
zlib: '1.2.11',
brotli: '1.0.9',
ares: '1.17.1',
modules: '93',
nghttp2: '1.42.0',
napi: '8',
llhttp: '6.0.1',
openssl: '1.1.1k+quic',
cldr: '39.0',
icu: '69.1',
tz: '2021a',
unicode: '13.0',
ngtcp2: '0.1.0-DEV',
nghttp3: '0.1.0-DEV' },
settings: { logLevel: 'info', httpPort: 1787, rpcPort: 1807, libraryDir: '/ps/library' },
NODE_ENV: 'production',
PS_FORCE_LOCAL_DB_REPLICA: 'false',
PS_LOG_LEVEL: 'info' }
2021-09-25T15:53:22.826Z sync-file-572 info Library(/ps/library) new()
2021-09-25T15:53:22.937Z sync-file-572 info currentLibraryLockOwner(/ps/library/.photostructure) currentLibraryLockOwner { result:
{ file: { nativePath: '/ps/library/.photostructure/opened-by/9sevj98-25.json' },
serviceName: 'main',
createdAt: 1632585166683,
updatedAt: 1632585166684,
hostname: '429c8e352d',
systemUID: '8n4u-mqpd-q6v0-116j',
v: '1.1.0',
pid: 25 } }
2021-09-25T15:53:23.005Z sync-file-572 info SettingsIO.importFileSettings(/ps/library/.photostructure/settings.toml) loaded { imported: { reportErrors: true } }
2021-09-25T15:53:23.020Z sync-file-572 info StatsDbDir Set up statsDbDir dir /ps/tmp/sync-state-8u3ak3a014
2021-09-25T15:53:23.034Z sync-file-572 info Vacuum checkRemoteVacuuming() received RPC value { value: false }
2021-09-25T15:53:23.043Z sync-file-572 info ModelDbJanitor(/ps/library/.photostructure) Finished setup { primaryDbDir: { nativePath: '/ps/library/.photostructure' },
backupDir: { nativePath: '/ps/library/.photostructure/models/backup' } }
2021-09-25T15:53:23.044Z sync-file-572 info ModelDbJanitor(/ps/library/.photostructure) ModelDb reading from /ps/library/.photostructure/models/db.sqlite3
2021-09-25T15:53:23.045Z sync-file-572 info Library(/ps/library) Library.setup() finished.
2021-09-25T15:53:23.052Z sync-file-572 warn SyncFileService copyAssetsToLibrary changed to false
2021-09-25T15:53:23.077Z sync-file-572 info Db(models) setting up new db connection to /ps/library/.photostructure/models/db.sqlite3 { priorWasNull: true }
2021-09-25T15:53:23.080Z sync-file-572 info mkdb() Dynamically setting dbCacheSize to 192 { db: '/ps/library/.photostructure/models/db.sqlite3', dbFileSize: 42024960 }
{"path":"/photo/2021/Micha","rejected":["File was deleted"],"elapsedMs":569}
{"ready":true}
{"path":"/ps/app/1.","rejected":["File was deleted"],"elapsedMs":20}
{"ready":true}
{"path":"/ps/app/Kindergarten/bild","rejected":["File was deleted"],"elapsedMs":2}
{"ready":true}
{"path":"/ps/app/01.JPG","rejected":["File was deleted"],"elapsedMs":2}
{"ready":true}
2021-09-25T15:53:23.644Z sync-file-572 info Service(sync-file) exit() { status: 0, reason: 'Processed files from argv', waitForJobs: true, ending: false }
2021-09-25T15:53:23.898Z sync-file-572 info Service(sync-file) exit(): waiting for enqueued jobs to complete... { pendingNames: [] }
2021-09-25T15:53:23.899Z sync-file-572 info Service(sync-file) exit(): enqueued jobs finished.
2021-09-25T15:53:23.901Z sync-file-572 info Endable endEndables() { isTest: false, isSingleSpecTests: false }
2021-09-25T15:53:23.906Z sync-file-572 info BaseFileCache stats { size: 3, cacheMisses: 3 }
2021-09-25T15:53:23.907Z sync-file-572 info PosixFileCache stats { size: 40, cacheHitsSync: 72, cacheMisses: 40 }
2021-09-25T15:53:23.910Z sync-file-572 info readdirCache stats { cacheHitsSync: 1, cacheMisses: 1 }
2021-09-25T15:53:23.912Z sync-file-572 info rpc.Client(db@localhost:1807#0) end() { ending: true, timings: {}, pendingSize: 0 }
2021-09-25T15:53:23.912Z sync-file-572 info rpc.Client(db@localhost:1807#0) close() { ended: true }
2021-09-25T15:53:23.964Z sync-file-572 info OpenedByIO() debug 'onEnd(): unlinking /ps/library/.photostructure/opened-by/9sevj98-572.json'
2021-09-25T15:53:23.966Z sync-file-572 info Library(/ps/library) onEnd(): finished.
2021-09-25T15:53:23.966Z sync-file-572 info Db(models) closing db Database {
name: '/ps/library/.photostructure/models/db.sqlite3',
open: true,
inTransaction: false,
readonly: false,
memory: false }
2021-09-25T15:53:23.970Z sync-file-572 info Service(sync-file) timings: { 'AssetFileFinder.byUri': { sumSec: 0.563, mean: 560, k: 1 },
'Service.startup()': { sumSec: 0.378, mean: 380, k: 1 },
'fs.stat': { sumSec: 0.017, mean: 17, k: 1 } }
2021-09-25T15:53:23.973Z sync-file-572 info Service(sync-file) memory use on exit { ok: [ 'Memory by sync-file (25 MB) is OK' ], warn: [], bad: [], fail: [] }
Ah, the filename has spaces in it. Either escape the spaces with backslash, or surround the filename with double quotes:
"/photo/2021/Micha 1 Kindergarten/bild 01.JPG"
oops sorry did miss that so here is the log output:
2021-09-25T17:25:58.232Z sync-file-19140 info SettingsIO.importFileSettings(/ps/config/settings.toml) loaded { imported:
{ copyAssetsToLibrary: false,
libraryDir: '/ps/library',
scanAllDrives: false,
scanPaths: [ '/photos' ] } }
2021-09-25T17:25:58.244Z sync-file-19140 info SettingsIO _libraryHasSettings { result: true,
settings: '/ps/library',
librarySettingsFile: '/ps/library/.photostructure/settings.toml' }
2021-09-25T17:25:58.253Z sync-file-19140 info SettingsIO.importFileSettings(/ps/library/.photostructure/settings.toml) loaded { imported: { reportErrors: true } }
2021-09-25T17:25:58.260Z sync-file-19140 info Sentry Sentry initialized { isElectron: false }
2021-09-25T17:25:58.263Z sync-file-19140 info Service(sync-file) setup() { version: '1.1.0',
start: 1632590757633,
argv:
[ '/usr/local/bin/node',
'/ps/app/bin/sync-file.js',
'--debug',
'/photos/2021/Micha 1. Kindergarten/bild 02.JPG' ],
arch: 'x64',
platform: 'linux',
isDocker: true,
os: 'Alpine Linux v3.13 on x64',
isPacked: true,
isElectron: false,
versions:
{ node: '16.2.0',
v8: '9.0.257.25-node.16',
uv: '1.41.0',
zlib: '1.2.11',
brotli: '1.0.9',
ares: '1.17.1',
modules: '93',
nghttp2: '1.42.0',
napi: '8',
llhttp: '6.0.1',
openssl: '1.1.1k+quic',
cldr: '39.0',
icu: '69.1',
tz: '2021a',
unicode: '13.0',
ngtcp2: '0.1.0-DEV',
nghttp3: '0.1.0-DEV' },
settings: { logLevel: 'info', httpPort: 1787, rpcPort: 1807, libraryDir: '/ps/library' },
NODE_ENV: 'production',
PS_FORCE_LOCAL_DB_REPLICA: 'false',
PS_LOG_LEVEL: 'info' }
2021-09-25T17:25:58.266Z sync-file-19140 info Library(/ps/library) new()
2021-09-25T17:25:58.336Z sync-file-19140 info currentLibraryLockOwner(/ps/library/.photostructure) currentLibraryLockOwner { result:
{ file: { nativePath: '/ps/library/.photostructure/opened-by/9sevj98-25.json' },
serviceName: 'main',
createdAt: 1632585166683,
updatedAt: 1632588046682,
hostname: '429c8e352d',
systemUID: '8n4u-mqpd-q6v0-116j',
v: '1.1.0',
pid: 25 } }
2021-09-25T17:25:58.354Z sync-file-19140 info SettingsIO.importFileSettings(/ps/library/.photostructure/settings.toml) loaded { imported: { reportErrors: true } }
2021-09-25T17:25:58.365Z sync-file-19140 info StatsDbDir Set up statsDbDir dir /ps/tmp/sync-state-8u3ak3a014
2021-09-25T17:25:58.373Z sync-file-19140 info Vacuum checkRemoteVacuuming() received RPC value { value: false }
2021-09-25T17:25:58.379Z sync-file-19140 info ModelDbJanitor(/ps/library/.photostructure) Finished setup { primaryDbDir: { nativePath: '/ps/library/.photostructure' },
backupDir: { nativePath: '/ps/library/.photostructure/models/backup' } }
2021-09-25T17:25:58.379Z sync-file-19140 info ModelDbJanitor(/ps/library/.photostructure) ModelDb reading from /ps/library/.photostructure/models/db.sqlite3
2021-09-25T17:25:58.380Z sync-file-19140 info Library(/ps/library) Library.setup() finished.
2021-09-25T17:25:58.384Z sync-file-19140 warn SyncFileService copyAssetsToLibrary changed to false
2021-09-25T17:25:58.477Z sync-file-19140 info ExifTool Started child process ExifTool:19154
2021-09-25T17:25:58.486Z sync-file-19140 info Renice Renice pid 19154 to BelowNormal
2021-09-25T17:25:59.018Z sync-file-19140 info Db(models) setting up new db connection to /ps/library/.photostructure/models/db.sqlite3 { priorWasNull: true }
2021-09-25T17:25:59.020Z sync-file-19140 info mkdb() Dynamically setting dbCacheSize to 192 { db: '/ps/library/.photostructure/models/db.sqlite3', dbFileSize: 42123264 }
2021-09-25T17:25:59.835Z sync-file-19140 info AssetFileImporter(/photos/2021/Micha 1. Kindergarten/bild 02.JPG) Setting timeout to 1 minute
2021-09-25T17:26:01.292Z sync-file-19140 info AssetFile(undefined).updateFromFile() after (1434ms) { '$ctor': 'models.AssetFile',
aperture: 6.3,
cameraId: 'InternalSerialNumber:000014000000',
capturedAtLocal: 2021082309520500,
capturedAtOffset: 0,
capturedAtPrecisionMs: 1000,
capturedAtSrc: 'tags:DateTimeCreated',
fileSize: 9756669,
focalLength: '105.0 mm',
height: 7952,
imageId: 'ShutterCount:5428',
iso: 2500,
lensId:
'LensID:Sony E 18-200mm F3.5-6.3 OSS LE or Tamron 18-200mm F3.5-6.3 Di III VC or Sony E 18-200mm F3.5-6.3 OSS or Sony E PZ 18-200mm F3.5-6.3 OSS',
make: 'Sony',
meanHash: 'Pyh4fFx8NADgGBg4PHz8/sfv/5eLzwcC',
mimetype: 'image/jpeg',
mode0: 658,
mode1: 689,
mode2: 219,
mode3: 693,
mode4: 661,
mode5: 700,
mode6: 2555,
model: 'α7R II',
mountpoint: '/photos',
mtime: 1631181405833,
rotation: 0,
sha: 'C1sMi1ZANfhM9YOvkrACnolhzM9uFk1V',
shutterSpeed: '1/2000',
uri: 'psfile://2EGUfSDRD/2021/Micha%201.%20Kindergarten/bild%2002.JPG',
version: 11,
width: 5304 }
2021-09-25T17:26:01.301Z sync-file-19140 info AssetFileFinder(2021/Micha 1. Kindergarten/bild 02.JPG) Found sibling AssetFile { '$ctor': 'models.AssetFile',
aperture: 6.3,
assetId: 39035,
cameraId: 'InternalSerialNumber:000014000000',
capturedAtLocal: 2021082309520500,
capturedAtOffset: 0,
capturedAtPrecisionMs: 1000,
capturedAtSrc: 'tags:DateTimeCreated',
createdAt: 1631751358717,
fileSize: 9783775,
focalLength: '105.0 mm',
height: 7952,
id: 39035,
imageId: 'ShutterCount:5428',
iso: 2500,
lensId:
'LensID:Sony E 18-200mm F3.5-6.3 OSS LE or Tamron 18-200mm F3.5-6.3 Di III VC or Sony E 18-200mm F3.5-6.3 OSS or Sony E PZ 18-200mm F3.5-6.3 OSS',
make: 'Sony',
meanHash: 'Pzh4fHx4cADgGBg4PHz8/sf/f5eby4IA',
mimetype: 'image/jpeg',
mode0: 658,
mode1: 689,
mode2: 219,
mode3: 693,
mode4: 661,
mode5: 660,
mode6: 2555,
model: 'α7R II',
mountpoint: '/photos',
mtime: 1631182307567,
rotation: 0,
sha: 'oWtrbDNsgInySMsniL2ZiVmCzCO3MvcV',
shown: true,
shutterSpeed: '1/2000',
updateCount: 6,
updatedAt: 1632589476705,
uri: 'psfile://2EGUfSDRD/2021/Micha%201.%20Kindergarten/DSC09697.JPG',
version: 11,
width: 5304 }
2021-09-25T17:26:01.345Z sync-file-19140 info AssetFileFinder(2021/Micha 1. Kindergarten/bild 02.JPG) apply(): byCapturedAtOrImageHash { assetId: 39035 }
2021-09-25T17:26:01.346Z sync-file-19140 info AssetFileFinder(2021/Micha 1. Kindergarten/bild 02.JPG) Found existing asset for file
2021-09-25T17:26:01.346Z sync-file-19140 info AssetFileRepository importAssetFile(psfile://2EGUfSDRD/2021/Micha%201.%20Kindergarten/bild%2002.JPG): importFile was a no-op
2021-09-25T17:26:01.351Z sync-file-19140 info AdvisoryLock withLocks() { lockNames: [ 'assetId:39035' ], acquired: true }
2021-09-25T17:26:01.351Z sync-file-19140 info UpdateAsset(39035) starting
2021-09-25T17:26:01.365Z sync-file-19140 info AssetPreviewBuilder(39035) build_(): asset file candidates: [ 'psfile://2EGUfSDRD/2021/Micha%201.%20Kindergarten/DSC09697.JPG' ]
2021-09-25T17:26:01.435Z sync-file-19140 info AssetPreviewBuilder(39035) _build(/photos/2021/Micha 1. Kindergarten/DSC09697.JPG) { uri: 'psfile://2EGUfSDRD/2021/Micha%201.%20Kindergarten/DSC09697.JPG' }
2021-09-25T17:26:01.439Z sync-file-19140 info UpdateAsset(39035) previews built { assetPreviewInfo:
{ assetFileId: 39035,
assetId: 39035,
uri: 'psfile://2EGUfSDRD/2021/Micha%201.%20Kindergarten/DSC09697.JPG',
width: 5304,
height: 7952,
mimetype: 'image/jpeg',
sha: 'oWtrbDNsgInySMsniL2ZiVmCzCO3MvcV',
rotation: 0,
path: '/photos/2021/Micha 1. Kindergarten/DSC09697.JPG',
mtime: 1631182307567,
filesize: 9783775,
fitSizes: 'uhd4k,fhd,wvga,qvga' } }
2021-09-25T17:26:01.440Z sync-file-19140 info AssetFile setShown() { assetId: 39035, assetFileId: 39035 }
2021-09-25T17:26:01.568Z sync-file-19140 info WhoTagger whoTagFiles() { result: [ [ 'Who', 'Micha Kräuchi' ] ],
names: [ 'Micha Kräuchi', 'Micha Kräuchi' ],
files: [ '/photos/2021/Micha 1. Kindergarten/DSC09697.JPG' ] }
2021-09-25T17:26:01.578Z sync-file-19140 info KeywordTagger processKeywords() { result:
[ [ 'Orte', 'Schweiz', 'Zürich' ],
[ { name: 'Who', ordinal: 4 }, 'Kräuchi', 'Micha Kräuchi' ] ] }
2021-09-25T17:26:01.580Z sync-file-19140 info displayNameForVolsha(2EGUfSDRD) find() { vol:
{ filesystem: 'shfs',
mountpoint: '/photos',
size: 8999628992512,
used: 5546273193984,
available: 3451908231168,
ignorable: false,
remote: false,
uuid: 'b2f092f3-454a-4391-e10e-0162c8fb577f' } }
2021-09-25T17:26:01.580Z sync-file-19140 info FilePathTagger tagAssetPaths() { uri: 'psfile://2EGUfSDRD/2021/Micha%201.%20Kindergarten/DSC09697.JPG',
tp:
[ 'fs',
{ name: '2EGUfSDRD', displayName: 'photos' },
'2021',
'Micha 1. Kindergarten' ] }
2021-09-25T17:26:01.582Z sync-file-19140 info AssetTagger tagAsset(/photos/2021/Micha 1. Kindergarten/DSC09697.JPG) { priorTagPaths:
[ [ 'When', '2021', '8' ],
[ 'Camera', 'Sony', 'α7R II' ],
[ 'Lens',
'Sony',
'E 18-200mm F3.5-6.3 OSS LE or Tamron 18-200mm F3.5-6.3 Di III VC or Sony E 18-200mm F3.5-6.3 OSS or Sony E PZ 18-200mm F3.5-6.3 OSS' ],
[ 'Orte', 'Schweiz', 'Zürich' ],
[ 'Type', 'Image', 'JPEG' ],
[ 'Who', 'Kräuchi', 'Micha Kräuchi' ],
[ 'Who', 'Micha Kräuchi' ],
[ 'fs', '2EGUfSDRD', '2021', 'Micha 1. Kindergarten' ] ],
after:
[ [ 'Camera', 'Sony', 'α7R II' ],
[ 'Lens',
'Sony',
'E 18-200mm F3.5-6.3 OSS LE or Tamron 18-200mm F3.5-6.3 Di III VC or Sony E 18-200mm F3.5-6.3 OSS or Sony E PZ 18-200mm F3.5-6.3 OSS' ],
[ 'When',
{ name: '2021', ordinal: 7979 },
{ name: '8', displayName: 'Aug', ordinal: 5 } ],
[ { name: 'Who', ordinal: 4 }, 'Micha Kräuchi' ],
[ 'Orte', 'Schweiz', 'Zürich' ],
[ { name: 'Who', ordinal: 4 }, 'Kräuchi', 'Micha Kräuchi' ],
[ 'Type', 'Image', 'JPEG' ],
[ 'fs',
{ name: '2EGUfSDRD', displayName: 'photos' },
'2021',
'Micha 1. Kindergarten' ] ],
result: { add: [], remove: [] } }
2021-09-25T17:26:01.584Z sync-file-19140 info AssetTagger tagAndUpsertAsset { add: [], remove: [] }
2021-09-25T17:26:01.618Z sync-file-19140 error AssetFileImporter(/photos/2021/Micha 1. Kindergarten/bild 02.JPG) _apply(): unexpected missing asset file id { path: '/photos/2021/Micha 1. Kindergarten/bild 02.JPG',
af:
{ '$ctor': 'models.AssetFile',
aperture: 6.3,
assetId: 39035,
cameraId: 'InternalSerialNumber:000014000000',
capturedAtLocal: 2021082309520500,
capturedAtOffset: 0,
capturedAtPrecisionMs: 1000,
capturedAtSrc: 'tags:DateTimeCreated',
fileSize: 9756669,
focalLength: '105.0 mm',
height: 7952,
imageId: 'ShutterCount:5428',
iso: 2500,
lensId:
'LensID:Sony E 18-200mm F3.5-6.3 OSS LE or Tamron 18-200mm F3.5-6.3 Di III VC or Sony E 18-200mm F3.5-6.3 OSS or Sony E PZ 18-200mm F3.5-6.3 OSS',
make: 'Sony',
meanHash: 'Pyh4fFx8NADgGBg4PHz8/sfv/5eLzwcC',
mimetype: 'image/jpeg',
mode0: 658,
mode1: 689,
mode2: 219,
mode3: 693,
mode4: 661,
mode5: 700,
mode6: 2555,
model: 'α7R II',
mountpoint: '/photos',
mtime: 1631181405833,
rotation: 0,
sha: 'C1sMi1ZANfhM9YOvkrACnolhzM9uFk1V',
shutterSpeed: '1/2000',
uri: 'psfile://2EGUfSDRD/2021/Micha%201.%20Kindergarten/bild%2002.JPG',
version: 11,
width: 5304 } }
2021-09-25T17:26:01.623Z sync-file-19140 warn Deferred(AssetFileImporter(/photos/2021/Micha 1. Kindergarten/bild 02.JPG).apply) .reject() { _apply(): unexpected missing asset file id
at Object.t.asError (/ps/app/bin/sync-file.js:9:611008)
at y.throw (/ps/app/bin/sync-file.js:9:359330)
at t.AssetFileImporter._apply (/ps/app/bin/sync-file.js:9:805277)
cause:
_apply(): unexpected missing asset file id
at Object.t.asError (/ps/app/bin/sync-file.js:9:611008)
at y.throw (/ps/app/bin/sync-file.js:9:359330)
at t.AssetFileImporter._apply (/ps/app/bin/sync-file.js:9:805277) }
2021-09-25T17:26:01.625Z sync-file-19140 warn Error onError(): Error: AssetFileImporter(/photos/2021/Micha 1. Kindergarten/bild 02.JPG) _apply(): unexpected missing asset file id {"path":"/photos/2021/Micha 1. Kindergarten/bild 02.JPG","af":{"$ctor":"models.AssetFile","aperture":6.3,"assetId":39035,"cameraId":"Inter…
_apply(): unexpected missing asset file id
at Object.t.asError (/ps/app/bin/sync-file.js:9:611008)
at y.throw (/ps/app/bin/sync-file.js:9:359330)
at t.AssetFileImporter._apply (/ps/app/bin/sync-file.js:9:805277) { event: 'nonFatal',
message: 'Failed to import /photos/2021/Micha 1. Kindergarten/bild 02.JPG⁶' }
{"path":"/photos/2021/Micha 1. Kindergarten/bild 02.JPG","error":"Error: AssetFileImporter(/photos/2021/Micha 1. Kindergarten/bild 02.JPG) _apply(): unexpected missing asset file id {\"path\":\"/photos/2021/Micha 1. Kindergarten/bild 02.JPG\",\"af\":{\"$ctor\":\"models.AssetFile\",\"aperture\":6.3,\"assetId\":39035,\"cameraId\":\"Inter…","elapsedMs":3249}
{"ready":true}
2021-09-25T17:26:01.630Z sync-file-19140 info Service(sync-file) exit() { status: 0, reason: 'Processed files from argv', waitForJobs: true, ending: false }
2021-09-25T17:26:01.645Z sync-file-19140 warn Sentry Sentry.beforeSend(): event quota exceeded { exception:
{ values:
[ { stacktrace: { frames: [Array] },
type: '_apply()',
value: 'unexpected missing asset file id' },
{ stacktrace: { frames: [Array] },
type: 'Error',
value:
'AssetFileImporter(/photos/2021/Micha 1. Kindergarten/bild 02.JPG) _apply(): unexpected missing asset file id {"path":"/photos/2021/Micha 1. Kindergarten/bild 02.JPG","af":{"$ctor":"models.AssetFile","aperture":6.3,"assetId":39035,"cameraId":"Internal...',
mechanism: { handled: true, type: 'generic' } } ] },
event_id: '16f4f7645edf43e5913f558ea5313209',
platform: 'node',
timestamp: 1632590761.639,
environment: 'production',
release: '1.1.0+20210815185424',
sdk:
{ integrations:
[ 'InboundFilters',
'FunctionToString',
'Console',
'Http',
'OnUncaughtException',
'OnUnhandledRejection',
'LinkedErrors' ] },
breadcrumbs:
[ { timestamp: 1632590758.261,
category: 'console',
level: 'log',
message:
'\x1B[90m2021-09-25T17:25:58.260Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mSentry\x1B[39m Sentry initialized { isElectron: \x1B[33mfalse\x1B[39m }' },
{ timestamp: 1632590758.264,
category: 'console',
level: 'log',
message:
"\x1B[90m2021-09-25T17:25:58.263Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mService(sync-file)\x1B[39m setup() { version: \x1B[32m'1.1.0'\x1B[39m,\n start: \x1B[33m1632590757633\x1B[39m,\n argv:\n [ \x1B[32m'/usr/local/bin/node'\x1B[39m,\n \x1B[32m'/ps/app/bin/sync-file.js'\x1B[39m,\n \x1B[32m'--debug'\x1B[39m,\n \x1B[32m'/photos/2021/Micha 1. Kindergarten/bild 02.JPG'\x1B[39m ],\n arch: \x1B[32m'x64'\x1B[39m,\n platform: \x1B[32m'linux'\x1B[39m,\n isDocker: \x1B[33mtrue\x1B[39m,\n os: \x1B[32m'Alpine Linux v3.13 on x64'\x1B[39m,\n isPacked: \x1B[33mtrue\x1B[39m,\n isElectron: \x1B[33mfalse\x1B[39m,\n versions:\n { node: \x1B[32m'16.2.0'\x1B[39m,\n v8: \x1B[32m'9.0.257.25-node.16'\x1B[39m,\n uv: \x1B[32m'1.41.0'\x1B[39m,\n zlib: \x1B[32m'1.2.11'\x1B[39m,\n brotli: \x1B[32m'1.0.9'\x1B[39m,\n ares: \x1B[32m'1.17.1'\x1B[39m,\n modules: \x1B[32m'93'\x1B[39m,\n nghttp2: \x1B[32m'1.42.0'\x1B[39m,\n napi: \x1B[32m'8'\x1B[39m,\n llhttp: \x1B[32m'6.0.1'\x1B[39m,\n openssl: \x1B[32m'1.1.1k+quic'\x1B[39m,\n cldr: \x1B[32m'39.0'\x1B[39m,\n icu: \x1B[32m'69.1'\x1B[39m,\n tz: \x1B[32m'2021a'\x1B[39m,\n unicode: \x1B[32m'13.0'\x1B[39m,\n ngtcp2: \x1B[32m'0.1.0-DEV'\x1B[39m,\n nghttp3: \x1B[32m'0.1.0-DEV'\x1B[39m },\n settings: { logLevel: \x1B[32m'info'\x1B[39m, httpPort: \x1B[33m1787\x1B[39m, rpcPort: \x1B[33m1807\x1B[39m, libraryDir: \x1B[32m'/ps/library'\x1B[39m },\n NODE_ENV: \x1B[32m'production'\x1B[39m,\n PS_FORCE_LOCAL_DB_REPLICA: \x1B[32m'false'\x1B[39m,\n PS_LOG_LEVEL: \x1B[32m'info'\x1B[39m }" },
{ timestamp: 1632590758.266,
category: 'console',
level: 'log',
message:
'\x1B[90m2021-09-25T17:25:58.266Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mLibrary(/ps/library)\x1B[39m new()' },
{ timestamp: 1632590758.337,
category: 'console',
level: 'log',
message:
"\x1B[90m2021-09-25T17:25:58.336Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mcurrentLibraryLockOwner(/ps/library/.photostructure)\x1B[39m currentLibraryLockOwner { result:\n { file: { nativePath: \x1B[32m'/ps/library/.photostructure/opened-by/9sevj98-25.json'\x1B[39m },\n serviceName: \x1B[32m'main'\x1B[39m,\n createdAt: \x1B[33m1632585166683\x1B[39m,\n updatedAt: \x1B[33m1632588046682\x1B[39m,\n hostname: \x1B[32m'429c8e352d'\x1B[39m,\n systemUID: \x1B[32m'8n4u-mqpd-q6v0-116j'\x1B[39m,\n v: \x1B[32m'1.1.0'\x1B[39m,\n pid: \x1B[33m25\x1B[39m } }" },
{ timestamp: 1632590758.355,
category: 'console',
level: 'log',
message:
'\x1B[90m2021-09-25T17:25:58.354Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mSettingsIO.importFileSettings(/ps/library/.photostructure/settings.toml)\x1B[39m loaded { imported: { reportErrors: \x1B[33mtrue\x1B[39m } }' },
{ timestamp: 1632590758.365,
category: 'console',
level: 'log',
message:
'\x1B[90m2021-09-25T17:25:58.365Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mStatsDbDir\x1B[39m Set up statsDbDir dir /ps/tmp/sync-state-8u3ak3a014' },
{ timestamp: 1632590758.374,
category: 'console',
level: 'log',
message:
'\x1B[90m2021-09-25T17:25:58.373Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mVacuum\x1B[39m checkRemoteVacuuming() received RPC value { value: \x1B[33mfalse\x1B[39m }' },
{ timestamp: 1632590758.379,
category: 'console',
level: 'log',
message:
"\x1B[90m2021-09-25T17:25:58.379Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mModelDbJanitor(/ps/library/.photostructure)\x1B[39m Finished setup { primaryDbDir: { nativePath: \x1B[32m'/ps/library/.photostructure'\x1B[39m },\n backupDir: { nativePath: \x1B[32m'/ps/library/.photostructure/models/backup'\x1B[39m } }" },
{ timestamp: 1632590758.38,
category: 'console',
level: 'log',
message:
'\x1B[90m2021-09-25T17:25:58.379Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mModelDbJanitor(/ps/library/.photostructure)\x1B[39m ModelDb reading from /ps/library/.photostructure/models/db.sqlite3' },
{ timestamp: 1632590758.38,
category: 'console',
level: 'log',
message:
'\x1B[90m2021-09-25T17:25:58.380Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mLibrary(/ps/library)\x1B[39m Library.setup() finished.' },
{ timestamp: 1632590758.384,
category: 'console',
level: 'log',
message:
'\x1B[90m2021-09-25T17:25:58.384Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[93mwarn \x1B[39m \x1B[34mSyncFileService\x1B[39m copyAssetsToLibrary changed to false' },
{ timestamp: 1632590758.477,
category: 'console',
level: 'log',
message:
'\x1B[90m2021-09-25T17:25:58.477Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mExifTool\x1B[39m Started child process ExifTool:19154' },
{ timestamp: 1632590758.486,
category: 'console',
level: 'log',
message:
'\x1B[90m2021-09-25T17:25:58.486Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mRenice\x1B[39m Renice pid 19154 to BelowNormal' },
{ timestamp: 1632590759.018,
category: 'console',
level: 'log',
message:
'\x1B[90m2021-09-25T17:25:59.018Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mDb(models)\x1B[39m setting up new db connection to /ps/library/.photostructure/models/db.sqlite3 { priorWasNull: \x1B[33mtrue\x1B[39m }' },
{ timestamp: 1632590759.02,
category: 'console',
level: 'log',
message:
"\x1B[90m2021-09-25T17:25:59.020Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mmkdb()\x1B[39m Dynamically setting dbCacheSize to 192 { db: \x1B[32m'/ps/library/.photostructure/models/db.sqlite3'\x1B[39m, dbFileSize: \x1B[33m42123264\x1B[39m }" },
{ timestamp: 1632590759.835,
category: 'console',
level: 'log',
message:
'\x1B[90m2021-09-25T17:25:59.835Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mAssetFileImporter(/photos/2021/Micha 1. Kindergarten/bild 02.JPG)\x1B[39m Setting timeout to 1 minute' },
{ timestamp: 1632590761.295,
category: 'console',
level: 'log',
message:
"\x1B[90m2021-09-25T17:26:01.292Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mAssetFile(undefined).updateFromFile()\x1B[39m after (1434ms) { \x1B[32m'$ctor'\x1B[39m: \x1B[32m'models.AssetFile'\x1B[39m,\n aperture: \x1B[33m6.3\x1B[39m,\n cameraId: \x1B[32m'InternalSerialNumber:000014000000'\x1B[39m,\n capturedAtLocal: \x1B[33m2021082309520500\x1B[39m,\n capturedAtOffset: \x1B[33m0\x1B[39m,\n capturedAtPrecisionMs: \x1B[33m1000\x1B[39m,\n capturedAtSrc: \x1B[32m'tags:DateTimeCreated'\x1B[39m,\n fileSize: \x1B[33m9756669\x1B[39m,\n focalLength: \x1B[32m'105.0 mm'\x1B[39m,\n height: \x1B[33m7952\x1B[39m,\n imageId: \x1B[32m'ShutterCount:5428'\x1B[39m,\n iso: \x1B[33m2500\x1B[39m,\n lensId:\n \x1B[32m'LensID:Sony E 18-200mm F3.5-6.3 OSS LE or Tamron 18-200mm F3.5-6.3 Di III VC or Sony E 18-200mm F3.5-6.3 OSS or Sony E PZ 18-200mm F3.5-6.3 OSS'\x1B[39m,\n make: \x1B[32m'Sony'\x1B[39m,\n meanHash: \x1B[32m'Pyh4fFx8NADgGBg4PHz8/sfv/5eLzwcC'\x1B[39m,\n mimetype: \x1B[32m'image/jpeg'\x1B[39m,\n mode0: \x1B[33m658\x1B[39m,\n mode1: \x1B[33m689\x1B[39m,\n mode2: \x1B[33m219\x1B[39m,\n mode3: \x1B[33m693\x1B[39m,\n mode4: \x1B[33m661\x1B[39m,\n mode5: \x1B[33m700\x1B[39m,\n mode6: \x1B[33m2555\x1B[39m,\n model: \x1B[32m'α7R II'\x1B[39m,\n mountpoint: \x1B[32m'/photos'\x1B[39m,\n mtime: \x1B[33m1631181405833\x1B[39m,\n rotation: \x1B[33m0\x1B[39m,\n sha: \x1B[32m'C1sMi1ZANfhM9YOvkrACnolhzM9uFk1V'\x1B[39m,\n shutterSpeed: \x1B[32m'1/2000'\x1B[39m,\n uri: \x1B[32m'psfile://2EGUfSDRD/2021/Micha%201.%20Kindergarten/bild%2002.JPG'\x1B[39m,\n version: \x1B[33m11\x1B[39m,\n width: \x1B[33m5304\x1B[39m }" },
{ timestamp: 1632590761.304,
category: 'console',
level: 'log',
message:
"\x1B[90m2021-09-25T17:26:01.301Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mAssetFileFinder(2021/Micha 1. Kindergarten/bild 02.JPG)\x1B[39m Found sibling AssetFile { \x1B[32m'$ctor'\x1B[39m: \x1B[32m'models.AssetFile'\x1B[39m,\n aperture: \x1B[33m6.3\x1B[39m,\n assetId: \x1B[33m39035\x1B[39m,\n cameraId: \x1B[32m'InternalSerialNumber:000014000000'\x1B[39m,\n capturedAtLocal: \x1B[33m2021082309520500\x1B[39m,\n capturedAtOffset: \x1B[33m0\x1B[39m,\n capturedAtPrecisionMs: \x1B[33m1000\x1B[39m,\n capturedAtSrc: \x1B[32m'tags:DateTimeCreated'\x1B[39m,\n createdAt: \x1B[33m1631751358717\x1B[39m,\n fileSize: \x1B[33m9783775\x1B[39m,\n focalLength: \x1B[32m'105.0 mm'\x1B[39m,\n height: \x1B[33m7952\x1B[39m,\n id: \x1B[33m39035\x1B[39m,\n imageId: \x1B[32m'ShutterCount:5428'\x1B[39m,\n iso: \x1B[33m2500\x1B[39m,\n lensId:\n \x1B[32m'LensID:Sony E 18-200mm F3.5-6.3 OSS LE or Tamron 18-200mm F3.5-6.3 Di III VC or Sony E 18-200mm F3.5-6.3 OSS or Sony E PZ 18-200mm F3.5-6.3 OSS'\x1B[39m,\n make: \x1B[32m'Sony'\x1B[39m,\n meanHash: \x1B[32m'Pzh4fHx4cADgGBg4PHz8/sf/f5eby4IA'\x1B[39m,\n mimetype: \x1B[32m'image/jpeg'\x1B[39m,\n mode0: \x1B[33m658\x1B[39m,\n mode1: \x1B[33m689\x1B[39m,\n mode2: \x1B[33m219\x1B[39m,\n mode3: \x1B[33m693\x1B[39m,\n mode4: \x1B[33m661\x1B[39m,\n mode5: \x1B[33m660\x1B[39m,\n mode6: \x1B[33m2555\x1B[39m,\n model: \x1B[32m'α7R II'\x1B[39m,\n mountpoint: \x1B[32m'/photos'\x1B[39m,\n mtime: \x1B[33m1631182307567\x1B[39m,\n rotation: \x1B[33m0\x1B[39m,\n sha: \x1B[32m'oWtrbDNsgInySMsniL2ZiVmCzCO3MvcV'\x1B[39m,\n shown: \x1B[33mtrue\x1B[39m,\n shutterSpeed: \x1B[32m'1/2000'\x1B[39m,\n updateCount: \x1B[33m6\x1B[39m,\n updatedAt: \x1B[33m1632589476705\x1B[39m,\n uri: \x1B[32m'psfile://2EGUfSDRD/2021/Micha%201.%20Kindergarten/DSC09697.JPG'\x1B[39m,\n version: \x1B[33m11\x1B[39m,\n width: \x1B[33m5304\x1B[39m }" },
{ timestamp: 1632590761.345,
category: 'console',
level: 'log',
message:
'\x1B[90m2021-09-25T17:26:01.345Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mAssetFileFinder(2021/Micha 1. Kindergarten/bild 02.JPG)\x1B[39m apply(): byCapturedAtOrImageHash { assetId: \x1B[33m39035\x1B[39m }' },
{ timestamp: 1632590761.346,
category: 'console',
level: 'log',
message:
'\x1B[90m2021-09-25T17:26:01.346Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mAssetFileFinder(2021/Micha 1. Kindergarten/bild 02.JPG)\x1B[39m Found existing asset for file' },
{ timestamp: 1632590761.346,
category: 'console',
level: 'log',
message:
'\x1B[90m2021-09-25T17:26:01.346Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mAssetFileRepository\x1B[39m importAssetFile(psfile://2EGUfSDRD/2021/Micha%201.%20Kindergarten/bild%2002.JPG): importFile was a no-op' },
{ timestamp: 1632590761.351,
category: 'console',
level: 'log',
message:
"\x1B[90m2021-09-25T17:26:01.351Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mAdvisoryLock\x1B[39m withLocks() { lockNames: [ \x1B[32m'assetId:39035'\x1B[39m ], acquired: \x1B[33mtrue\x1B[39m }" },
{ timestamp: 1632590761.351,
category: 'console',
level: 'log',
message:
'\x1B[90m2021-09-25T17:26:01.351Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mUpdateAsset(39035)\x1B[39m starting' },
{ timestamp: 1632590761.366,
category: 'console',
level: 'log',
message:
"\x1B[90m2021-09-25T17:26:01.365Z\x1B[39m \x1B[40m\x1B[36msync-file-19140\x1B[39m\x1B[49m \x1B[36minfo \x1B[39m \x1B[34mAssetPreviewBuilder(39035)\x1B[39m build_(): asset file candidates: [ \x1B[32m'psfile://2EGUfSDRD/2021/Micha%201.%20Kindergarten/DSC09697.JPG'\x1B[39m ]" },
'… (25 total items)' ] }
2021-09-25T17:26:01.882Z sync-file-19140 info Service(sync-file) exit(): waiting for enqueued jobs to complete... { pendingNames: [] }
2021-09-25T17:26:01.883Z sync-file-19140 info Service(sync-file) exit(): enqueued jobs finished.
2021-09-25T17:26:01.884Z sync-file-19140 info Endable endEndables() { isTest: false, isSingleSpecTests: false }
2021-09-25T17:26:01.886Z sync-file-19140 info rawTagsCache stats { size: 2, cacheHitsSync: 16, cacheMisses: 2 }
2021-09-25T17:26:01.886Z sync-file-19140 info BaseFileCache stats { size: 16, cacheHitsSync: 2, cacheMisses: 16 }
2021-09-25T17:26:01.887Z sync-file-19140 info PosixFileCache stats { size: 23, cacheHitsSync: 59, cacheMisses: 23 }
2021-09-25T17:26:01.887Z sync-file-19140 info readdirCache stats { size: 5, cacheHitsSync: 1, cacheMisses: 5 }
2021-09-25T17:26:01.888Z sync-file-19140 info rpc.Client(db@localhost:1807#0) end() { ending: true, timings: {}, pendingSize: 0 }
2021-09-25T17:26:01.888Z sync-file-19140 info rpc.Client(db@localhost:1807#0) close() { ended: true }
2021-09-25T17:26:01.888Z sync-file-19140 info noMediaDirsCache stats { size: 2, cacheMisses: 2 }
2021-09-25T17:26:01.889Z sync-file-19140 info imageHashCache stats { size: 1, cacheMisses: 1 }
2021-09-25T17:26:01.943Z sync-file-19140 info ExifTool childEndCounts [Function: childEndCounts]
2021-09-25T17:26:01.948Z sync-file-19140 warn ChildProcess endProcess(19154) { killed: false, connected: false }
2021-09-25T17:26:01.948Z sync-file-19140 info ExifTool on(childExit) 19154
2021-09-25T17:26:01.949Z sync-file-19140 info Pids killPid { pid: 19154, force: false }
2021-09-25T17:26:01.959Z sync-file-19140 info OpenedByIO() debug 'onEnd(): unlinking /ps/library/.photostructure/opened-by/9sevj98-19140.json'
2021-09-25T17:26:01.962Z sync-file-19140 info Library(/ps/library) onEnd(): finished.
2021-09-25T17:26:01.962Z sync-file-19140 info Db(models) closing db Database {
name: '/ps/library/.photostructure/models/db.sqlite3',
open: true,
inTransaction: false,
readonly: false,
memory: false }
2021-09-25T17:26:01.972Z sync-file-19140 info Service(sync-file) timings: { 'AssetFileFinder.assetByCapturedAtOrImageHash': { sumSec: 1.45, mean: 1500, k: 1 },
'AssetFileFinder.newAssetFile': { sumSec: 1.44, mean: 1400, k: 1 },
'img.imageHash': { sumSec: 1.36, mean: 1400, k: 1 },
'tag.read()': { sumSec: 0.54, mean: 270, sd: 280, k: 2 },
'Service.startup()': { sumSec: 0.204, mean: 200, k: 1 },
'fs.sha': { sumSec: 0.118, mean: 59, sd: 8.5, k: 2 },
'img.AssetPreviewBuilder.build()': { sumSec: 0.077, mean: 77, k: 1 },
'img.sharpReadable.JPG': { sumSec: 0.055, mean: 55, k: 1 },
'img.read.imgFromExif.ThumbnailImage': { sumSec: 0.054, mean: 54, k: 1 },
'fs.stat': { sumSec: 0.035, mean: 35, k: 1 } }
2021-09-25T17:26:01.973Z sync-file-19140 info Service(sync-file) memory use on exit { ok: [ 'Memory by sync-file (29 MB) is OK' ], warn: [], bad: [], fail: [] }
Ah crap, sorry about that: there was an assertion I added in v1.1 (unexpected missing asset file id
) that is causing you grief. I’ve fixed it in the next build (that I’m hoping to release Real Soon Now)…
no problem good to know it’s going to be fixed some day! thx
if I can help in testing a new version let me know
I also realized I have few assets missing in one of folders in 1.1. Upgraded to 2.0. Still missing.
Ran ./photostructure sync-file --debug /path/to/file.jpg and grepped for below
Nothing showed up. No errors either, only few warns. Should I also provide debug output here?
Thanks.
K