✅ Asset search support

Feature Requests
, ,
7

Thanks for starting working on it. Can’t wait… How do you plan to handle terms with spaces or colons?

8

The query parser will look for double or single quotes, so who:"Sam whatshisface" will work.

1 Like
9

2 Likes
10

:flight_departure: I’ve got the grammar nailed down.

Update 2021-03-24: I was able to add before: and after: namespaces, but as the MATCH operator doesn’t have a NOT MATCH sibling, I had to remove the - query negation from the grammar.

Update 2021-03-24: OMG I figured out how to do negation, but only for namespaced query terms.

Update 2021-04-10: The grammar has changed sufficiently to delete most of this post: see the instructions on the website instead.

3 Likes
11

I was able to add --query support to the list tool, so queries are going to be available to both the UI and the CLI:

$ ./photostructure list --help
Usage: list [options]

List paths in a Library.

Options:
...
  --query <TERMS...>   Apply the given query and return all matching asset file
                       paths. Must be the last argument. Incompatible with
                       --where.
  -j --json            Emit a JSON object for every asset file with "hidden"
                       and "shown" flags
  --dump               Emit a JSON object that includes all column values
  --where <WHERE>      Apply a SQL WHERE clause to filter the rows returned.
                       (default: "")
  --orderby <ORDERBY>  Apply a SQL ORDER BY clause to filter the rows returned.
                       (default: "")
3 Likes
12

I had a bit of a hitch in my search giddyup: SQLite doesn’t support multiple MATCH clauses, so I had to teach PhotoStructure how to condense these queries down to disjunctive normal form. This seems like it works now, and I’m building out the frontend now.

If you’ve emailed me logs or asked for support for the last day or two, know that I’ll be getting to your message soon.

1 Like
13

Have you considered using INTERSECT instead? Then you could use one MATCH per subquery

14

I may be misunderstanding you, but this would still mean I have to reduce to conjunctive normal form (rather than disjunctive normal form), right?

FWIW, I’ve got the SQL generator working…

15

Well, I assumed you have multiple AND conditions that’s why I suggested INTERSECT but I think if you rewrite the generating code to use INTERSECT for AND clause and UNION ALL for OR clause, you don’t need to convert to any normal forms and use it as is. But obviously my idea requires at least performance check

1 Like
16

Yeah, if UNION turns out to have horrible performance, I will certainly try INTERSECT. Thanks!

Goodness knows that I’ve been surprised by seemingly equivalent SQL that was 10x+ faster (especially when query planner wasn’t very sophisticated).

1 Like
17

Have you tried to do UNION ALL and then DISTINCT at the end of the query? They say UNION ALL has better performance than just UNION

1 Like
18

I’ll check that out, thanks.

20

@mrm I’ve checked the documentation for your search feature and it looks really awesome. I suggest to consider more convenient syntax for date ranges which, in my opinion, going to be one of the most used filters.

Please also consider some libraries for parsing dates like date:"last Friday"

For example Sugar - Home or GitHub - wanasit/chrono: A natural language date parser in Javascript

I would be nice if you also think how to fix one annoying issue all DAMs have. If you have hierarchical keywords a/b/c and you look for a/b there should be a way to distinguish cases when you want to find exactly that keyword without its descendants. Maybe it could be solved by kw:a/b AND -kw:a/b/* but it would be nice to have neater syntax for that case

21

This is interesting to me, because I actually don’t think I’ll use date ranges very much. I’m much more interested in searching by people and keywords, especially in combination.

Obviously neither of us is right or wrong, but it does underscore that there are lots of different preferred use cases.

1 Like
22

Thanks for those links!

I thought non-ISO-formatted dates would be neat, too, and did a bit of looking for a date parser that supported internationalization, but (like most of npm), were either English-only, had no tests, had been abandoned, or all of the above.

I think just a could regexs would get year, month, week, and day parsing. Here’s a google sheet: please ask for edit access if you’d like to add a language or other examples.

Would tag:/a/b/c suffice (where /a/b/c is the entire path of the tag)?

23

Just as more feedback… I do want to deal with date ranges a lot, but personally never seem to use the “natural language” type date formats (last week, etc.). Maybe it’s the programmer in me that finds them too ambiguous (or I’m just too old :-)).

Anyhow, I certainly don’t mind if they are there for folks that like them, I can see the use, but just to toss in my 2 cents…

24

Sorry, I didn’t explain better about tags

What if I want to find tag:/a/b but I don’t want to include tag:/a/b/c or any other descendants of tag:/a/b

Real use case example kw:dog but you want to exclude kw:dog/Fido, kw:dog/Rex etc. So basically, you want to find only unknown dogs.

Hope this example explains my concern better

25

I’m also a programmer and yyyy-MM-dd is the only thing that I prefer most of the time, but as your product’s supposed main audience is non-IT, I assume date:"last summer" is more preferable to type than after:2020-06-01 AND before:2020-09-01

26

BTW, relative dates you can find even in a very IT stuff

--date=relative

https://git-scm.com/docs/git-log

1 Like
27

Ah! I already added operators for date terms: perhaps tag:=/keywords/dog means only look at that tag, and not match descendant tags: would that work?

Edit 20210410: deleted the remainder of the post, as it has been superseded by the documentation.

1 Like