feat: store also Hypothes.is tags

[ankostis]

(address #195) store Hypothesis tags in a new context line, prefixed by hash(#), separated by space.

[ankostis]

Unfortunately, when tags are stored as #tag-text searches fail, and all visits match (see karlicoss/promnesia#195).

Any ideas why? [edit: I have no experience with SQLAlchemy]

[ankostis]

I did the change in server code discused in #194, but i'm not confident to build and install the extension, to test it, so i will leave the respective change to you @karlicoss.
You may merge this PR after i made some documentation changes for the new feature.

[ankostis]

This PR is ready by me.

Fixes required

Respective hanges to the JS-code of searching bookmarks & history are pending,
because currently it fetches all of them when searching for a #hash-tag.

Enhancement for the future

Currently it matches also url-fragments in context e.g #foo matches https://example.com#foo-bar.

An improvement would be to execute this SQL with parameter {tag} when a hash-tag is queried:

SELECT * FROM visits WHERE context REGEXP '(?<!\w)#{tag}'

Also need to explicitly extract tags from pocket, takeout (labels), etc.

[karlicoss]

Thanks! I guess it makes sense, will give it a think what would be a proper strategy for handling these.

An improvement would be to execute this SQL with parameter {tag} when a hash-tag is queried:

Yep. But again, need to think first what's the most reasonable generic way to do this -- I want to avoid feature bloat :)

because currently it fetches all of them when searching for a #hash-tag.

Yeah. This is one of the most annoying bits to me as well, that the logic (especially normalization logic) is spread out between python and JS. At some point I was even considering automatic translation of the code, but it turned to be out more tedious than I expected. Maybe a good intermediate solution would be to rely on the backend for normalization & some other logic when it's present, and without the backed, fallback on the 'simple' normalization, at least for now.

If it's too spammy for you, you can disable local history & bookmarks in the settings for now, before it's properly fixed. (another nice feature for search to have -- to make it possible to specify which sources to search over)

P.S. CI jobs failed, but it's already fixed on master, so not a problem. Also need to finally figure out how to trigger github actions on an incoming pull request...

I suppose you're happy with running off a git version? I wanted to do a few other fixes and maybe release at the end off week.

[ankostis]

I suppose you're happy with running off a git version?

I do run all of your stuff like that, for editing on the sport in case bugs arise.
For instance i'm consistently bugged by issues when twitter.all & github.all don't have both backends (e.g. gdpr-archive & events) - they fail.

[karlicoss]

Yeah, for twitter.all/github.all -- I guess need a more generic defensive mechanism (it's hard to make it dynamic enough & mypy-safe at the same time)

Although modyfying in place is perfectly valid way of configuring, you can also use overlays with your own twitter/all.py or github/all.py to exclude gdpr archives etc https://github.com/karlicoss/HPI/blob/master/doc/SETUP.org#addingmodifying-modules

You also might find karlicoss/HPI#102 useful (and maybe if you have some thoughts on the subject, feel free to share!)