Possibility to upstream

[j605]

I was in the #notmuch channel on IRC, when the issue of an easy way search addresses came up again and I suggested they use notmuch-addrlookup-c. It was suggested since this works well in surfacing the addresses that one needs, it will be great to have it as part of notmuch-address itself. Did you think about this before? It would be great have it upstream! @bremner was also interested in bringing this functionality over.

[aperezdc]

Definitely not a bad idea! I will look into submitting the tool for inclusion in the Notmuch repository.

[bremner]

Adrian Perez <[email protected]> writes:

Definitely not a bad idea! I will look into submitting the tool for inclusion in the Notmuch repository.

Hi Adrian, Hi all; Hopefully it is possible to merge nomuch-addrlookup-c into "notmuch address", rather providing a seperate tool. Does anyone know of any (technical) blockers for that? d

[aperezdc]

@bremner That is what I have in mind. I'll try to find some time to prepare a patch during the weekend 😄

[anarcat]

Does anyone know of any (technical) blockers for that?

I just reviewed both code bases and here's what I found.

notmuch-address is basically a wrapper around notmuch-search, which means it's limited in what it can do. like notmuch-search, it can't sort by frequency, so it can't reproduce the nice heuristics setup in addrlookup. here's the notmuch-address entry point, from what I could find:

https://sources.debian.org/src/notmuch/0.29.3-1/notmuch-search.c/#L873

One problem is we only have NOTMUCH_SORT_OLDEST_FIRST, NOTMUCH_SORT_NEWEST_FIRST, NOTMUCH_SORT_MESSAGE_ID and NOTMUCH_SORT_UNSORTED in the notmuch_sort_t enum. It would certainly be nice to extend that to include (say) NOTMUCH_SORT_FREQUENCY_ASC and NOTMUCH_SORT_FREQUENCY_DESC (for completeness), without necessarily adding it to notmuch-search itself in a first stage. But then it could be used in notmuch-address by passing it in the search_context_t...

The other thing is that notmuch-address only does one query. addrlookup does several, and a variable number. There's also a special "address book" tag (set in addrbook_tag parameter of the notmuch config file, defaulting to addressbook) which allows the user to tag messages to have their From address "be" in the address book in priority.

Those heuristics are set in:

notmuch-addrlookup-c/notmuch-addrlookup.c

Line 144 in f6a296d

create_queries (notmuch_database_t *db,

specifically:

notmuch-addrlookup-c/notmuch-addrlookup.c

Line 148 in f6a296d

/* Pass 1: looks at all the "from:" addresses with the address book tag */

notmuch-addrlookup-c/notmuch-addrlookup.c

Line 157 in f6a296d

/* Pass 2: looks at all "to:" addresses sent from the primary e-mail */

notmuch-addrlookup-c/notmuch-addrlookup.c

Line 186 in f6a296d

/* Pass 3: With only a few hits, add all the "from:" addresses */

In other words:

1. search From: addresses in messages tagged with the addrbook_tag
2. search To: addresses in messages sent from the primary e-mail (primary_email set in the nomuch config)
3. if pass 1 and 2 yield less than 10 results, search any From email

The notmuch-address heuristic seem to be:

1. pass whatever query string to notmuch-search and return the results

One improvement we could already do with notmuch-address would be to enforce searching for to/from headers, which doesn't seem to be the case right now. I have a wrapper script where I do basically this:

exec notmuch address from:"$*"

I would love to be able to do to:"$*" here, but that gives the unexpected result of the From addresses instead of To in that search... And --output=recipients does not fix that at all...

Anyways, that's all I got... I hope that's useful!

[dkg]

fwiw, i'd also be very happy to see this kind of feature upstreamed. I can't do the integration work myself, but if patches were presented on-list, i'd be happy to help review them.

[anarcat]

it seems to me that just shoving notmuch-addrlookup.c into notmuch as notmuch-search.c and hijacking notmuch_address_command would be a somewhat "good first step". the blockers are:

1. this code is MIT, notmuch is GPL-3, which are compatible, but the copyright notices would need to be included in debian/copyright
2. the test suite would need to pass, and be expanded to cover for the new behavior
3. the current notmuch address options would need to be supported

@aperezdc are you still interested in upstreaming this?

I have been bitten by notmuch-address a few times already: i often send email to strangers which come up earlier in the search because they post frequently to mailing lists i subscribe to, and share a first name with a family member... i have a silly workaround which involves adding the email to an address book by hand but i'd love to see this adopted in notmuch...

[bremner]

Feel free to start notmuch-address.c if that makes life easier. The current integration / code-borrowing with notmuch-search is an implementation detail which need not be preserved. I think it will help keep us all sane if the new contribution is relicensed as GPL3+. If that's really a blocker we can talk.

[bremner]

Some incremental improvements in progress upstream:
https://nmbug.notmuchmail.org/nmweb/show/20211219181854.115918-1-david%40tethera.net

[anarcat]

i did this recently, which made outgoing email show up as well, at the cost of making things slower:

https://gitlab.com/anarcat/scripts/-/commit/13e20dff39cffd3ffd853c5e13bbc4ede5bb434b

obviously far from upstream, but maybe food for thought.

thanks for working on this!