MRG: refactor fastgather/fastmultigather CSV output to use mpsc channels

[ctb]

Note: PR into #568

This PR adds support for a single output file with -o for fastmultigather on a non-RocksDB database. It also refactors fastgather to use the same underlying mpsc mechanism, so all three gather output mechanisms share more code in common. The ultimate goal is to enable a better internal API, ref #569 and #552.

In particular, this means that now -o works the same way on a RocksDB database and on a non-RocksDB database :).

This PR also disables the current functionality of creating individual output files for each query, which simplifies matters greatly, but does break backwards compatibility. See long-ranging discussions over in sourmash-bio/sourmash#2722 and sourmash-bio/sourmash#2328.

Finally, one last breakage: --create-empty-results now only creates empty prefetch results files, and not gather results files.

This PR also:

• Uses new sourmash Collection::min_max_scaled method (ref update code to use min_max_scaled #527);
• makes sure utils::prefetch(...) only yields non-zero overlaps, preventing infinite loops;
• fixes clippy messages about complex return tuples/types;

This PR:

• Fixes fastmultigather --output is only being used when searching against a rocksdb #239
• Fixes Can't get fastmultigather to write to a specific output file #299
• Fixes update code to use min_max_scaled #527

TODO:

• adjust output_empty_results - ref MRG: optionally, create empty results files for in-memory fastmultigather #446
• update docs
• fix remaining tests
• write example/demo code to split results out by query

[ctb]

Ready for review @bluegenes!

[bluegenes]

a couple documentation qualms - otherwise lgtm 🎉

[ctb]

Here's the complete updated doc section:

Running fastmultigather

fastmultigather takes a collection of query metagenomes and a collection of sketches as a database, and outputs a CSV file containing all the matches.

sourmash scripts fastmultigather queries.manifest.csv database.zip --cores 4 --save-matches -o results.csv

We suggest using standalone manifest CSVs wherever possible, especially if
the queries are large.

The main advantage that fastmultigather has over running
fastgather on multiple queries is that fastmultigather only needs
to load the database once for all queries, unlike with fastgather;
this can be a significant time savings for large databases.

Output files for fastmultigather

fastmultigather will output a gather file containing all results in
one file, specified with -o/--output. fastmultigather gather CSVs
provide the same columns as fastgather, above.

In addition, on a database of sketches (but not on RocksDB indexes)
fastmultigather will output a prefetch file containing all
overlapping matches between that query and the database. The prefetch
CSV will be named {signame}.prefetch.csv, where {signame} is the
name of your sourmash signature.

--save-matches is an optional flag that will save the matched hashes
for each query in a separate sourmash signature
{signame}.matches.sig. This can be useful for debugging or for
further analysis.

Warning: At the moment, if two different queries have the same
{signame}, the output files for one query will be overwritten by
the results from the other query. The behavior here is undefined in
practice, because of multithreading: we don't know what queries will
be executed when or files will be written first.