Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: debug multisigfile test #445

Open
wants to merge 94 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
480f319
refactor & rename & consolidate
ctb Aug 17, 2024
e6b1c5b
remove 'lower'
ctb Aug 17, 2024
153f246
Merge branch 'main' of github.com:sourmash-bio/sourmash_plugin_branch…
ctb Aug 18, 2024
0d7a556
add cargo doc output for private fn
ctb Aug 18, 2024
df753db
Merge branch 'main' of github.com:sourmash-bio/sourmash_plugin_branch…
ctb Aug 18, 2024
1da0cf3
add a few comments/docs
ctb Aug 18, 2024
2e7f027
switch to dev version of sourmash
ctb Aug 18, 2024
6b9e00f
tracking
ctb Aug 18, 2024
2747935
cleaner
ctb Aug 18, 2024
4f49ef8
cleanup
ctb Aug 18, 2024
af1c82d
load rocksdb natively
ctb Aug 18, 2024
53924d6
foo
ctb Aug 18, 2024
e5faed8
Merge branch 'main' of github.com:sourmash-bio/sourmash_plugin_branch…
ctb Aug 19, 2024
7649375
Merge branch 'main' of github.com:sourmash-bio/sourmash_plugin_branch…
ctb Aug 19, 2024
e4618f0
Merge branch 'ctb_misc_cleanup' into ctb_misc2
ctb Aug 19, 2024
3462f92
cargo fmt
ctb Aug 19, 2024
9823ef6
upd
ctb Aug 20, 2024
bfb5053
upd
ctb Aug 20, 2024
c311a69
fix fmt
ctb Aug 20, 2024
28b43d8
MRG: create `MultiCollection` for collections that span multiple file…
ctb Aug 20, 2024
a1b19ae
clippy fixes
ctb Aug 20, 2024
51a14ac
compiling again
ctb Aug 20, 2024
99bd174
cleanup
ctb Aug 20, 2024
36d33a5
bump sourmash to v0.15.1
ctb Aug 21, 2024
02bf7e9
Merge branch 'bump_sourmash' into ctb_misc2
ctb Aug 21, 2024
7f0b010
check if is rocksdb
ctb Aug 21, 2024
b9972c6
Merge branch 'main' of github.com:sourmash-bio/sourmash_plugin_branch…
ctb Aug 21, 2024
5561911
weird error
ctb Aug 21, 2024
dfe56d3
use remove_unwrap branch of sourmash
ctb Aug 21, 2024
e6e80f3
get index to work with MultiCollection
ctb Aug 21, 2024
fed4db3
old bug now fixed
ctb Aug 21, 2024
f5331ef
clippy, format, and fix
ctb Aug 21, 2024
8f90129
make names clearer
ctb Aug 21, 2024
4511347
ditch MultiCollection for index, at least for now
ctb Aug 21, 2024
4ea6730
testy testy
ctb Aug 21, 2024
ac35b24
getting closer
ctb Aug 21, 2024
741a44a
update sourmash
ctb Aug 22, 2024
d429205
mark failing tests
ctb Aug 22, 2024
994fcec
upd
ctb Aug 24, 2024
8451259
cargo fmt
ctb Aug 24, 2024
91b04b5
MRG: test exit from `pairwise` and `multisearch` if no loaded sketche…
ctb Aug 24, 2024
b3e5b81
MRG: switch to more efficient use of `Collection` by removing cloning…
ctb Aug 24, 2024
97db857
MRG: add tests for RocksDB/RevIndex, standalone manifests, and flexib…
ctb Aug 24, 2024
551758f
reenable and fix test_fastgather.py::test_indexed_against
ctb Aug 24, 2024
e3e95fc
impl Deref for MultiCollection
ctb Aug 24, 2024
86d175f
debugging multisigfile test
ctb Aug 24, 2024
8d39a4f
clippy
ctb Aug 24, 2024
3439592
switch to using load_sketches method
ctb Aug 24, 2024
d3fa529
deref doesn't actually make sense for MultiCollection
ctb Aug 24, 2024
5a20381
update to latest sourmash code
ctb Aug 27, 2024
6563b0a
update to latest sourmash code
ctb Aug 27, 2024
45608c7
simplify
ctb Aug 27, 2024
bd256dd
update to latest sourmash code
ctb Aug 27, 2024
675974e
Merge branch 'main' of github.com:sourmash-bio/sourmash_plugin_branch…
ctb Aug 27, 2024
afa0faf
remove unnecessary flag
ctb Aug 27, 2024
89b1c08
Merge branch 'main' of github.com:sourmash-bio/sourmash_plugin_branch…
ctb Sep 8, 2024
d120547
MRG: support & test loading of standalone manifests within pathlists …
ctb Sep 9, 2024
73c7f53
MRG: documentation updates based on new collection loading (#444)
ctb Sep 9, 2024
e32638a
Merge branch 'main' of github.com:sourmash-bio/sourmash_plugin_branch…
ctb Sep 9, 2024
fcae8e7
Merge branch 'main' of github.com:sourmash-bio/sourmash_plugin_branch…
ctb Sep 10, 2024
74d7367
Update src/lib.rs
ctb Sep 10, 2024
4e28d64
switch unwrap to expect
ctb Sep 10, 2024
74e2217
Merge branch 'ctb_misc2' of github.com:sourmash-bio/sourmash_plugin_b…
ctb Sep 10, 2024
de35cd5
move unwrap to expect
ctb Sep 10, 2024
1e5ac07
minor cleanup
ctb Sep 10, 2024
388a49a
cargo fmt
ctb Sep 11, 2024
7be1883
provide legacy method to avoid xfail on index loading
ctb Sep 15, 2024
679b972
switch to using reference
ctb Sep 15, 2024
a9143d0
update docs to reflect pathlist behavior
ctb Sep 15, 2024
574cd28
test recursive nature of MultiCollection
ctb Sep 15, 2024
a5b4299
re-enable test that is now passing
ctb Sep 15, 2024
74b9ae6
update to latest sourmash
ctb Sep 16, 2024
9df421d
upd sourmash
ctb Sep 16, 2024
34e94f2
Merge branch 'ctb_misc2' into fix_multisig_foo
ctb Sep 16, 2024
ccd26da
Merge branch 'main' of github.com:sourmash-bio/sourmash_plugin_branch…
ctb Sep 16, 2024
4a9a580
Merge branch 'ctb_misc2' into fix_multisig_foo
ctb Sep 16, 2024
551c04e
upd sourmash
ctb Sep 16, 2024
661ae71
more better debug
ctb Sep 16, 2024
847917f
update sourmash
ctb Sep 17, 2024
ca33026
Merge branch 'ctb_misc2' into fix_multisig_foo
ctb Sep 17, 2024
97c18ed
switch to exp branch of sourmash
ctb Sep 17, 2024
bfc12ea
narrow debugging
ctb Sep 17, 2024
730de67
add some debugging code
ctb Sep 17, 2024
90cb8e3
upd
ctb Sep 17, 2024
9733d47
mut MultiCollection
ctb Sep 18, 2024
019fd1b
cleanup
ctb Sep 18, 2024
780fbda
update after merge of sourmash-bio/sourmash#3305
ctb Sep 21, 2024
84934a7
fix contains_revindex
ctb Sep 22, 2024
56fb948
add trace commands for tracing loading
ctb Sep 22, 2024
cf485a3
Merge branch 'ctb_misc2' into fix_multisig_foo
ctb Sep 24, 2024
2e2f4ef
update to latest sourmash branch
ctb Sep 24, 2024
3b815dd
Merge branch 'main' of github.com:sourmash-bio/sourmash_plugin_branch…
ctb Oct 27, 2024
f3be839
upd
ctb Oct 27, 2024
80e842a
let test continue to fail
ctb Oct 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 7 additions & 8 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,9 @@ crate-type = ["cdylib"]
[dependencies]
pyo3 = { version = "0.22.5", features = ["extension-module", "anyhow"] }
rayon = "1.10.0"
sourmash = { git = "https://github.com/sourmash-bio/sourmash.git", branch = "debug_multisigfile", features = ["branchwater"] }
serde = { version = "1.0.210", features = ["derive"] }
sourmash = { version = "0.16.0", features = ["branchwater"] }
#sourmash = { version = "0.16.0", features = ["branchwater"] }
serde_json = "1.0.132"
niffler = "2.4.0"
log = "0.4.22"
Expand Down
7 changes: 5 additions & 2 deletions src/python/tests/test_fastgather.py
Original file line number Diff line number Diff line change
Expand Up @@ -310,8 +310,8 @@ def test_bad_against_3(runtmp, capfd):
assert 'InvalidArchive' in captured.err


@pytest.mark.xfail(reason="should work, bug")
def test_against_multisigfile(runtmp, zip_against):
#@pytest.mark.xfail(reason="should work, bug")
def test_against_pathlist_multisigfile(runtmp, zip_against, indexed_against):
# test against a sigfile that contains multiple sketches
query = get_test_data('SRR606249.sig.gz')
against_list = runtmp.output('against.txt')
Expand All @@ -327,6 +327,9 @@ def test_against_multisigfile(runtmp, zip_against):
if zip_against:
against_list = zip_siglist(runtmp, against_list, runtmp.output('against.zip'))

if indexed_against:
against_list = index_siglist(runtmp, against_list, runtmp.output('db'))

g_output = runtmp.output('gather.csv')
p_output = runtmp.output('prefetch.csv')

Expand Down
Loading