Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(feat): custom reopen with read_elem_as_dask for remote h5ad #1665

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Merge branch 'main' into ig/reopen_for_read_elem_as_dask

abc13bd
Select commit
Loading
Failed to load commit list.
Draft

(feat): custom reopen with read_elem_as_dask for remote h5ad #1665

Merge branch 'main' into ig/reopen_for_read_elem_as_dask
abc13bd
Select commit
Loading
Failed to load commit list.
scverse-benchmark / benchmark succeeded Oct 17, 2024 in 7m 10s

Benchmark

Benchmark run successful

Details

All benchmarks:

Change Before [d7643e9] <0.11.0rc2~16> After [abc13bd] Ratio Benchmark (Parameter)
1707515.0 1707439.0 1 anndata.GarbargeCollectionSuite.track_peakmem_garbage_collection
134M 145M 1.08 readwrite.H5ADBackedWriteSuite.peakmem_write_compressed('pbmc3k')
137M 135M 0.98 readwrite.H5ADBackedWriteSuite.peakmem_write_full('pbmc3k')
692±3ms 675±4ms 0.97 readwrite.H5ADBackedWriteSuite.time_write_compressed('pbmc3k')
195±6ms 188±0.5ms 0.96 readwrite.H5ADBackedWriteSuite.time_write_full('pbmc3k')
14.49609375 14.48046875 1 readwrite.H5ADBackedWriteSuite.track_peakmem_write_compressed('pbmc3k')
14.515625 14.484375 1 readwrite.H5ADBackedWriteSuite.track_peakmem_write_full('pbmc3k')
91325715 91326169 1 readwrite.H5ADInMemorySizeSuite.track_actual_in_memory_size('pbmc3k')
23564294 23564294 1 readwrite.H5ADInMemorySizeSuite.track_in_memory_size('pbmc3k')
23.6M 23.6M 1 readwrite.H5ADReadSuite.mem_readfull_object('pbmc3k')
129M 127M 0.98 readwrite.H5ADReadSuite.peakmem_read_backed('pbmc3k')
150M 151M 1 readwrite.H5ADReadSuite.peakmem_read_full('pbmc3k')
137±2ms 139±4ms 1.01 readwrite.H5ADReadSuite.time_read_full('pbmc3k')
1.2099052942089739 1.1779316573837122 0.97 readwrite.H5ADReadSuite.track_read_full_memratio('pbmc3k')
150M 152M 1.02 readwrite.H5ADWriteSuite.peakmem_write_compressed('pbmc3k')
144M 154M 1.06 readwrite.H5ADWriteSuite.peakmem_write_full('pbmc3k')
566±0.7ms 571±6ms 1.01 readwrite.H5ADWriteSuite.time_write_compressed('pbmc3k')
91.1±0.8ms 90.2±0.5ms 0.99 readwrite.H5ADWriteSuite.time_write_full('pbmc3k')
6.8125 7.2265625 1.06 readwrite.H5ADWriteSuite.track_peakmem_write_compressed('pbmc3k')
6.515625 6.3359375 0.97 readwrite.H5ADWriteSuite.track_peakmem_write_full('pbmc3k')
163M 154M 0.95 sparse_dataset.SparseCSRContiguousSlice.peakmem_getitem((10000, 10000), '0:1000')
158M 161M 1.02 sparse_dataset.SparseCSRContiguousSlice.peakmem_getitem((10000, 10000), '0:9000')
163M 160M 0.99 sparse_dataset.SparseCSRContiguousSlice.peakmem_getitem((10000, 10000), ':9000:-1')
152M 161M 1.06 sparse_dataset.SparseCSRContiguousSlice.peakmem_getitem((10000, 10000), '::-2')
161M 165M 1.03 sparse_dataset.SparseCSRContiguousSlice.peakmem_getitem((10000, 10000), 'alternating')
152M 162M 1.07 sparse_dataset.SparseCSRContiguousSlice.peakmem_getitem((10000, 10000), 'arange')
159M 150M 0.95 sparse_dataset.SparseCSRContiguousSlice.peakmem_getitem((10000, 10000), 'array')
158M 157M 0.99 sparse_dataset.SparseCSRContiguousSlice.peakmem_getitem((10000, 10000), 'first')
150M 152M 1.01 sparse_dataset.SparseCSRContiguousSlice.peakmem_getitem_adata((10000, 10000), '0:1000')
158M 161M 1.02 sparse_dataset.SparseCSRContiguousSlice.peakmem_getitem_adata((10000, 10000), '0:9000')
160M 156M 0.98 sparse_dataset.SparseCSRContiguousSlice.peakmem_getitem_adata((10000, 10000), ':9000:-1')
161M 154M 0.96 sparse_dataset.SparseCSRContiguousSlice.peakmem_getitem_adata((10000, 10000), '::-2')
158M 154M 0.97 sparse_dataset.SparseCSRContiguousSlice.peakmem_getitem_adata((10000, 10000), 'alternating')
161M 158M 0.98 sparse_dataset.SparseCSRContiguousSlice.peakmem_getitem_adata((10000, 10000), 'arange')
154M 155M 1 sparse_dataset.SparseCSRContiguousSlice.peakmem_getitem_adata((10000, 10000), 'array')
163M 159M 0.98 sparse_dataset.SparseCSRContiguousSlice.peakmem_getitem_adata((10000, 10000), 'first')
- 1.55±0.01ms 1.37±0.1ms 0.88 sparse_dataset.SparseCSRContiguousSlice.time_getitem((10000, 10000), '0:1000')
7.24±0.6ms 7.45±0.5ms 1.03 sparse_dataset.SparseCSRContiguousSlice.time_getitem((10000, 10000), '0:9000')
509±6ms 509±6ms 1 sparse_dataset.SparseCSRContiguousSlice.time_getitem((10000, 10000), ':9000:-1')
2.49±0s 2.52±0.02s 1.01 sparse_dataset.SparseCSRContiguousSlice.time_getitem((10000, 10000), '::-2')
526±5ms 529±3ms 1.01 sparse_dataset.SparseCSRContiguousSlice.time_getitem((10000, 10000), 'alternating')
513±3ms 509±3ms 0.99 sparse_dataset.SparseCSRContiguousSlice.time_getitem((10000, 10000), 'arange')
2.60±0.2ms 2.52±0.1ms 0.97 sparse_dataset.SparseCSRContiguousSlice.time_getitem((10000, 10000), 'array')
1.01±0.07ms 1.01±0.07ms 0.99 sparse_dataset.SparseCSRContiguousSlice.time_getitem((10000, 10000), 'first')
75.9±0.2μs 74.7±0.4μs 0.98 sparse_dataset.SparseCSRContiguousSlice.time_getitem_adata((10000, 10000), '0:1000')
76.2±0.9μs 75.9±0.3μs 1 sparse_dataset.SparseCSRContiguousSlice.time_getitem_adata((10000, 10000), '0:9000')
75.3±0.5μs 75.4±0.4μs 1 sparse_dataset.SparseCSRContiguousSlice.time_getitem_adata((10000, 10000), ':9000:-1')
76.3±0.8μs 75.1±0.1μs 0.98 sparse_dataset.SparseCSRContiguousSlice.time_getitem_adata((10000, 10000), '::-2')
268±0.8μs 284±20μs 1.06 sparse_dataset.SparseCSRContiguousSlice.time_getitem_adata((10000, 10000), 'alternating')
124±0.5μs 124±0.9μs 1 sparse_dataset.SparseCSRContiguousSlice.time_getitem_adata((10000, 10000), 'arange')
114±0.9μs 112±0.6μs 0.99 sparse_dataset.SparseCSRContiguousSlice.time_getitem_adata((10000, 10000), 'array')
77.8±0.3μs 77.9±0.5μs 1 sparse_dataset.SparseCSRContiguousSlice.time_getitem_adata((10000, 10000), 'first')
View more details on scverse-benchmark