20240610 developer call notes

[tomwhite]

20240610

Pre-notes

• Notes from previous meeting

PRs

Issues

Discussions

Notes

Attendees

1. Jeff Hammerbacher
2. Tom White
3. Jerome Kelleher
4. Ben Jeffery
5. Rafal Wojdyla
6. Eric Czech

Discussion

JK: Paper is ready to go! Waiting for sign-off from Tim’s org.

What next?

JH: Good time to step back and reevaluate sgkit - e.g. do QC pipeline that works well from scratch. E.g. use JAX not Numba. Also Cubed.

TW: Cubed on HPC interest too: cubed-dev/cubed#467

JK: Implement part of bcftools view on VCF Zarr. Would help large biobanks a lot.

JH: Look at top of funnel process. We have a large API surface - may be a problem, so good to re-start sgkit from new codebase.

JH: Local alleles?

JK: Helpful in general, but not for GeL data

JK: Add more converters to bio2zarr. Plink (¾ done). BGEN too.

TW: Move VCF writing to another repo?

JK: Good question - depends what we all think.

JK: I’d like to work on the bio2zarr cli, but very overcommitted. Hope more people show up.

TW: I’d like to move Hypothesis VCF to its own repo and python package

JK: Does Hypothesis API change much?

RW: No, very stable

Zarr integrity

JH: Does TensorStore have a concept of transactions we could use?

TW: ArrayLake another approach.

New tech

JH: New array store: https://github.com/spiraldb/vortex

EC: Any experience of Mojo?

JH: Focus on PyTorch and JAX…