Skip to content

Pipeline for the European Variation Archive, implemented using Luigi from Spotify (https://github.com/spotify/luigi)

Notifications You must be signed in to change notification settings

jorizci/eva-pipeline

 
 

Repository files navigation

If you want to make the best use of the cluster, the pipeline must be launched 
in 3 steps.

1) Accessioning: A study must be run sequentially, so the file with chromosome 
1 is always processed before chromosome 2, and so on. But all chromosome 1 for 
different studies can be processed at the same time.

2) Transformation: This can be run in parallel without any conflicts, so feel 
free to launch as many files as possible simultaneously.

3) Loading: If a study is composed of a single file or split in chromosomes, 
then all its files can be processed in parallel.

About

Pipeline for the European Variation Archive, implemented using Luigi from Spotify (https://github.com/spotify/luigi)

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%