The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
- Template update 2.11 --> 2.14.1
- Input validation change based on template update
- Updated Cellpose, Mesmer and MultiQC
- Updated pipeline tests to account for Cellpose3 changes
- Introduced bug - new MultiQC does not read the custom files - fix ASAP
- updated Zenodo doi to cite all versions
- CUSTOMDUMPSOFTWAREVERSIONS based on template update
- mindagap_boxsize, mindagap_loopnum, mindagap_edges parameters
- updated mindagap version and adapted molkart.nf and modules.config to match updated mindagap module version
- CI testing with nf-test
- updated base.config to account for ilastik's high memory use
- updated metromap
- Fixed a bug in maskfilter for calculating the number of filtered objects.
- Changed naming of columns in multiqc table for filtered cells
- Updated version numbers for all local modules to molkart-local container v0.0.4 due to a bug in version v0.0.3 that prevented the modules to be properly run with singularity.
- Added createanndata process to workflow. This process will generate a spatial anndata object from the spot2cell output. The anndata object will be written to /anndata in the output folder.
- added tests for createanndata
- Updated version numbers for all local modules using the molkart-local container to v0.0.3
- spot2cell - removed tag, output name now required, output name defined in modules.config
- output documentation for create training subset
- formatting in local modules
Fixed parameter specification for Maskfilter in modules.config, where both min and max area params in python were passed as min_area.
- Changed file prefix used in CLAHE to prevent file naming collisions if user used dots in filenames
- Changed label for CLAHE and MASKFILTER to process_medium to make them pass first try on real life datasets
- Added config file for full test dataset
Crop overview is provided to Multiqc - now when create_training_subset is run, multiqc and customdumpsoftwareversions are also run.
- removed CropSummary.txt from published outputs - it gets collected at multiqc step and published there
- moved crop_overview.png to MultiQC folder
- gitpod container is nf-core/gitpod:dev instead of latest to include new versions of nf-tools and nf-test
- MOLKARTQCPNG process to add name to png for multiqc report, and combine if multiple samples are processed
Local module revamp - all should use the same Docker image to save space.
- renamed CREATEILASTIKTRAININGSUBSET to CROPHDF5
- renamed TIFFTRAININGSUBSET to CROPTIFF
- local modules now use the ghcr.io/schapirolabor/molkart-local:v0.0.1 container
- CREATE_STACK when clause - also applied the size check logic in molkart.nf
- Added crop_hdf5.py script instead of using mcmicro-ilastik container
- pattern to only return cropped images and overview (not versions or full hdf5 image)
- clahe does not use aicsimageio anymore
- create stack outputs a pyramidal tif (Palom)
- updated mesmer module - accordingly added prefix logic (and for maskfilter)
Added MASKFILTER module.
- MASKFILTER module with respective script, parameters, qc measures that are passed to MOLKARTQC and MULTIQC
- renamed molcart_qc to MOLKARTQC
- versions to main local modules (MOLKARTQC, SPOT2CELL)
- CREATE_STACK when clause (so that it does not show in the progress when it doesn't run)
- comments in molkart.nf for clarity
- collect_QC average area is now rounded
- prefix handling in some modules
- SAMPLESHEETCHECK subworkflow and Python script
Replaced local module for mindagap/duplicatefinder with nf-core module.
- installed mindagap/duplicatefinder via nf-core tools
- removed local mindagap_duplicatefinder.nf in local modules
Changes to clahe - more nf-core compliant, script change, versions, updated tests.
- Clahe now outputs versions
- --clahe_pyramid_tile parameter (hidden)
- clahe local module now follows nf-core guidelines with output naming defined through ext.prefix
- In all cases, the same writer will be used for clahe now
- Fixed CLAHE metadata
- renamed process from CLAHE_DASK to CLAHE
- renamed tilesize parameter to mindagap_tilesize for clarity
- clahe_skip_pyramid parameter
Fixed file naming schema for mindagap and spot2cell. If only mesmer is used for segmentation, create stack does not have to be run.
- Mindagap outputs, in case the filenames were the same, would overwrite each other.
- spot2cell outputs, in case the filenames and segmentation method were the same, would overwrite each other.
- removed hardcoded memory requirement for CREATEILASTIKTRAININGSUBSET
- if only mesmer is used for segmentation, create stack does not have to be run.
Added first nf-tests for the pipeline.
- nf-test for 3 runs
- main.nf where the input only has the nuclear channel (does not run clahe or ilastik)
- main.nf where the input has both nuclear and membrane image (runs clahe, does not run ilastik)
- main.nf where the input only has the nuclear channel (does not run clahe), creates training subset
Upgraded workflow, fixed multisample cellpose segmentation with custom model. Added options necessary to make testing work on small images.
- white background in metromap
- clahe_skip_pyramid parameter to skip pyramid generation in the clahe step - necessary for smaller data
- Cellpose custom model functions with multiple samples now.
Added documentation - usage.md and output.md
- usage.md documentation
- output.md documentation
- segmentation outputs are all moved to a segmentation folder.
- updated nf-core module versions
- CITATIONS.md updated
- README.md updated
- WorkflowMolkart.groovy updated to return citations if tools are used (added commas)
Implemented the tilesize parameter for Mindagap_mindagap and mindagap_duplicatefinder so that smaller representative images can be used as test.
- tilesize param
- tilesize passing to mindagap and duplicatefinder in modules.config
- Replace
PROJECT_SPOTSandMCQUANTmodules with spot2cells. This new (for now local) module reduces the RAM requirements drastically, because it doesn't create a multi-channel stack for the spots. Spots are assigned by looking up cell IDs at x,y, positions and iterating over the deduplicated spots table. - Added process labels to many modules to fix linting warnings
- Added meta map to molcart_qc output to remove linting warning -- adjusted script for multiqc input accordingly
- Added duplicated spots counts to collect_qc.py and multiqc_config.yml so that they also get counted.
- Added tag option to spot2cell so that the output names with same sample id and different segmentation methods can be differentiated (they were overwriting each other previously)
- removed project spots and mcquant from modules.config
- changed pattern for molcart_qc as it was not matching the files (removed {})
- added meta value to segmethod input in molcart_qc
- spot counts are now int values
- QC metrics rounded to 2 decimals
Replaced the clahe param with skip_clahe so that the default value for running CLAHE is False.
- skip_clahe param (default False)
- removed clahe param
- adjusted workflow to check the params.skip_clahe value instead of the params.clahe
- adjusted the ext.when in modules.config
Added barebones version of multiqc output.
- emit value for png overview for createtrainingtiff
- molcart-qc: added sampleid-segmentation tag as sample id, as multiqc was only showing the second row if sample id is same - can this be fixed to unique row?
- input for multiqc are the csv files produced by molcart qc
Molkart adapted to most nf-core standards with optional parameters, multiple segmentation options, as well as membrane channel handling. Started work on creating training subset functionality.
- parameters for pipeline execution
- ext.args logic for almost all modules with external parameters
- channel logic for membrane handling
- create stack process if membrane image present for Cellpose
- optional clahe
- started work on create subset functionality
Initial release of nf-core/molkart, created with the nf-core template.