Mass Spectroscopy File Converter

Introduction:

The integration of the latest machines and technologies, combined with a diverse array of existing systems, expands the range of data formats and types, thereby introducing new challenges. To effectively navigate these complexities, it is crucial to implement robust governance mechanisms that ensure seamless conversion of diverse data formats into standardized forms.

The converter service is built on the Common Workflow Language (CWL), selected for its exceptional interoperability, portability, and reproducibility. The source code is housed in a GitLab repository, complete with a CI/CD pipeline that not only automates the generation of conversion and validation summary reports but also fosters community collaboration. These reports offer a detailed overview of the conversion and validation process, ensuring transparency and efficiency across various file formats.

The repository includes all the necessary code to run a converter, which is built on Flask, adheres to the OpenAPI specification, and provides a REST API for seamless integration with other NFDI4Chem services. The entire system is also packaged in a Docker image for easy deployment, and with a Helm chart available to streamline deployment in Kubernetes environments.

This repository is specifically designed to facilitate the conversion of mass spectrometry files from various formats into the mzML format.

Requirements:

• Python version 3.9 or higher
• docker Instructions to install docker
• cwltool Instructions to install cwltool

There are three methods for using the converter:

• Local Installation
• Docker Usage
• Kubernetes Deployment

Overview

Tab 1: Local Installation

1. Prerequisites: python 3.9 or higher
2. Clone the repository git clone https://git.rwth-aachen.de/linsherpa/ms_converter.git
3. Create a python virtual environment and activate it. Tips to create a python vitrual environment
4. Install necessary requirements pip install -r requirements.txt
5. Run the main file rest_api.py :

python rest_api.py

or

 flask --app rest_api.py run

1. The URL will be available on http://localhost:5000/api-docs#

Tab 2: Docker Usage

1. Pulling the docker image: docker pull docker pull lincoln1010/mass_spectrometry_file_converter:v1

2. Running the container: docker run --privileged -d -ti --name < name of the container > -p 5000:5000 lincoln1010/mass_spectrometry_file_converter:v1.0

3. Enabling the docker in docker (which is necessary): docker exec < name of the container > dockerd >/tmp/docker.stdout 2>/tmp/docker.stderr &

4. Copy extra parameters on config.txt file:

   • a : create a config.txt file with content , for example, --mzXML (without any indentation)
   • b : docker cp < path to your config.txt file > < name of the container >:/app/config.txt (copying the config.txt file to /app/config.txt inside the container)

5. The URL will be available on http://localhost:5000/api-docs# with RESTAPI

Tab 3: Kubernetes Deployment

The deployment of Kubernetes instance is done via [Helm chart](https://helm.sh/)

1. For deployment of helm chart of ms converter, please follow guidelines of Msconverter Helm Chart

MS-Converter Server:

If the server has been deployed correctly following the instructions, then the following image would be seen.

Mass Spectrometry File Converter

It consists of three functions:

a. msconvert:Conversion

The aim of conversion is to convert a format (in this case a Mass Spectrometry File) into a standardized (mzML) format.
The converter is based on proteowizard image
Input Files: It works for three types of input.

1. Folder with .D or .d extention:
   The folder needs to be converted to .tar extention with maintaining the name < name of folder >.d.tar
   
2. File with .RAW or .raw extention:
   The File can be directly used for conversion.
   
3. File that requires another secondary file i.e. .wiff and .wiff.scan extention.
   Both of the files need to be compressed together into .tar extention.
   

b. FileInfo:Validation

The input file for it is file with extention .mzML. The validation is based on FileInfo command from Openms The image used for validation is biocontainers/openms

c. FileConvert:Conversion

The input file for it is file with extention .mzML. The main use of this tool is to convert data from external sources to the formats used by OpenMS/TOPP. The validation is based on Fileconverter command from Openms. The image used for validation is the same biocontainers/openms

Acknowledgement:

Funded by the Deutsche Forschungsgesellschaft (DFG, German Research Foundation) under the National Resaerch Data Infrastructure – NFDI/1 – Project number 441958208