Discussion_Limitations

Discussion & Limitations & Results

Systems biology studies generate many data files for the simulation of biological systems. These data files can be stored in COMBINE archives, standardized containers that facilitate the exchange of information from simulation studies [1], such as the fully-featured COMBINE archive described by Scharm & Waltemath [2].

COMBINE archive

As a result of this project, we built a fully-featured COMBINE archive ('blood, sweat and tears') of the Bachmann model [3] using the web platform CombineArchiveWeb. We used a similar folder structure as suggested by Scharm & Waltemath, which is presented in the section 'Structure of COMBINE archive'. We could upload all files from our repository, but report XML files for validations. On this step, the CobineArchiveWeb interface throws the following error: "Unknown Error: Cannot upload file". We have made a new issue in the GitHub repository from WebCAT (issue #146) but did not encounter the settlement of the problem during the course of the project.

After creating the COMBINE archive, we selected the option "Simulate" to automatically run the archive in SED-ML WebTools. This simulation shows the species and its behaviour in the reactions, but only from the default simulation. We have previously tested a simulation without the Default SED-ML file, which functioned too, showing the result of one of our simulations. This point is registered in another issue (#147) in the GitHub of the developer of CombineArchiveWeb. In addition, the COMBINE archive was downloaded and all files were successfully simulated in Tellurium v2.2.0 as described here.

Creation and simulation of SED-ML files

As part of this COMBINE archive we created SED-ML files to reproduce the simulation experiments presented by Bachmann et al. based on published models and information. Initially, we attempted to generate and simulate files in COPASI, a well-established tool for systems biology. While it allows to easily modify model parameters via the user interface, COPASI is limited in its functionality to support simulations using more than one model. This is, however, required in our case, for instance to run simulations in the wildtype and overexpression condition in parallel. We therefore changed our approach and eventually created and simulated SED-ML files using the Python platform Tellurium as described in our documentation.

Reproducibility of experiments

While we were able to re-create some (sub-) figures of the original paper using the model selected for this COMBINE archive, we encountered several problems. These will be addressed in separate GitHub issues for future reference (maybe for the authors of the original paper).

• Some parameters required to simulate specific experimental conditions were not included in the model (e.g. SHP1oe).
• Some model outputs are defined by observation functions and were not included as parameters in the model itself (e.g. pJAK2.au, tSTAT5). It probably would be possible to define these parameters either in the SBML file itself or in the experiment-specific SED-ML files based on information included in the supplementary material. However, we did not address this issue given the limited time frame of this project.
• Several experiments presented in the paper are based on testing a range of Epo levels. We could not fully reproduce these experiments as it was not clear how to simulate different Epo concentrations using the published model.
• The knockout of CIS and/or SOCS3 could not be simulated using the information provided in the paper.

References

[1] Bergmann, F.T. et al.. COMBINE archive and OMEX format: one file to share all information to reproduce a modeling project. BMC Bioinformatics 15, 369 (2014). https://doi.org/10.1186/s12859-014-0369-z
[2] Scharm, M. and Waltemath, D. A fully featured COMBINE archive of a simulation study on syncytial mitotic cycles in Drosophila embryos. F1000Research 5, 2421 (2016). https://doi.org/10.12688/f1000research.9379.1
[3] Bachmann, J. et al. Division of labor by dual feedback regulators controls JAK2/STAT5 signaling over broad ligand range. Molecular Systems Biology 7, 516 (2011). https://doi.org/10.1038/msb.2011.50