Enhance Charts

[guerler]

This keeps track of chart related enhancements and ideas.

1. Add log-scale and axis min/max options for tabular visualizations

2. Anton's demo video demonstrating limitations which we have to fix in order to make histograms useful. Most important moments (time markers 1:31, 2:32, and 3:50):
   https://drive.google.com/file/d/0B7a9KfygumZFMTQ3cmppTmhsY3M/view?usp=sharing

3. Augment meta-data access within charts to allow for PDB-chain identification and other dataset related properties. Currently only tabular columns are resolved

[malloryfreeberg]

So I'm just going to throw this out there: Instead of investing time/energy/resources into improving built-in chart/plotting features in Galaxy, does it make more sense instead to invest in wrapping established plotting tools? I'm thinking specifically of R ggplots but I know there are others. I get a strong "reinventing the wheel" vibe when thinking about how to fix/enhance the Charts feature.

[bgruening]

@malloryfreeberg I guess this is exactly what Charts is made for. Charts is a plugin-system which can embed other interactive visualisations, like all BioJS components for example. Have a look at this one for example: #3601

This is not reinvented, it's just a wrapper around the original visualization, like XML files are for Galaxy tools. I guess we need to make it clearer why charts is a plugin system inside the plugin-system of Galaxy VIS - but as of now I would say that Charts is trying to integrate as many useful visualizations as possible.

Interesting point is that if you have more than >10k data points Charts can actually use ggplot or R to submit jobs to a cluster and get a image back.

[MoHeydarian]

I agree with @malloryfreeberg, having a commonly used plotting environment that produces publication quality plots is more valuable than being able to interactively visualize your data. Most users have figured out how to work with ggplot2, bringing ggplot2 into Galaxy would be a lifesaver.

This wouldn't have to replace Charts, it would be an alternative option that would provide the flexibility that Charts is lacking, as demonstrated from @nekrut's (excellently produced) video. This lack of flexibility majorly limits the visualization capabilities within Galaxy and forces users to take their data outside of Galaxy to visualize/plot, at which point the user loses reproducibility and transparency of their pipeline.

I've started wrapping a few plotting features of ggplot (scatterplot, histogram, and violin/box plot) and they work wonderfully on my local Galaxy instance and on Cloudman, providing the flexibility and quality that Charts doesn't have. My point is that we've already made some progress in getting an established, high-quality plotting environment into Galaxy that addresses the flexibility issues in the video above, why not bring ggplot2 into Galaxy and leave Charts for interactive applications.

Here is the link to the ggplot2 scatterplot wrapper in TTS: https://testtoolshed.g2.bx.psu.edu/repository?repository_id=b510b5f45928a733&changeset_revision=5a901c0cb006

[guerler]

I agree it would be very useful to make ggplot2 available, and if it solves the limitations in the video, we might not need to enhance charts. We have to keep an eye on not re-inventing the wheel. On the other hand, adding ggplot2 or any other visualizations does not interfere with what charts is attempting to do. Charts is basically a minimal tool form for strictly js-based, 3rd party visualizations. Limitations of charts are often the result of limitations set by the underlying plugins. Some of these limitations can be resolved with custom additions to these plugins. Nevertheless, there are so many js-based visualizations out there and it is good that we benefit from these developments. Here are some more examples #2875.

[jxtx]

These perspective represent opposite ends of the analysis process and both are important. Charts is meant to help with exploratory data analysis, when you are initially trying to understand new data, so interactivity is very helpful. Publication quality plotting is neccesary at the end, when you have figured out all your analyses and are preparing to publish. We definitely need more of the latter but I don't think it means the former is less useful.

[malloryfreeberg]

Ok ok, I'm convinced - we need Charts :) I guess it wasn't clear to me that Charts was meant to be more interactive/exploratory, but I 100% agree that this is important. If it works ;)

I also wonder about how Jupyter notebooks fit into all of this. I'll admit I don't have extensive experience using Jupyter inside Galaxy, but it seems that Jupyter should at least be considered when talking about Charts & interactive environments.

[bgruening]

@MoHeydarian if you submit this tool as WIP to the IUC will can give you a hand, this is a very important tool for workflow figures ... not the interactive ones.

[bgruening]

@MoHeydarian you might also look at these wrappers: https://github.com/galaxyproject/tools-iuc/blob/master/tools/vegan/vegan_diversity/vegan_diversity.xml#L17

They use the configfile feature of Galaxy so you save yourself from all the parameter parsing/transferring.

[FredericBGA]

Would it be possible to add the name of the file in the history as the default chart name (instead of New Chart ?)

[guerler]

Consolidated the enhancement suggestions here with #5423.