Visualizing Model Coefficients (Cell Painting Features)

[gwaybio]

There are a couple different ways to visualize model coefficients that are output from the cell health machine learning models. Currently, I visualize model coefficients based on model coefficients (per cell painting feature importance score) in aggregate across models (see #64).

This visualization scheme is not very informative. Instead, Rohban et al. 2017 describe several alternative methods. I will use this issue to document various methods and determine which one would work best for this project.

[gwaybio]

Strategies

Based on strategies in Rohban et al. 2017 Figure 4 (pasted below).

1. Rank features by model coefficients (importance scores)
2. Create a "Feature Grid"
   • This involves grouping features together based on the type of measurement
3. Create a "Feature Map"
   • Embedding method and then replace points with the actual name of the feature
4. Single cell clustering (k-means k = 20) and manually inspecting images in single cell profiles most different from controls
   • Identify which perturbations are overrepresented in the cluster
5. Additional creative ways are encouraged!
   • Must be informative and useful!

[gwaybio]

I will present model coefficients for every model in some sort of automatically generated figure layout

[shntnu]

In #123 @gwaygenomics showed an example of how the model can be visualized via cell images.

This comment is a bookmark so I remember to add feedback here vs the PR.

What is different between this use case and work by @mrohban in Rohban et al 2017 (shown in #97 (comment))?

1. Rohban et al. visualized single cells that had been "scored" (via clustering) vs here, where we are scoring the aggregated profile of the entire well, and showing some sample cells
2. Rohban et al. had just two categories - enriched and de-enriched - whereas here we have the spectrum.

Some notes

1. Given 1., it is not required (but not incorrect) to show crops of single cells, because that is no more informative than a field of view
2. Showing multiple levels is great for seeing gradual changes, but adds clutter. Aside: this was in part the inspiration for work by @mlafarge (in MIDL 2019) @pawni @goldsborough (in NIPS MLCB 2017)

[AnneCarpenter]

I believe I’ve made other comments about this cell view... possibly in a slide deck somewhere?

[gwaybio]

I believe I’ve made other comments about this cell view... possibly in a slide deck somewhere?

It's all in slack 😹 - I will copy here

Conversation

GW: is this informative at all? How should I be thinking about a figure like this? The ultimate goal is to show, visually, the model predictions (albeit made on aggregate profiles) in single cells to see if anything jumps out. (In this case we were looking for RNA localization at nucleolus)

DS: With so many image panels it's not immediately clear what I'm supposed to be looking at. It might benefit from some arrows or something demonstrating the key feature of interest, but alone I'm not sure there's an obvious trend as you move from left to right... Unless I'm totally misunderstanding it

DS: The dominating visual feature is cell density, so to the eye I think that distracts quite a lot

GW: this is helpful, thanks. I have next to zero experience looking at these types of figures. So it is very helpful to see how experts do and what ya’ll look for

BC: The cells in columns 1 and 4 are also dead- were these randomly sampled, or "most representative"?

GW: randomly sampled. also, cool to know those are dead cells. For a while I was trying to troubleshoot if my data structures were saving the wrong picture for certain channel/perturbations. it seems relatively straightforward to detect dead cells then... if we are interested in honing in on the "before-death" effects of perturbations, would it be worth filtering obviously dead cells from profile aggregation? (it wouldn't make sense if we think the death is the super important part of the profile) it also seems like viability-based phenotypes dominate after profile aggregation. I wonder if this would be the case if we remove dead cells. Maybe then more subtle phenotypes can get enough space to shine ☀️

BC: I think if possible, we want to show ideally the "most representative", not random.
It's definitely possible that removing dead cells would make the phenotype space clearer- AFAIK that's not an experiment we've ever tried.

AC: I assume you are showing a random cell from a sample that scored high vs low? Because I feel like I want to see a sampling at each score rather than just one cell. I’m on my phone but it’s hard to see much trend, though in combination with the info about which features and channels are important, it might be easier to know what to attend to. Can you score single cells?

GW:

I assume you are showing a random cell from a sample that scored high vs low?

Yep

Because I feel like I want to see a sampling at each score rather than just one cell.

👍 I figured

though in combination with the info about which features and channels are important, it might be easier to know what to attend to.

yeah! good point

Can you score single cells?

Yeah!!! This is the ultimate goal, as was proposed in aim 3 of my K99. I do think it is beyond the scope of this project, but it is definitely in the foreground of the background of my thinking

AC: ok! So if there is lots of heterogeneity then the one cell you choose for each score level could actually be a bad representative…. therefore showing a sampling of maybe 10 cells at each level would be valuable. But 10 cells x 5 channels is overwhelming. You could either show a pseudocolor overlay, or just show teh one most-important channel (if shape features are important, then show DNA for nucleus and AGP for cell/cytoplasm), with the other channels available in a secondary figure if a person wants to drill down.

[gwaybio]

FWIW, our strategy was used in Schiff et al. 2020 and was the paper's first citation:

To see if these best performing features held any mechanistic clues, we grouped them based on their type of measurement (e.g. shape, texture and intensity) and their origin by cellular compartment (cell, nucleus or cytoplasm) or image channel (DAPI, ER, RNA, AGP, and MITO)24. Such groupings resulted in features implicated in “area and shape”, “radial distribution” of signal within the RNA and AGP channels, and the “granularity” of signal in the mitochondria channel