diff --git a/README.md b/README.md index 3768185..b65f807 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ - + # cytoviewer diff --git a/vignettes/cytoviewer.Rmd b/vignettes/cytoviewer.Rmd index 8bc4899..1fb4afe 100644 --- a/vignettes/cytoviewer.Rmd +++ b/vignettes/cytoviewer.Rmd @@ -1,5 +1,5 @@ --- -title: "Interactive multi-channel image visualization in R" +title: "cytoviewer - Interactive multi-channel image visualization in R" date: "`r BiocStyle::doc_date()`" package: "`r BiocStyle::pkg_ver('cytoviewer')`" author: @@ -18,10 +18,10 @@ output: toc_float: yes bibliography: library.bib abstract: | - This R package supports interactive visualization of multi-channel images - and segmentation masks generated by imaging mass cytometry and other highly - multiplexed imaging techniques using shiny. The cytoviewer interface is - divided into image-level (Composite and Channels) and cell-level + This R/Bioconductor package supports interactive visualization of multi-channel + images and segmentation masks generated by imaging mass cytometry and other + highly multiplexed imaging techniques using shiny. The cytoviewer interface + is divided into image-level (Composite and Channels) and cell-level visualization (Masks). It allows users to overlay individual images with segmentation masks, integrates well with SingleCellExperiment and SpatialExperiment objects for metadata visualization and @@ -41,6 +41,11 @@ knitr::opts_chunk$set(error=FALSE, warning=FALSE, message=FALSE, library(BiocStyle) ``` +`r fontawesome::fa(name = "github", fill = "#333")` \@lassedochreden +`r fontawesome::fa(name = "github", fill = "#333")` \@nilseling + +![](imgs/cytoviewer_sticker.png) + ```{r library, echo=FALSE} library(cytoviewer) ``` @@ -54,7 +59,7 @@ highly multiplexed imaging techniques can be interactively visualized and explored. The `cytoviewer` package builds on top of the `r Biocpkg("cytomapper")` -Bioconductor package and extends the static visualization strategies +Bioconductor package [@Eling2020] and extends the static visualization strategies provided by `cytomapper` via an **interactive Shiny application**. The `cytoviewer` package leverages the image handling, analysis and visualization strategies provided by the `r Biocpkg("EBImage")` @@ -65,6 +70,8 @@ data. In addition, building up on `r Biocpkg("SingleCellExperiment")`, classes, the `cytoviewer` package integrates into the Bioconductor framework for single-cell and image analysis. +Read the **pre-print** [here](https://doi.org/10.1101/2023.05.24.542115). + ## Highly multiplexed imaging Highly multiplexed imaging allows simultaneous spatially and single-cell @@ -125,7 +132,40 @@ The `cytoviewer` interface is broadly divided into [cell-level](#CellLevel) visualization (Masks). It allows users to overlay individual images with segmentation masks, integrates well with `SingleCellExperiment` and `SpatialExperiment` objects for metadata -visualization and supports [image downloads](#Download). +visualization and supports [image downloads](#Download) **(Figure 2B)**. + +![**Figure 1: cytoviewer interface and functionality.** **(A)** The +supported functionality (right) of *cytoviewer* depends on the data +inputs (left). To match information between the objects, cell (cell_id) +and image (img_id) identifiers can be provided. SCE/SPE = +*SingleCellExperiment*/*SpatialExperiment*. **(B)** The graphical user +interface of *cytoviewer* is divided into a body, header, and sidebar. +The body of *cytoviewer* includes the image viewer, which has three +tabs: Composite (Image-level), Channels (Image-level), and Mask +(Cell-level). Zooming is supported for Composite and Mask tabs. The +package version, R session information, help page, and a drop-down menu +for image downloads are located in the header. The sidebar menu has +controls for sample selection, image visualization, mask visualization, +and general settings. Scale bar: 150 µm **(C)** *cytoviewer* supports +different viewing modes. Top: The "channels" tab of image-level +visualization displays individual channels. Shown are Ecad (magenta), +CD8a (cyan), and CD68 (yellow) marking tumor cells, CD8+ T cells, and +myeloid cells, respectively. Center: The "composite" tab of image-level +visualization visualizes image composites combining multiple channels. +These composite images can be overlayed with cell outlines, which can be +colored by cell-specific metadata. Shown here are cell outlines colored +by cell area (continous value) and cell type (categorical value; tumor +cells in white). Channel color settings are as follows for all markers: +Contrast: 2,5; Brightness: 1; Gamma: 1.2. Bottom: The "mask" tab can be +used to visualize segmentation masks that can be colored by +cell-specific metadata. Shown here are segmentation masks colored by +cell area (continuous) and cell type (categorical; tumor cells in +magenta). Scale bars: 150 µm. **(D)** "Image appearance" controls can be +used to add legends or titles and to change the scale bar length for +image-level (top) and cell level (bottom) visualization. The cell-level +mask plot depicts tumor (magenta), myeloid (yellow), and CD8+ T cells +(cyan). Scale bars: 100 µm. Adapted from +[@Meyer2023]](imgs/cytoviewer_overview.png) ### Data input format {#DataInputFormat} @@ -164,6 +204,16 @@ can be specified: contains the cell identifiers. These should be integer values corresponding to pixel-values in the segmentation masks. +For more detailed information on the input objects, +please refer to the respective documentation (e.g. the vignettes of +the `r Biocpkg("cytomapper")` or `r Biocpkg("SingleCellExperiment")`/ +`r Biocpkg("SpatialExperiment")` packages). + +In the [Read in data](#ReadData) section, we provide +example code to directly read in images and masks (e.g. in .tiff format) +into a `CytoImageList` object and create a `SingleCellExperiment` object +from them, which we can then visualize with `cytoviewer`. + ### Data input variations {#DataInputVariation} The functionality of `cytoviewer` depends on which input objects are @@ -248,6 +298,11 @@ For more detailed information on the dataset, please refer to the respective documentation (e.g. via `?pancreasImages` or the vignette of the `r Biocpkg("cytomapper")` package). +We also provide example code to directly read in images and masks +(e.g. in .tiff format) into a `CytoImageList` object and create a +`SingleCellExperiment` object from them in the +[Read in data](#ReadData) section. + ```{r dataset} # Load example datasets library(cytomapper) @@ -296,25 +351,32 @@ The **Header** includes package version information, access to session information and the help page as well as a dropdown-menu for [image downloads](#Download). -The **Body** features a Tabset-Panel layout allowing the user to switch between -three image modes: [Image-level](#ImageLevel) (Composite and Channels) -and [Cell-level](#CellLevel) (Mask). Furthermore, the Composite and Mask -tabs have zoom controls. +The **Body** features a Tabset-Panel layout allowing the user to switch +between three image modes: [Image-level](#ImageLevel) (Composite and +Channels) and [Cell-level](#CellLevel) (Mask). Furthermore, the +Composite and Mask tabs have zoom controls. -The **Sidebar** panel is subdivided into four sections: *Sample selection*, -*Image-level*, *Cell-level* and [*General*](#General) +The **Sidebar** panel is subdivided into four sections: *Sample +selection*, *Image-level*, *Cell-level* and [*General*](#General) controls. -![cytoviewer_interface](cytoviewer_vignette.png) - ## Image-level visualization {#ImageLevel} -Image visualization control is split into *basic* and *advanced controls*. +Image visualization control is split into *basic* and *advanced +controls*. *Basic controls* supports the selection of up to six markers/channels for `image` display. Each marker has color control settings that allow the user to set contrast, brightness, gamma and select a channel color. +![**Figure 2: Image-level visualization - Basic controls**. The graphical user +interface of cytoviewer for image-level-composite with basic controls. For +image-level visualization, Ecad (magenta), CD8a (cyan) and CD68 (yellow) +marking tumor cells, CD8+ T cells and myeloid cells, respectively, are shown +and channel color settings are as follows for all markers: Contrast: 2,5; +Brightness: 1; Gamma: 1.2. Note that the Composite tab is zoomable. +Scale bars: 150 µm. Adapted from [@Meyer2023]](imgs/cytoviewer_image_basic.png) + In the *advanced controls* part, the user can choose to overlay the displayed images with provided segmentation `masks`. Outline color and mask thickness can be adjusted by the user. Moreover, the masks can be @@ -325,6 +387,14 @@ Of note, for categorical and continuous metadata entries the user can choose between discrete colors and continuous color palettes (viridis, inferno, plasma), respectively. +![**Figure 3: Image-level visualization - Advanced controls**. The graphical user +interface of cytoviewer for image-level-composite with advanced controls. For +image-level visualization, Ecad (magenta), CD8a (cyan) and CD68 (yellow) +marking tumor cells, CD8+ T cells and myeloid cells, respectively, are shown +and channel color settings are as follows for all markers: Contrast: 2,5; +Brightness: 1; Gamma: 1.2. Note that the Composite tab is zoomable. +Scale bars: 150 µm. Adapted from [@Meyer2023]](imgs/cytoviewer_image_advanced.png) + ## Cell-level visualization {#CellLevel} Cell visualization has *basic controls*. @@ -337,10 +407,15 @@ Please note again that for categorical and continuous metadata entries the user can choose between discrete colors and continuous color palettes (viridis, inferno, plasma), respectively. +![**Figure 4: Cell-level visualization - Basic controls**. The graphical user +interface of cytoviewer for cell-level-mask with basic controls. For cell-level +visualization, tumor cells (magenta) are highlighted. Note that the Mask tab +is zoomable. Adapted from [@Meyer2023]](imgs/cytoviewer_cells.png) + ## General controls {#General} -General controls is subdivided into an *Image appearance* and -*Image filters* part. +General controls is subdivided into an *Image appearance* and *Image +filters* part. In the *Image appearance* section, the user can adjust the scale bar length and include legend/image titles, while the *Image filters* @@ -360,6 +435,100 @@ The user can specify a file name, select the image of interest clicking the download button, a pop-window should appear where the user can specify the download location. +# Additional Information + +## Read in data {#ReadData} + +To conveniently read in images and segmentation masks into a `CytoImageList` +object and then visualize these using `cytoviewer`, +the `cytomapper` package provides the `loadImages` function. + +The `loadImages` function returns a `CytoImageList` object containing the +multi-channel images or segmentation masks. Refer to the `?loadImages` function +to see the full functionality. + +As an example, we will read in multi-channel images and segmentation masks +provided by the `cytomapper` package. + +To correctly scale pixel values of the segmentation masks when reading them in, +we will need to set `as.is = TRUE`. Users needs to take care that pixel values +are scaled correctly in more complex cases. + +```{r read-in-images} +library(cytomapper) + +# Data directory that stores images and masks in tiff format +data_path <- system.file("extdata", package = "cytomapper") + +# Read in images +cur_images <- loadImages(data_path, pattern = "_imc.tiff") +cur_images + +# Read in masks +cur_masks <- loadImages(data_path, pattern = "_mask.tiff", as.is = TRUE) +cur_masks +``` + +### Add metadata + +To link images between the two `CytoImageList` objects and the corresponding +`SingleCellExperiment` object, the image ids need to be added to the +`elementMetadata` slot of the `CytoImageList` objects. + +```{r add-metadata} +names(cur_images) +names(cur_masks) +mcols(cur_masks)$ImageNb <- mcols(cur_images)$ImageNb <- c("E34", "G01", "J02") +``` + +### Add channel names + +To access the correct images in the multi-channel `CytoImageList` object, the +user needs to set the correct channel names. For this, the `cytomapper` package +provides the `?channelNames` getter and setter function: + +```{r channelNames-example} +channelNames(cur_images) <- c("H3", "CD99", "PIN", "CD8a", "CDH") +``` + +### Generating the object + +Based on the processed segmentation masks and multi-channel images, +`cytomapper` can be used to measure cell-specific intensities and morphological features. +Here, these features are stored in form of a `SingleCellExperiment` object. + +```{r measureObjects} +cur_sce <- measureObjects(image = cur_images, + mask = cur_masks, + img_id = "ImageNb") +cur_sce +``` + +### Run cytoviewer + +Next, we can again call `cytoviewer` with the generated `image`, `mask` and +`object` data and leverage all provided functionality. + +```{r cytoviewer-manual} +# Use cytoviewer with images, masks and object +app_1 <- cytoviewer(image = cur_images, + mask = cur_masks, + object = cur_sce, + img_id = "ImageNb", + cell_id = "object_id") + +if (interactive()) { + + shiny::runApp(app_1, launch.browser = TRUE) + + } +``` + +For more detailed information on the input objects, +please refer to the respective documentation (the vignettes of +the `r Biocpkg("cytomapper")` or `r Biocpkg("SingleCellExperiment")`/ +`r Biocpkg("SpatialExperiment")` packages). + # Session info {.unnumbered} ```{r sessionInfo, echo=FALSE} diff --git a/vignettes/cytoviewer_vignette.png b/vignettes/cytoviewer_vignette.png deleted file mode 100644 index 511715c..0000000 Binary files a/vignettes/cytoviewer_vignette.png and /dev/null differ diff --git a/vignettes/imgs/cytoviewer_cells.png b/vignettes/imgs/cytoviewer_cells.png new file mode 100644 index 0000000..22c7af2 Binary files /dev/null and b/vignettes/imgs/cytoviewer_cells.png differ diff --git a/vignettes/imgs/cytoviewer_image_advanced.png b/vignettes/imgs/cytoviewer_image_advanced.png new file mode 100644 index 0000000..f74c21a Binary files /dev/null and b/vignettes/imgs/cytoviewer_image_advanced.png differ diff --git a/vignettes/imgs/cytoviewer_image_basic.png b/vignettes/imgs/cytoviewer_image_basic.png new file mode 100644 index 0000000..3c1a9f4 Binary files /dev/null and b/vignettes/imgs/cytoviewer_image_basic.png differ diff --git a/vignettes/imgs/cytoviewer_overview.png b/vignettes/imgs/cytoviewer_overview.png new file mode 100644 index 0000000..3317d7b Binary files /dev/null and b/vignettes/imgs/cytoviewer_overview.png differ diff --git a/vignettes/cytoviewer_sticker.png b/vignettes/imgs/cytoviewer_sticker.png similarity index 100% rename from vignettes/cytoviewer_sticker.png rename to vignettes/imgs/cytoviewer_sticker.png diff --git a/vignettes/library.bib b/vignettes/library.bib index 1609792..63f8d51 100644 --- a/vignettes/library.bib +++ b/vignettes/library.bib @@ -61,4 +61,23 @@ @article{Angelo2014 Title = {Multiplexed ion beam imaging of human breast tumors}, Volume = {20}, Year = {2014} +} + +@article{Meyer2023, + year = {2023}, + author = {Lasse Meyer and Nils Eling and Bernd Bodenmiller}, + title = {cytoviewer: an R/Bioconductor package for interactive visualization and exploration of highly multiplexed imaging data}, + url = {https://doi.org/10.1101/2023.05.24.542115} +} + +@article{Eling2020, + doi = {10.1093/bioinformatics/btaa1061}, + url = {https://doi.org/10.1093/bioinformatics/btaa1061}, + year = {2020}, + volume = {36}, + pages = {5706–-5708}, + number = {24}, + author = {Eling, Nils and Damond, Nicolas and Hoch, Tobias and Bodenmiller, Bernd}, + title = {cytomapper: an R/Bioconductor package for visualization of highly multiplexed imaging data}, + journal = {Bioinformatics} } \ No newline at end of file