Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update heatmaps docs - fixation based #720

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions neon/pupil-cloud/visualizations/heatmap/index.md
Original file line number Diff line number Diff line change
@@ -1,38 +1,38 @@
# Heatmap

The output of the [Reference Image Mapper](/pupil-cloud/enrichments/reference-image-mapper/), [Marker Mapper](/pupil-cloud/enrichments/marker-mapper/), and [Manual Mapper](/pupil-cloud/enrichments/manual-mapper/) enrichments can be visualized as a traditional heatmap. This shows you which parts of your reference image or surface were gazed at more often by an observer.
The output of the [Reference Image Mapper](/pupil-cloud/enrichments/reference-image-mapper/), [Marker Mapper](/pupil-cloud/enrichments/marker-mapper/), and [Manual Mapper](/pupil-cloud/enrichments/manual-mapper/) enrichments can be visualized as a traditional heatmap. This shows you which parts of your reference image or surface were fixated more often by an observer.

For example, below and to the left is a view of a kitchen that was used as a reference image. On the right, you can see the output of the Heatmap Visualization for a recording that was made while the observer was preparing ingredients.

![An example of a heatmap from Pupil Cloud. On the left is a photo of a kitchen countertop. On the right is the same photo with a gaze heatmap overlayed.](heatmap_example.png)
![An example of a heatmap from Pupil Cloud. On the left is a photo of a kitchen countertop. On the right is the same photo with a fixation heatmap overlayed.](heatmap_example.png)

Check warning on line 7 in neon/pupil-cloud/visualizations/heatmap/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (countertop)

Check warning on line 7 in neon/pupil-cloud/visualizations/heatmap/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (overlayed)

The Heatmap is a Gaussian blurred 2D histogram of gaze data from all selected recordings. No normalization for recording time is performed, so longer recordings will carry more weight and contribute more to the Heatmap.
The Heatmap is a Gaussian blurred 2D histogram of fixation data from all selected recordings mapped to the reference image. No normalization for recording time is performed, so longer recordings will carry more weight and contribute more to the Heatmap.

The colors in the heatmap range from 0 to 100%, as indicated by the color bar to the right. A value of 0% means a point was never gazed at and 100% means it had the longest gaze duration.
The colors in the heatmap range from 0 to 100%, as indicated by the color bar to the right. A value of 0% means a point was never fixated and 100% means it had the longest fixation duration.

The output of the Heatmap Visualization is two image files: one image with just the Heatmap and another where it is overlayed on the reference image.

Check warning on line 13 in neon/pupil-cloud/visualizations/heatmap/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (overlayed)

:::: details Implementation Details

1. Compute the 2D histogram over the raw gaze data of all recordings. The histogram has the same aspect ratio as the reference image, with the wider side set to 300 bins:
1. Compute the 2D histogram over the raw fixation data of all recordings. The histogram has the same aspect ratio as the reference image, with the wider side set to 300 bins:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Compute the 2D histogram over the raw fixation data of all recordings. The histogram has the same aspect ratio as the reference image, with the wider side set to 300 bins:
1. Compute the 2D histogram over the fixation data of all selected recordings. The histogram has the same aspect ratio as the reference image, with the wider side set to 300 bins:


```
gaze_histogram2d = hist2d(gaze_x, gaze_y, bins=[nbins_x, nbins_y])
fixation_histogram2d = hist2d(fixation_x, fixation_y, bins=[nbins_x, nbins_y])

Check warning on line 20 in neon/pupil-cloud/visualizations/heatmap/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (nbins)

Check warning on line 20 in neon/pupil-cloud/visualizations/heatmap/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (nbins)
```

2. Apply a 2D Gaussian blur to the 2D histogram and normalize the resulting values to the maximum:

```
gaze_heatmap = GaussianBlur(gaze_hist2d, 0.01 * scale)
gaze_heatmap /= max(gaze_heatmap)
fixation_heatmap = GaussianBlur(fixation_histogram2d, 0.01 * scale)
fixation_heatmap /= max(fixation_heatmap)
```

3. Resize the Heatmap using Lanczos smoothing interpolation:

Check warning on line 30 in neon/pupil-cloud/visualizations/heatmap/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (Lanczos)

```
final_heatmap = resize(gaze_heatmap,
final_heatmap = resize(fixation_heatmap,
(ref_img_width, ref_img_height),
interpolation=LANCZOS)

Check warning on line 35 in neon/pupil-cloud/visualizations/heatmap/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (LANCZOS)
```

::::
::::
Loading