GSoC: Cartograms in Grapher

[larsyencken]

DIFFICULTY: HARD, SIZE: 350 HOURS

Overview

Our World in Data has the mission to educate about the world’s most important problems and one of the most important means of this mission is to use chloropleth maps to give readers a quick and intuitive overview of important indicators in all the countries of the world.

Unfortunately, this very useful and popular view suffers from the fact that the size of a country is of course unrelated to its population size. For many important problems, the size of the population happens to be one of the most important aspects of a problem: if malnutrition is high in a very large country then this is in many ways much worse than if malnutrition is high in a tiny country. But since human visual perception associates area with importance, choropleth maps are often a suboptimal tool for communicating the scale of various problems.

One alternative that would be very useful for the kind of communication we do is mosaic cartograms. This is a visualisation form that is a hybrid between a map and an area chart scaled by population. To help fast visual processing, standardised tiles corresponding to a fixed number of people are laid out in such a way that countries try to retain their shape but they are now sized to the number of tiles proportional to their population. This gives a quick visual reading that is proportional to the population of a country, but at the same time allows viewers to easily orient themselves by laying out countries roughly where they usually can be found on a map.

Creating a layout for the world's countries using the population for a single year can be done manually, leading to good results when it comes to resembling continent/country shapes. But to be truly useful for us we would need to be able to plot such cartograms for arbitrary years given our population dataset, which is why we are looking for an algorithm to create such maps. One complication of doing this for arbitrary years is that it would be desirable to try to minimise visual jumps between years as the population grows at different rates in different countries.

Expected outcomes

• Creation of an algorithm in a language of the participants choice to create for a given year the mapping of countries to tiles in a fixed raster using our standardised population dataset as an input.
• The tile system could either be composed of many small squares or of many small hexagons.
• A good outcome of the project would be an algorithm that can produce cartogram mappings that closely match human intuitions about continent and country shapes for arbitrary years.
• Svgs and json mapping files could then be pre-generated for many years and be available for our Grapher library. This generation step and the actual visualisation of the created cartograms would not be part of this project.

Related links:

• https://research.tue.nl/nl/publications/mosaic-drawings-and-cartograms
• https://pitchinteractiveinc.github.io/tilegrams/
• https://github.com/kittles/cartogrampy
• https://github.com/mattdzugan/World-Population-Cartogram

Potential mentors

• @danyx23

Feel free to ask any questions you like below!

[addu390]

@larsyencken @danyx23 Hi, I'm currently working on handling large spatial data (Thesis), and this project as a whole is intriguing. From what I understand, the gist of the project is to create data points/tiles for a given data set (Population is one such example) and plot a world map (Typically, an actual word map has variable density tiles across countries, and we flatten that while ensuring the visual representation of the map stays.)

I'm super interested in the proposal and would love to work on it.

Some of the questions I have:

1. Is this an independent project outside of owid-grapher?
2. The project's goal is to generate the output, which would be used to visualize using the Grapher library later on? (i.e., the algorithm takes the population dataset of any arbitrary year and returns a raster dataset containing cells or outline of the world map).
3. Lastly, this algorithm is not limited to Population, but rather to any given metric?

Edited

[addu390]

Hi @danyx23
Thank you so much for the answer. I'm working on a draft already; I'll send my first draft today, EOD. On the other hand, I do see much more future scope for the project, such as a GIF/Video of the map for time-series data that shows the transition.

And yes, I was looking at the reference (https://github.com/mattdzugan/World-Population-Cartogram)[https://github.com/mattdzugan/World-Population-Cartogram], I get the over the idea, and the algorithm should dissolve territories to the relative scale of populated based datasets; the data generated to plot the map in the above reference is the output of the algo.

Lastly, would it be possible to share a sample dataset for "our standardized population dataset" quoted in the description?

[danyx23]

Hi @addu390!

we are currently working on making our datasets easier to find and re-use. If you have worked with python than you can use our experimental owid-catalog-py library to fetch the dataset like so:

from owid import catalog
# fetch a data frame for a specific match over HTTPS
t = catalog.find_one("population", dataset="key_indicators", namespace="owid")

[addu390]

Perfect! Thank you, I'll try it out.
I was planning on using python for the project anyway; I could add support to use this library for the data input for the service I would write.

[addu390]

Hi 👋🏼
As planned:

• Introduction email sent ✅
• First Proposal Draft sent ✅
• Putting relevant content together (WIP): population-cartogram

I see the dataset contains: country, year, population, country's share of world pop.; Is it fair to assume that the data set from OWID will be converted to the cells and borders as mentioned here: output or do you expect the output to be of a specific format?

[addu390]

Hello 👋🏼
I feel a lot more confident to work on the project after spending time researching algorithm(s)

• Diffusion Method looks promising and viable for our use case.
• I wrote a few utility functions to transform the world map raster dataset - added a few more images on this in the proposal - section 4.3
• The approach I'm planning to take is in section 3.2.2 of the proposal.

PS: I'm just dropping in updates and questions when I come across or accomplish something - in this thread.

[adisidev]

Hiya @larsyencken and @danyx23 !

I just saw your proposal on Google summer of Code and was extremely excited to see cartograms on there. I am extremely familiar with cartograms; in fact, they’re pretty much all I work on. In your proposal above, you referenced a package, namely cartogrampy. It turns out, the package uses the Gastner-Newman cartogram algorithm. I actually currently work with Professor Gastner, the co-creator of said algorithm, and I have made significant contributions to the latest implementation of the algorithm at cartogram-cpp. Me and a few other students are currently trying to refine his algorithm in code; The aforementioned repository is a complete re-write of Professor Gastner’s improved algorithm, which was initially written in C, and somewhat unstructured. Thus, we transitioned to C++. Since I have written a significant portion of the code that makes up the repository, I am extremely familiar with how it works. In fact, I was quite inspired by your mosaic cartograms, and decided to quickly hotfix a new branch with an initial code draft for creating them(mosaic-cartogram). I was already able to produce some results that the average eye might find pleasing. The algorithm is not restricted to the world map (although that is probably what you’re most interested in), or to a particular metric (like population). One advantage to such a method is that we could apply whatever map projection on our starting map, and some may lead to more pleasing results than others. I noticed that you both are from Germany and Switzerland, and thus decided to quickly see what the algorithm could produce for those countries, as well as the world, at the moment. Here is the result:

Germany Original:

Germany Cartogram:

Switzerland Original:

Switzerland Cartogram:

World Original:

World Cartogram:

All of these images (the unprojected mosaic and the final cartogram) were produced by the branch I just committed to. Although it may look pleasing, there is still loads to be improved. In fact, the world map is current not entirely correct and has a relatively high area error (what the area should be vs. what it actually is). Further, smaller countries are not represented on the map at all, instead of being represented as triangles (as with your map). These are changes that I am currently working on anyway, since representing very small areas is a challenge on non-mosaic cartograms as well. Thus, I would love to work with you all over the summer to make something specific to your use (perhaps, even a fork of cartogram-cpp).

Other than cartograms, a main reason that I'd like to work with you, and as a Google Summer of Code contributor at all, is to become more familiar with the open source world. The aforementioned repository, although technically open source, is much more of a research project rather than something maintained by an open source foundation. Thus, I would like to get an inner look at how open source foundations truly function, and hopefully meet some wonderful people along the way. I'd love to chat, if you'd be open to a short zoom call!

[danyx23]

Hi @adisidev!

Thank you for your interest in contributing with us! It's great to hear that you already have experience with Cartograms! To participate as part of Google Summer of Code you have to write and submit your proposal until April 19 on the GSoC website. I'm afraid we don't have the capacity to have a zoom call before that date - if you send a proposal draft soon to [email protected] then I can try to have a look and give you feedback but I can't make any promises that I'll for sure be able to do this.

One note on the cartogram algorithm: I think that outline preservation is one of the most important properties of mosaic cartograms. In the algorithm you showed above it looks similar to these rubber sheet style maps that I personally find not very helpful. E.g. the outline of Australia looks nothing like Australia and it would be much more useful if a small group of pixels would try to form this same outline albeit with fewer pixels than to try to fit similar extents but with very thin strips of land like in your example. But of course these are details that we could always still tweak before the actual work begins.

I'm looking forward to your proposal!

[adisidev]

Thank you for writing back! I agree, outline preservation is definitely one area for improvement. Fortunately, for Australia, and any other island countries, this should be quite easy to achieve - as they don't have borders with any other countries, we could simply scale them down. For other countries, this would definitely be something we tweak and work more on. Further, I also do agree to what was said in the initial proposal: increasing the size of land masses (where people actually live), and decreasing the size of the oceans would be helpful (and may also allow us more space to play around with the pixels themselves. I'm going to try and send a proposal by later today; I do understand that the deadline is tight, so I'd completely understand that you can't promise feedback.

[adisidev]

I have a few updates I'd like to share and a few questions, @danyx23 ! The pressing questions I have at the moment are at the end of this comment.

I decided that it would be best if I work slightly more on the project before submitting a draft. One caveat mentioned in the proposal above was that we would want to minimise the visual shift between years. I realised that the method I proposed above provides a really nice way to ensure that: we could simply use the previous map as the input for the next map.

I did some cleaning on the Max Roser's GeoJSON for the world cartogram, and I think the results look quite nice. Below are maps of population data from 1960-2018. The maps show a steady increase in Africa's size, indicating that Africa's population grew faster relative to the rest of the world.

EDIT: ADDED GIF

1960

1980

2000

2018

Again, there's lots to be improved the attached maps are not really polished. However, having experimented more in depth, I'm reasonably confident that I'd be able to help out in the given timeline.

As far as the proposal goes, I have something ready. However, I am unsure on how I'd go about submitting it: GSOC says that we're allowed to iterate on our proposal as many times as we'd like before submitting it. However, I can't find a "submit" button on the GSOC homepage. Would submitting simply mean sending an email to [email protected] ? The reason I brought up iterations on the proposal is that if submitting simply means sending an email to the aforementioned address, does "iterating" simply mean sending multiple emails to the aforementioned email, where only the last one would be counted? That feels quite troublesome for the person handling the email address!

[danyx23]

Hi @adisidev! Excellent work! Submitting the proposal must happen via the GSoC website. This part is managed by google and if you only send it to us via email you will unfortunately not be eligible for a GSoC stipend! I don't know exactly where to do this but it has to be on the GSoC website where you log in with you user and then there must be a way to upload your proposal somewhere. Create a new discussion asking other GSoC applicants for help if you have trouble locating the upload section on the GSoC website. Looking forward to your final proposal!

[addu390]

Good to see more contributors to the project. @adisidev you will have to submit it on the GSoC portal. Head to: https://summerofcode.withgoogle.com/proposals; search for the organization and upload the proposal (it's a form).
And right on! to use Max Roser's cartogram by mattdzugan as the reference.

All the best :)