GSoC: Changing world map over time in Grapher

[larsyencken]

DIFFICULTY: HARD, SIZE: 350 HOURS

Project

In most topics we cover, we try to find the longest-run series that help us see the big changes over time. Often these extend several decades in the past and include entities that no longer exist. For example, our meat supply per person chart contains Czechoslovakia, Yugoslavia and USSR, which cannot be plotted on a map of the same data.

We would like to be able to create maps that have changing borders over time – as you move the timeline backwards and forwards, you would see different country borders.

It is likely a difficult task to find the “right” country borders, and something we will be tweaking over time. The goal of the project is the technical implementation to make these changing-borders-over-time possible.

Required skills

• Javascript/Typescript
• React (bonus)

Expected outcomes

• Find, extract or create a reusable dataset of historical geographic entity shapes from 1900 onwards
• Find or create geo codes that align with the historical entity shapes
• Create a new grapher map visualisation which supports historical shape data
• Create an example visualisation in grapher that uses historical shape data (e.g. population, gdp per capita)
• Document the Grapher API required for creating historical charts, so that others can use your work

Potential mentors

• @danyx23
• @larsyencken
• @danielgavrilov

Resources

• https://historicalmapchart.net/
• http://old.thenmap.net/
• The CShapes project maintains a dataset of country outlines changing over time: https://cshapes.ethz.ch/

Some data in the Correlates of War project might be helpful.

[rishits321]

Hello developers,

I am Rishikesh Suryavanshi and I am currently pursuing a bachelor's in Information Technology at Pune Institute of Computer Technology.

Being a second-year student I have become confident and fairly experienced in Web development languages including Javascript, CSS, Bootstrap, React, and different frontend as well as backend frameworks. This project falls well in my current capabilities and I will be very excited to work on this.
The world map and countries boundaries are highly debated topics and will require a lot of research and testing different datasets and I am happily willing to devote my time this summer to this project. I am fairly good in Javascript and React but please let me know if any different software will be required.
I would like to connect with mentors for this project to guide me, I also want to get familiar with the codebase used in this organization so any important guidelines or tips will be much appreciated.

I look forward to hearing from you and further working on this project.

Thank you for reading,
Rishikesh

[danyx23]

Hi @rishits321!

Yes, this project would definitely have a smaller research aspect regarding the geo json country outlines to use. Maps are never neutral of course but we would like this project to find an upstream project that does a reasonable and sensible collection of country borders over time that follow a rough international consensus as used by institutions like the UN. Since we only need world maps at a rather coarse level we hope that such material can be found.

As for getting started it would be great if you can try to follow our (recently updated) documentation and try to build our grapher codebase and run the tests. Maybe there is also a small "good first issue" issue that you could try to tackle so we can see that you know how to make small contributions via Github.

It will also be helpful if you can sketch out in your own words what you think this project could be if you would work on it. Feel free to ask further questions here!

[BhCh7051]

Hello OWID community,

My name is Bharat Chandwani, and I am a computer science undergraduate at the Indian Institute of Information Technology, Gwalior. Right now, I have set up Grapher locally and opened some PR.
Regarding the proposal: Changing the world map over time in Grapher, I think the CShape Dataset would be a perfect starting point. However, I have a question.

How will you tweak the borders of any specific country in case of conflict? (I assume that the borders would be a GeoJson MultiPolygon).

Thank You
Bharat Chandwani

[danyx23]

Hi Bharat!

I think we would not want to change map outlines ourselves and instead rely fully on the CShape project. The only processing that I can see us doing is geometry simplification for shorter download times or maybe making sure that some very small islands etc would still be visible.

[toni-sharpe]

I came at this from the other direction, my sliders start at the beginning, so do OWIDs I think, so I started by creating a key 0 and adding all the countries into that, then as a POC I made three changes occur: Czechoslovakia/Czech Republic/Slovakia, Sudan/South Sudan, Georgia/Abzhakia.

Go to World Map - zoom to 3 and center on Tunisia - select "Internet Use" then select the time slider at 1988 and the arrow keys to move along through the years. See at '90, '11 and '15 the above countries split off.

I was going to go back to 1900, then I remembered The Cold War, WWII, the end of Colonialism and thought, not right now.

Screen-shots

Czech / Slovakia

Sudan / S. Sudan

Georgia / Abzhakia

Data structure

The data structure is converted using reduce into one structure for this map. Starting with the first year before the start year, then merging each one after that.

Looking at OWID maps here on :3030 they seem to load with the latest year selected but with a play button ready to roll through the years. This actually points me towards the sensible solution of doing this data build once using CShapes, manually updating, then selecting a sub-range for the map; and, probably deleting and adding rather than deep merging (so when 2015 is hit, element 63 is deleted then 63 and 224 are added) unlike my code.

CShapes is a lot more detailed (700+ entities) and more data, but not in order and not with any label or data centering. It's 26mb and draws 1.17mil lines (search ],[), mine is 127kb (with about 1/3rd items, so 380kb) and 30000 lines. Slightly smaller than the current path used in SVG @ OWID, but mine is off on a CDN, away from CI, away from page data.

A benefit of this structure is it's not too "codey" for a researcher. They can add a new year, use the codes and structure, and co-ordinates (copied probably)

{   "0": {   
       "1": { "countryBorder": [[ ... ]], ... "countryName": "Heard Is." },
... ... ... ... ...             
     "223": { "countryBorder": [[ ... ]], ... "countryName": "Niue" }
  },
  // CZECH REP/SLOVAKIA
  "1992": { // as we started with two in the original list, 43 is edited there and 150 becomes the new country
      "43": { "countryBorder": [[ ... ]], ... "countryName": "Czech Rep." },
     "150": { "countryBorder": [[ ... ]], ... "countryName": "Slovakia" }
  },
  // SUDAN/SOUTH SUDAN
  "2011": { 
     "156": { "countryBorder": [[ ... ]], ... "countryName": "S. Sudan" }, // South Sudan is alphabetically first in the original list
     "159": { "countryBorder": [[ ... ]], ... "countryName": "Sudan" }
  },
  // GEORGIA/ABZHAKIA
  "2015": {
      "63": { "countryBorder": [[ ... ]], ... "countryName": "Georgia" },
     "224": { "countryBorder": [[ ... ]], ... "countryName": "Abkhazia" } // This is the first new ID added to the list for a new country
  }
}