slides.html

<!DOCTYPE html>
<html>
  <head>
    <title>Title</title>
    <meta charset="utf-8">
    <link rel="stylesheet" type="text/css" href="img/slides.css">
<!--
    <link rel="stylesheet" href="./img/font-awesome-4.7.0/css/font-awesome.min.css">
    <style>
      @import url(https://fonts.googleapis.com/css?family=Yanone+Kaffeesatz);
      @import url(https://fonts.googleapis.com/css?family=Droid+Serif:400,700,400italic);
      @import url(https://fonts.googleapis.com/css?family=Ubuntu+Mono:400,700,400italic);

      body { font-family: 'Droid Serif'; }
      h1, h2, h3 {
        font-family: 'Yanone Kaffeesatz';
        font-weight: normal;
      }
      .remark-code, .remark-inline-code { font-family: 'Ubuntu Mono'; }
      #slideshow .slide .content .cols.two .col { width: 48%; }
    </style>
-->
  <style>
   .section_background {background-color:#c2c444; color: #fff}
   .day_background {background-color:#ececec; color: #000}
  </style>
  <script type="text/x-mathjax-config">
    MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});
  </script>
  <script type="text/javascript" async
    src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/MathJax.js?config=TeX-MML-AM_CHTML">
  </script>
  </head>
  <body>
    <textarea id="source">

class: center, middle

# Python for GIS and Geoscience

Specialist course Doctoral schools of Ghent University,<br>
5-6-8 October 2020

Joris Van den Bossche, Stijn Van Hoey

https://github.com/jorisvandenbossche/DS-python-geospatial

---

### Joris Van den Bossche

<a href="https://twitter.com/jorisvdbossche"><i class="fa fa-twitter" aria-hidden="true"></i> jorisvdbossche</a>,
<a href="https://github.com/jorisvandenbossche"><i class="fa fa-github" aria-hidden="true"></i> jorisvandenbossche</a><br>

* Open source software developer and teacher
* Pandas, GeoPandas, scikit-learn, Apache Arrow

---

### Stijn Van Hoey

<a href="https://twitter.com/svanhoey"><i class="fa fa-twitter" aria-hidden="true"></i> SVanHoey</a>,
<a href="https://github.com/stijnvanhoey"><i class="fa fa-github" aria-hidden="true"></i> stijnvanhoey</a>

* Research software engineer

.center[
![:scale 90%](./img/work_stijn_1.png)]

---
class: middle, section_background

# Setting up a working environment

---

## Setting up a working environment

For the setup instructions, see the [setup page](https://jorisvandenbossche.github.io/DS-python-geospatial/setup.html).

---
class: left, middle

0. Everyone has conda installed and the environment setup? If not, see [1-install-python-and-the-required-python-packages](https://jorisvandenbossche.github.io/DS-python-geospatial/setup.html#1-install-python-and-the-required-python-packages)
1. Make sure to (re)download ALL the course material, see [2-getting-the-course-materials](https://jorisvandenbossche.github.io/DS-python-geospatial/setup.html#2-getting-the-course-materials) also if you already did this before. We had to update the material this morning...
2. Next, also do section 3 and 4 of the [setup](https://jorisvandenbossche.github.io/DS-python-geospatial/setup.html)
3. Log in to [Ufora](https://ufora.ugent.be/d2l/home/329089) and enter the course room of the '2020-10-05-morning-session' session

> If you succesfully done 0, 1, 2 and 3, give a thumbs up in the discord `#status` channel.

Next:

- Surf to and fill in [the questionnaire](https://hackmd.io/L9fDTVspSbG7BIBJAT2xLQ?both)
- In Jupyter Lab, start with the 'notebooks/00-jupyter_introduction.ipynb'.

Installation or setup issues? Post them on the discord `#installation-issues` channel

---
class: center, middle

When you see something like this...


![:scale 100%](./img/startup.png)

...relax, you're ready to start!


---
class: center, middle

![:scale 100%](https://i.ytimg.com/vi/PlaYMh-u-2w/maxresdefault.jpg)


---
class: center, middle

# Why python?

---
class: center, middle

# Python is...

---

## …WIDESPREAD
* Large world-wide community ensures **fast development**
* Used in **scientific** and non-scientific institutes <br> (Google, NASA, ESRI, VMM,...)
* Python is a standard in **GIS** <br> (commercial and non-commercial tools)
* **Multiplatform** (linux, windows,…)

---

## …REPRODUCIBLE

.center[

instead of

> *what did you/I clicked a few months ago?’-style WYSISYG*

]

--

## …MULTI-FUNCTIONAL

* Research and development: the python *scientific ecosystem*
* Webdevelopment (Django, Flask)
* Daily use: great as ducttape/glue
* **Automation** of time-consuming returning tasks

--

## …OPEN SOURCE
* No license costs!

---

## ...SO WHAT? The Holy Grail ? No!
* Commercial packages give specific support  and more direct help-functions (cfr. MATLAB)
* Commercial packages can be more client-friendly
* For hard number-crunching stuff, NOT the fastest solution
* R, Fortran, C++ etc are also ‘widespread’, so python is just a language amongst the others
* Risk of getting lost in the forest/flood of ‘useful packages’

---

# Conda

### Why using conda?

- Consistent package manager across Windows, Mac and Linux
- Many precompiled packages available
- Less problems with installation

--

### Why different environments?

- Manage the dependencies of a specific project/paper/group/...
- You can install different version of Python and other packages alongside on your computer
- Easily share environments with other

---
## Small overview of conda commands

Creating a new environment:

```
conda create -n my_env python=3.7 pandas

# or from environment file
conda env create -f environment.yml
```

Activating an environment:

```
conda activate my_env
```

Install a new package:

```
conda install matplotlib     # if not working, try: pip install ...
```


List all installed packages: `conda list`

List all your environments: `conda info -e`

See the docs: http://conda.pydata.org/docs/using/index.html

---
class: center, middle

### Keep track of your python ecosystem <br>with an  `environment.yml`
<br><br>

```
conda env export > environment.yml
```

---
class: middle, section_background

# Working with Python

---

## IPython console

<br>
.center[![:scale 75%](./img/ipython.png)]

---

## Interactive Development Environment (IDE)

* [**Spyder**](https://pythonhosted.org/spyder/) is shipped with Anaconda. The familiar environment for Matlab/Rstudio-users...
* [**PyCharm**](https://www.jetbrains.com/pycharm/): Popular for web-development and Django applications, powerful when doing 'real' development (packages, libraries, software)
* [Eclipse + **pyDev plugin**](http://www.pydev.org/): If you like working in Eclipse, just add the python environment
* [**VS Code**](https://code.visualstudio.com/), [**Atom**](https://atom.io/), ...

---

## Jupyter Notebook
<small>(*previously called IPython notebook*)</small>

**Jupyter notebook** provides an **interactive** scripting environment,<br> ideal for exploration, prototyping,...

.center[![:scale 70%](./img/notebook.png)]

--

 ...the stuff we're dealing with in this course!


---
class: middle, left

### Time is divided between

- group sessions: we explain new concepts (aka 'theory')
- break-out sessions: you work on exercises

In case of questions, remarks, suggestions, you can always post messages in the discord `#daily-channel`

### Status check

We will regularly ask for a check (ready with exercise, installation succesfull...). Use the discord `#status` channel to put 👍 (`thumbs up`).

### Feel lost?

Feel free to send a discord direct message to one of us, we are stand by.


---
class: middle, center

![:scale 80%](./img/issuetracker.png)

Report bugs, typo's, suggestions... as issues ([New issue](https://github.com/jorisvandenbossche/DS-python-geospatial/issues/new))

or see the [contributing guidelines](https://github.com/jorisvandenbossche/DS-python-geospatial/blob/master/CONTRIBUTING.md)

---
class: middle, center


## Let's start


---
class: middle, center


...


---
class: center, middle

# How are you feeling?

![:scale 100%](http://esq.h-cdn.co/assets/15/51/980x490/landscape-1450137389-john-cleese.JPG)

### https://forms.gle/PhxSAW768fTBqLJz8

Please fill in the questionnaire!


---
class: center, middle

# Closing notes

---
class: center, middle

# Python's scientific/GIS ecosystem


#### Inspired by figure by Jake VanderPlas

---
class: center, middle, bgheader
background-image: url(./img/ecosystem-gis-overview_6.svg)
background-size: cover

---
class: center, middle, bgheader
background-image: url(./img/ecosystem-gis-overview_5.svg)
background-size: cover

---
class: center, middle, bgheader
background-image: url(./img/ecosystem-gis-overview_3.svg)
background-size: cover

---
class: center, middle, bgheader
background-image: url(./img/ecosystem-gis-overview_2.svg)
background-size: cover

---
class: center, middle, bgheader
background-image: url(./img/ecosystem-gis-overview_1.svg)
background-size: cover

---
class: center, middle, bgheader
background-image: url(./img/ecosystem-gis-overview.svg)
background-size: cover

---

# A rich ecosystem of packages

<br>

![:scale 60%](https://pysal.org/assets/pysal_nav_logo.svg)

Python Spatial Analysis Library (https://pysal.org). Collection of packages for:

* detection of spatial clusters, hot-spots, and outliers
* construction of graphs from spatial data
* spatial regression and statistical modeling on geographically embedded networks
* spatial econometrics
* exploratory spatio-temporal data analysis
* ...


---

# A rich ecosystem of packages

<br>

Building up on / integrating with GeoPandas:

**MovingPandas** (https://anitagraser.github.io/movingpandas/): working with movement data

**Momepy** (http://docs.momepy.org/en/latest/): quantitative analysis of urban morphology

**geoplot** (https://residentmario.github.io/geoplot/):  high-level Python geospatial plotting library (combining GeoPandas, matplotlib and cartopy)

**...**

---

# A rich ecosystem of packages

<br>

Building up on / integrating with xarray:

**Satpy** (https://satpy.readthedocs.io/en/stable/): package for earth-observing satellite data processing

**xesmf** (https://xesmf.readthedocs.io/): Universal Regridder for Geospatial Data

**verde** (https://www.fatiando.org/verde/): interpolating spatial data on regular grids

**rioxarray** (https://github.com/corteva/rioxarray): linking rasterio functionality with xarray

**...**

---

# A rich ecosystem of packages

<br>

Several more domain/application specific packages:

**MetPy** (https://unidata.github.io/MetPy): working with weather data

**xgcm** (https://xgcm.readthedocs.io/en/latest/index.html): working with output from General Circulation Models

**osmnx** (https://github.com/gboeing/osmnx): street networks 

**...**


---


# A rich ecosystem of packages

<br>

**Machine learning**: scikit-learn, tensorflow, pytorch, keras, chainer, ...

**Performance**: Numba, Cython, Numexpr, Pythran, C/Fortran wrappers, ...

**Visualisation**: Bokeh, Seaborn, Plotnine, Altair, Plotly, Mayavi, HoloViews, datashader, vaex ...

**Data structures and parallel/distributed computation**: Xarray, Dask, Distributed, Cupy, ...

Specialized packages in many **scientific fields**: astronomy, natural language processing, image processing, geospatial, ...

**Packaging and distribution**: pip/wheels, conda, Anaconda, Canopy, ...

---
class: center, middle

### Reading advice

[Good Enough Practices in Scientific Computing](https://arxiv.org/pdf/1609.00037v1.pdf)

> "*However, while most scientists are careful to validate their laboratory and field equipment, most do not know how reliable their software is*"


---
class: center, middle

## Good luck!

![](http://www.backwaterreptiles.com/images/pythons/angolan-python-for-sale.jpg)


    </textarea>
<!--    <script src="https://gnab.github.io/remark/downloads/remark-latest.min.js">
</script>-->
    <script src="img/remark.min.js" type="text/javascript">
    </script>
    <script>
      var slideshow = remark.create();
    </script>
    <script>
        remark.macros.scale = function (percentage) {
          var url = this;
          return '<img src="' + url + '" style="width: ' + percentage + '" />';
      };
      var slideshow = remark.create();
    </script>
    <!-- Latest compiled and minified CSS -->
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
    <!-- jQuery library -->
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
    <!-- Latest compiled JavaScript -->
    <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js"></script>
  </body>
</html>