This lesson is still being designed and assembled (Pre-Alpha version)

previous episode

Interactive Maps in the Jupyter Notebook

lesson home

Managing vector data with `ipyleaflet`

Overview

Teaching: 0 min
Exercises: 0 min
Questions

  • How to add Markers to your map?

  • What kind of Markers could I use?

  • What is a Choropleth map?

  • How to generate Choropleth map with ipyleaflet?

Objectives

  • Understand how to add markers, circle on an ipyleaflet map

  • Understand Chorophleth maps with ipyleaflet

Add Markers, circles to an ipyleaflet map

Let’s take a concrete example. We would like to reproduce the map of all Carpentries instructors.

Source: https://github.com/carpentries/carpentries.org/gh-pages/files/geojson/all_instructors_by_airport.geojson

We briefly mentioned it at the very beginning of the lesson but Github renders geojson file directly. However, here we wish to cutomize this plot, so we will first learn how to add markers in ipyleaflet.

View a map of all Carpentries instructors

Let’s first create a map, centred on Manchester (UK).

from ipyleaflet import Map, basemaps


# Define map centred on Manchester for CarpentryConnect 2019


# First is latitude and second is longitude; both in degrees
center = (53.483959, -2.244644)

map = Map(center=center, interpolation='nearest', basemap=basemaps.Stamen.Terrain)

Get geojson file from Carpentries website

import urllib.request

url = 'https://raw.githubusercontent.com/carpentries/carpentries.org/gh-pages/files/geojson/all_instructors_by_airport.geojson'

# Download the file from `url` and save it locally under `data/all_instructors_by_airport.json`:
urllib.request.urlretrieve(url, 'data/all_instructors_by_airport.geojson')

Read GeoJson file

The purpose of this episode is not to learn about geojson data format but we still need to be able to read the data file to add it into our map. For this, we will be using another python package called json. It can be installed from Anaconda conda-forge channel.

import json
# Open GeoJson file and load data
with open('data/all_instructors_by_airport.geojson') as f:
    geojson = json.load(f)

Visualize geojson as a table

Let’s have a look at our dataset:

from pandas.io.json import json_normalize

features = geojson['features']
json_normalize(features)

To use the function json_normalize, pandas python package is required. It should be available by default, and is available from Anaconda conda-forge channel.

geometry.coordinates geometry.type properties.details properties.marker-color type
0 [10.6190004, 56.2999992] Point [BAB, LWJ, AS, KT] #2b3990 Feature
1 [-106.616704, 35.048665] Point [KB, AF, JMG, CJ, NM, TR, MS, JW] #2b3990 Feature
2 [38.7993011, 8.97789] Point [YA, MDC, YAE, DYM] #2b3990 Feature
3 [-2.8684399, 56.3728981] Point [IB, AK, PM, MT] #2b3990 Feature
4 [174.791667, -37.008056] Point [VF, EJ, NJ, CMJT, CM, SS] #2b3990 Feature
5 [-73.8016968, 42.7482986] Point [DT] #2b3990 Feature
6 [4.767438, 52.309686] Point [KdH, MG, JH, VK, MK, PL, CMO, AS, RS] #2b3990 Feature
7 [17.918611, 59.651944] Point [LDS, HG, WNkerstrm, OV, RVG, TW] #2b3990 Feature
8 [-97.669722, 30.194444] Point [AD, VP, MS, MS, RT] #2b3990 Feature
9 [2.078333, 41.296944] Point [AOC] #2b3990 Feature
10 [-72.683056, 41.938889] Point [SBA, CD, JD, SK, AL, JM, TEM, SN, PN, SKO, RP... #2b3990 Feature
11 [26.3024006, -29.0926991] Point [FR] #2b3990 Feature
12 [-68.8281021, 44.8073997] Point [SM] #2b3990 Feature
13 [-86.753333, 33.562778] Point [VPL] #2b3990 Feature
14 [-1.747778, 52.453611] Point [SB, MB, MH, AJ, SM, CS, LDW] #2b3990 Feature
15 [-101.479347, 20.990772] Point [NS] #2b3990 Feature
16 [11.289168, 44.536189] Point [GP] #2b3990 Feature
17 [-86.66945, 36.131312] Point [SB, CF, JT] #2b3990 Feature
18 [153.1175, -27.384167] Point [IB, MF, FG, SG, NH, PAM, AM, BM, KP, BW] #2b3990 Feature
19 [-116.2229996, 43.5643997] Point [MC, MH, EJ] #2b3990 Feature
20 [72.867897, 19.0886993] Point [SS] #2b3990 Feature
21 [-71.005, 42.364167] Point [TB, DB, MF, JG, IG, KL, YL, EM, CM, MP, PR, S... #2b3990 Feature
22 [-2.718889, 51.3825] Point [CE, EH, DM, JM, DRB] #2b3990 Feature
23 [4.487055, 50.902447] Point [LG, TOG, SVH] #2b3990 Feature
24 [7.530066, 47.595156] Point [BB, GF] #2b3990 Feature
25 [-73.1532974, 44.4719009] Point [SGC] #2b3990 Feature
26 [-78.7322006, 42.9404984] Point [BTT] #2b3990 Feature
27 [-118.3590012, 34.2006989] Point [GC, LS] #2b3990 Feature
28 [-76.668333, 39.175278] Point [DK, PLL, DS] #2b3990 Feature
29 [-111.1529999, 45.7775002] Point [SM, AST] #2b3990 Feature
... ... ... ... ... ...
163 [-76.106111, 43.111111] Point [EM] #2b3990 Feature
164 [-84.34527, 30.39537] Point [JK, DP] #2b3990 Feature
165 [24.80074, 59.416582] Point [DF, LK, ES] #2b3990 Feature
166 [34.876667, 32.009444] Point [BG] #2b3990 Feature
167 [18.916328, 69.681079] Point [RB] #2b3990 Feature
168 [-82.536413, 27.983099] Point [JM, KR] #2b3990 Feature
169 [13.466389, 45.827778] Point [SC] #2b3990 Feature
170 [146.766907, -19.248454] Point [DB] #2b3990 Feature
171 [-110.941389, 32.116111] Point [GA, UKD, BJ, JO, TLS] #2b3990 Feature
172 [13.2876997, 52.5597] Point [CCB, SC, TZ] #2b3990 Feature
173 [-84, 35.81] Point [DEB, AS] #2b3990 Feature
174 [25.28217, 54.640024] Point [MRC] #2b3990 Feature
175 [20.9671001, 52.165699] Point [ZJS, AP] #2b3990 Feature
176 [174.808049, -41.328932] Point [JdL, WH, JW] #2b3990 Feature
177 [-117.636366, 49.296744] Point [JH] #2b3990 Feature
178 [-113.579722, 53.309722] Point [CM, JS, AS] #2b3990 Feature
179 [-66.527212, 45.871104] Point [JB] #2b3990 Feature
180 [-76.595936, 44.226524] Point [GL] #2b3990 Feature
181 [-63.508611, 44.880833] Point [RMD, RD] #2b3990 Feature
182 [-75.669167, 45.3225] Point [CA, RC, KC, CM] #2b3990 Feature
183 [-71.3975, 46.788333] Point [MB] #2b3990 Feature
184 [-82.9448, 42.2749] Point [MW] #2b3990 Feature
185 [-73.741389, 45.468056] Point [GAD, JG, DH, MJ, SR, PLSO] #2b3990 Feature
186 [-123.181944, 49.195] Point [MHB, DL, MR, AR, YT, TT, LJW] #2b3990 Feature
187 [-81.153889, 43.035556] Point [PB] #2b3990 Feature
188 [-114.012405, 51.128778] Point [PP, AS] #2b3990 Feature
189 [-123.425833, 48.646944] Point [AT] #2b3990 Feature
190 [-52.8124826, 47.6211888] Point [IAA, OC, EG, DQ, OS] #2b3990 Feature
191 [-79.630556, 43.677222] Point [SA, BB, MBF, BC, SEC, KC, HG, JG, TG, ARH, DH... #2b3990 Feature
192 [8.560132, 47.457767] Point [LD, SH, DLT, BMM, FP, SP, MR, FT, TT, LW] #2b3990 Feature

193 rows × 5 columns

Add our json data as a new layer to the map

ipyleaflet (as many similar interactive map packages) can handle geojson data format using GeoJSON:

from ipyleaflet import GeoJSON

geo = GeoJSON(data=geojson)
map.add_layer(geo)

Save map as HTML

As before with raster layers, we can save the resulting map in an HTML file:

from ipywidgets.embed import embed_minimal_html

embed_minimal_html('carpentries_instructors_basic.html', views=[map], title='The Carpentries Instructors')

The map is not exactly similar to what is shown in Github yet.

Add a pop

We added all the instructors locations (nearest airport) on our interactive map but it would be nice to add labels (using available information such as list of instructors):

from ipyleaflet import GeoJSON, Marker
from ipywidgets import HTML

features = geojson['features']
for i in range(len(features)):
    location=(features[i]['geometry']['coordinates'][1],features[i]['geometry']['coordinates'][0])
    instructors = features[i]['properties']['details']
    html = """
    <p>
      <h4><b>Instructors</b>:        """ + " ".join(instructors) + """</h4>
    </p>
    """
    marker = Marker(location=location)

    # Popup associated to a layer
    marker.popup = HTML(html)
    map.add_layer(marker)

Customize markers

# First is latitude and second is longitude; both in degrees
center = (53.483959, -2.244644)

map = Map(center=center, interpolation='nearest', basemap=basemaps.Stamen.Terrain)
map

And now we will be using the Carpentries logo 

from ipyleaflet import Icon

features = geojson['features']
for i in range(len(features)):
    location=(features[i]['geometry']['coordinates'][1],features[i]['geometry']['coordinates'][0])
    instructors = features[i]['properties']['details']
    html = """
    <p>
      <h4><b>Instructors</b>:        """ + " ".join(instructors) + """</h4>
    </p>
    """
    icon = Icon(icon_url='https://github.com/carpentries/carpentries.org/raw/gh-pages/assets/img/Badge_Carpentries.png', 
                icon_size=[22, 22])
    marker = Marker(location=location, icon=icon)

    # Popup associated to a layer
    marker.popup = HTML(html)
    map.add_layer(marker)

Marker cluster


from ipyleaflet import Icon, Marker, MarkerCluster

# First is latitude and second is longitude; both in degrees
center = (53.483959, -2.244644)

zoom = 2
map = Map(center=center, interpolation='nearest', zoom = zoom, basemap=basemaps.Stamen.Terrain)

markers = ()
features = geojson['features']
for i in range(len(features)):
    location=(features[i]['geometry']['coordinates'][1],features[i]['geometry']['coordinates'][0])
    instructors = features[i]['properties']['details']
    html = """
    <p>
      <h4><b>Instructors</b>:        """ + " ".join(instructors) + """</h4>
    </p>
    """
    marker = Marker(location=location)

    markers = markers + (marker,)
    # Popup associated to a layer
    marker.popup = HTML(html)

map.add_layer(MarkerCluster(markers = markers, name='Instructors'))

Control map layers

from ipyleaflet import LayersControl
map.add_control(LayersControl())

Circle marker

The main advantage is that

We will be using it to visualize the Carpentries instructors:

from ipyleaflet import Icon, CircleMarker

# First is latitude and second is longitude; both in degrees
center = (53.483959, -2.244644)

zoom = 2
map = Map(center=center, interpolation='nearest', zoom = zoom, basemap=basemaps.Stamen.Terrain)

features = geojson['features']
for i in range(len(features)):
    location=(features[i]['geometry']['coordinates'][1],features[i]['geometry']['coordinates'][0])
    instructors = features[i]['properties']['details']
    html = """
    <p>
      <h4><b>Instructors</b>:        """ + " ".join(instructors) + """</h4>
    </p>
    """
    # set color from the number of instructors per location
    if len(instructors) == 1:
        color = "blue"
    elif len(instructors) < 10:
        color = "orange"
    else:
        color = "red"
    marker = CircleMarker(location=location, radius = len(instructors), 
                          color="darkblue", fill_color=color, 
                          fill_opacity=0.8, opacity=0.6)

    # Popup associated to a layer
    marker.popup = HTML(html)
    map.add_layer(marker)

Choropleth maps

A choropleth map is a type of thematic map in which areas (in our example the areas will be countries) are displayed with different colors, shades or patterns which are defined as a function of the plotted parameter. This parameter can be for instance the population density or any statistical quantity, and in our case the color will depend on the temperature.

from rasterstats import zonal_stats
import geopandas as gpd
import pandas as pd
import json

country_file = gpd.datasets.get_path('naturalearth_lowres')
zs=zonal_stats(country_file, "data/t2m_ERA5_25122018_shift_C.nc", stats="mean")
df_zonal_stats = pd.DataFrame(zs)

countries = gpd.read_file(country_file)

df = pd.concat([countries, df_zonal_stats], axis=1)
ddf = df.dropna()

ddf.head()
pop_est continent name iso_a3 gdp_md_est geometry mean
1 53950935 Africa Tanzania TZA 150600.0 POLYGON ((33.90371119710453 -0.950000000000000... 27.045495
2 603253 Africa W. Sahara ESH 906.5 POLYGON ((-8.665589565454809 27.65642588959236... 20.714767
3 35623680 North America Canada CAN 1674000.0 (POLYGON ((-122.84 49.00000000000011, -122.974... -22.414753
4 326625791 North America United States of America USA 18560000.0 (POLYGON ((-122.84 49.00000000000011, -120 49.... -5.690464
5 18556698 Asia Kazakhstan KAZ 460700.0 POLYGON ((87.35997033076265 49.21498078062912,... -16.670960 
t2m =  dict(zip(ddf['name'].rename(columns={'name': 'id'}).tolist(), ddf['mean'].tolist()))
t2m

    {'Tanzania': 27.045494995406656,
     'W. Sahara': 20.71476744692734,
     'Canada': -22.41475295756037,
     'United States of America': -5.69046373681444,
     'Kazakhstan': -16.67096033269782,
     'Uzbekistan': 1.764118679849665,
     'Papua New Guinea': 24.81196305280159,
     'Indonesia': 25.875466322164467,
     'Argentina': 21.752232377674307,
     'Chile': 11.135730382916863,
     'Dem. Rep. Congo': 29.569072289446247,
     'Somalia': 30.30170728961632,
     'Kenya': 31.67995189114872,
     'Sudan': 27.486022838258766,
     'Chad': 26.692429239353384,
     'Dominican Rep.': 22.675683920426366,
     'Russia': -23.900154647163287,
     'Norway': -3.7497136369111415,
     'Greenland': -27.526973941152935,
     'South Africa': 33.44356093718852,
     'Mexico': 11.942822093680562,
     'Uruguay': 23.08714209232309,
     'Brazil': 24.905459109351693,
     'Bolivia': 20.032272190567276,
     'Peru': 17.16049273626683,
     'Colombia': 21.798475487111606,
     'Panama': 24.725506654058677,
     'Nicaragua': 21.889041330767355,
     'Honduras': 21.265986259753504,
     'El Salvador': 22.105261355411415,
     'Guatemala': 19.12085601525837,
     'Venezuela': 23.089038759205923,
     'Guyana': 23.260331487062643,
     'Suriname': 23.828620301834405,
     'France': 8.417135003615556,
     'Ecuador': 19.973881945816895,
     'Cuba': 23.046956462742855,
     'Zimbabwe': 32.16859650239843,
     'Botswana': 34.630019657943635,
     'Namibia': 31.81408359835583,
     'Senegal': 30.52788481539531,
     'Mali': 24.546538352410437,
     'Mauritania': 23.354622926381126,
     'Benin': 30.22631478102333,
     'Niger': 22.469154678136825,
     'Nigeria': 29.489976969825147,
     'Cameroon': 30.451354306672624,
     'Ghana': 31.08194862854576,
     "Côte d'Ivoire": 30.11038101780956,
     'Guinea': 29.309276343168847,
     'Guinea-Bissau': 28.614622097327697,
     'Liberia': 29.126722155695234,
     'Sierra Leone': 29.253324670124982,
     'Burkina Faso': 28.821453620901707,
     'Central African Rep.': 33.250194500675555,
     'Congo': 29.470682694898777,
     'Gabon': 28.80239211872915,
     'Eq. Guinea': 27.67434949015842,
     'Zambia': 26.890681219020223,
     'Malawi': 25.979440546978083,
     'Mozambique': 30.113032040839002,
     'Angola': 27.406082694876346,
     'Burundi': 24.782406660543984,
     'Israel': 18.70121846742944,
     'Madagascar': 28.40409207333217,
     'Gambia': 31.051364875059903,
     'Tunisia': 16.61156572925742,
     'Algeria': 17.589869296643716,
     'United Arab Emirates': 22.528455153645723,
     'Iraq': 13.12075033138536,
     'Oman': 24.151883463678914,
     'Vanuatu': 27.55201447621505,
     'Cambodia': 29.06413214856144,
     'Thailand': 26.60842270200081,
     'Laos': 22.496923066718466,
     'Myanmar': 20.29569889916325,
     'Vietnam': 22.37866588657321,
     'North Korea': -4.511850428322532,
     'South Korea': 2.0963740748619557,
     'Mongolia': -23.30693864195563,
     'India': 20.059997088207787,
     'Bangladesh': 20.210491139451506,
     'Nepal': 5.11491941890614,
     'Pakistan': 13.369403359726045,
     'Afghanistan': 5.918473954937194,
     'Tajikistan': -14.939487866831556,
     'Kyrgyzstan': -12.59378509947581,
     'Turkmenistan': 7.281742790873814,
     'Iran': 12.432886152984791,
     'Syria': 11.346418462485985,
     'Sweden': 0.217369902365661,
     'Belarus': -4.743480871389693,
     'Ukraine': -1.6838653944593165,
     'Poland': 2.0825046982811415,
     'Austria': 0.9910914488852995,
     'Hungary': 2.8823054144399123,
     'Romania': -0.029552417444435264,
     'Lithuania': 0.29406636944059983,
     'Latvia': -3.528902816289275,
     'Estonia': 0.4676113892207354,
     'Germany': 3.5804481725834227,
     'Bulgaria': 2.352424104045724,
     'Greece': 6.263588299827774,
     'Turkey': 5.532040235679036,
     'Albania': 4.457724344001122,
     'Switzerland': 1.1006239613694788,
     'Belgium': 5.082201915177109,
     'Portugal': 11.437221389502099,
     'Spain': 11.216733864371605,
     'Ireland': 10.672627552356118,
     'New Caledonia': 25.312999221019197,
     'New Zealand': 14.604062375448564,
     'Australia': 30.106313166468738,
     'Sri Lanka': 26.886284400337274,
     'China': -7.326497924451795,
     'Italy': 6.3752545625551535,
     'Denmark': 6.9727046306506395,
     'United Kingdom': 7.44461905943794,
     'Iceland': 4.226212442614155,
     'Azerbaijan': 0.2912213691163288,
     'Philippines': 26.56784757832856,
     'Malaysia': 25.98548617266715,
     'Slovenia': 3.1191516914348654,
     'Finland': -3.5334074001360327,
     'Slovakia': 0.9462826937781585,
     'Eritrea': 32.005862483850535,
     'Japan': 1.5481425123762391,
     'Paraguay': 26.81089189174427,
     'Yemen': 23.016799459305652,
     'Saudi Arabia': 21.604067131620234,
     'Antarctica': -19.436817208438438,
     'Morocco': 18.714198781408868,
     'Egypt': 20.496632518218867,
     'Libya': 17.524020555478355,
     'Ethiopia': 30.042507438644986,
     'Djibouti': 27.882034513829694,
     'Somaliland': 27.28102819532893,
     'Uganda': 27.406208209596514,
     'Bosnia and Herz.': 2.58286913031111,
     'Macedonia': 3.099236689165025,
     'Serbia': -0.2863136967092714,
     'S. Sudan': 35.95899043441552}

ddf.to_file("data/output.json", driver="GeoJSON")
geojson_data = json.load(open("data/output.json",'r'))

for feature in geojson_data['features']:
    properties = feature['properties']
    feature.update(id=properties['name'])
    print(feature['id'])

    Tanzania
    W. Sahara
    Canada
    United States of America
    Kazakhstan
    Uzbekistan
    Papua New Guinea
    Indonesia
    Argentina
    Chile
    Dem. Rep. Congo
    Somalia
    Kenya
    Sudan
    Chad
    Dominican Rep.
    Russia
    Norway
    Greenland
    South Africa
    Mexico
    Uruguay
    Brazil
    Bolivia
    Peru
    Colombia
    Panama
    Nicaragua
    Honduras
    El Salvador
    Guatemala
    Venezuela
    Guyana
    Suriname
    France
    Ecuador
    Cuba
    Zimbabwe
    Botswana
    Namibia
    Senegal
    Mali
    Mauritania
    Benin
    Niger
    Nigeria
    Cameroon
    Ghana
    Côte d'Ivoire
    Guinea
    Guinea-Bissau
    Liberia
    Sierra Leone
    Burkina Faso
    Central African Rep.
    Congo
    Gabon
    Eq. Guinea
    Zambia
    Malawi
    Mozambique
    Angola
    Burundi
    Israel
    Madagascar
    Gambia
    Tunisia
    Algeria
    United Arab Emirates
    Iraq
    Oman
    Vanuatu
    Cambodia
    Thailand
    Laos
    Myanmar
    Vietnam
    North Korea
    South Korea
    Mongolia
    India
    Bangladesh
    Nepal
    Pakistan
    Afghanistan
    Tajikistan
    Kyrgyzstan
    Turkmenistan
    Iran
    Syria
    Sweden
    Belarus
    Ukraine
    Poland
    Austria
    Hungary
    Romania
    Lithuania
    Latvia
    Estonia
    Germany
    Bulgaria
    Greece
    Turkey
    Albania
    Switzerland
    Belgium
    Portugal
    Spain
    Ireland
    New Caledonia
    New Zealand
    Australia
    Sri Lanka
    China
    Italy
    Denmark
    United Kingdom
    Iceland
    Azerbaijan
    Philippines
    Malaysia
    Slovenia
    Finland
    Slovakia
    Eritrea
    Japan
    Paraguay
    Yemen
    Saudi Arabia
    Antarctica
    Morocco
    Egypt
    Libya
    Ethiopia
    Djibouti
    Somaliland
    Uganda
    Bosnia and Herz.
    Macedonia
    Serbia
    S. Sudan

import branca.colormap as cm
step = cm.StepColormap(['blue', 'cyan','yellow', 'red'],
                           vmin = int(ddf['mean'].min()), vmax = int(ddf['mean'].max()))
colors = step.to_linear()

colors
-2735 
from ipyleaflet import Map, Choropleth, basemaps, WidgetControl
import branca.colormap as cm
import matplotlib.pyplot as plt
from ipywidgets import widgets

map = Map(center=(52.3,8.0), zoom = 3, basemap= basemaps.Esri.WorldTopoMap)

geo_data = Choropleth(geo_data = geojson_data, choro_data = t2m,colormap = colors,
    border_color='black', hover_style={ 'fillOpacity': 0.4},
    style={'fillOpacity': 0.8},
                   name = 'Countries')


map.add_layer(geo_data)
out = widgets.Output(layout={'border': '1px solid black'})
out.append_stdout('Temperature (degrees C) 25th December 2018')
with out:
    display(colors)
widget_control = WidgetControl(widget=out, position='topright')
map.add_control(widget_control)

map

Add widget for zoom

from ipyleaflet import Map, Choropleth, basemaps, WidgetControl, Popup, Marker
import branca.colormap as cm
import matplotlib.pyplot as plt
from ipywidgets import widgets, IntSlider, jslink

map = Map(center=(52.3,8.0), zoom = 3, basemap= basemaps.Esri.WorldTopoMap)

geo_data = Choropleth(geo_data = geojson_data, choro_data = t2m,colormap = colors,
    border_color='black', hover_style={ 'fillOpacity': 0.4},
    style={'fillOpacity': 0.8},
                   name = 'Countries')


map.add_layer(geo_data)
out = widgets.Output(layout={'border': '1px solid black'})
out.append_stdout('Temperature (degrees C) 25th December 2018')
with out:
    display(colors)
widget_control = WidgetControl(widget=out, position='topright')
map.add_control(widget_control)
zoom_slider = IntSlider(description='Zoom level:', min=0, max=15, value=2)
jslink((zoom_slider, 'value'), (map, 'zoom'))
widget_control_zoom = WidgetControl(widget=zoom_slider, position='bottomright')
map.add_control(widget_control_zoom)

map

Save the resulting map

from ipywidgets.embed import embed_minimal_html

embed_minimal_html('temperature_per_country.html', views=[map], title='Temperature (degrees C), 25th December 2018')

Key Points

  • vector data in ipyleaflet

  • markers, circles, choropleth maps

previous episode

lesson home