08.02.2024, RDC Campus test run: Jupyter Lab

tags: RDC-Campus RDC Jupyter IOER

Topic: Jupyter Lab & TUD HPC
Time: 12:30 – 14:00
Location: IOER-Library

Facilitator: Alexander Dunkel

---

Jupyter Lab is an interactive browser based code development environment that can be used to draft code, comment workflows and generate visuals. It is interoperable and can be used from anywhere (laptop, tablet), without requiring setup of software on clients. The code runs locally or on a remote computer, such as the High Performance Cluster of the TU Dresden. In this workshop, we will introduce working with Jupyter Lab. You will work through four Jupyter notebooks yourself.

Note

The workshop is a collaboration between the TUD Institute of Cartography and the IOER RDC. The workshop is extensive an can be used beyond the workshop date. By signing up, you will be able to use the workshop with the TUD HPC resources until the end of 2024.

Highlights

• You will get a brief introduction to working with the Jupyer ab interface
• The workshop is self-guided, based on your own pace
• Four sample notebooks provided, e.g. to query Web APIs, process data or visualize interactive maps
• Level: Intermediate/Advanced
• Pre-requisite: basic Python coding experience preferred, but not required

Application Examples

Below are HTML versions of the notebooks that can be explored in the workshop.

1. Working with Web APIs, quering data from Instagram and the Wikimedia API, producing an interactive Geoviews map
2. Privacy-aware processing of Social Media data using HyperLogLog
3. Tag Maps Visualizationand exploration for a self-chosen area
4. Topic modelling and semantic clustering

These are some of the visuals you will produce in the workshop:

• A World Wide Map of Flickr photos
• A Geoviews map for places queried from the Wikimedia API near the Großer Garten
• A heatmap showing nature-related discussions in the Grosser Garten, Dresden

See some more example notebooks:

• GenerativeAI
• Milvus milvus data processing
• Mapnik rendering
• ChatGPT

These are results from the Cartography Student class in 2023:

• Student results 2023

Lectures

These are the links to the two preparation lectures from December 2023:

• Analysis and Visualization of Geo-Social Media
• Python data science 101 for Social Media Cartography (and beyond)

FAQ

After the workshop, there were some questions that we collect here.

Do you have a tutorial on how to create a python environment in the HPC hub?

There are different ways to do this. The simplest is to create an environment within a Jupyter notebook with the following commands.

Create a new environment within your current folder:

!python -m venv ./wikidata_venv

Install packages to the venv:

!./wikidata_venv/bin/python -m pip install qwikidata ipykernel pandas

Install the Jupyter Kernel to the venv, so it can be selected in the top-right corner of Jupyter Lab:

!./wikidata_venv/bin/python -m ipykernel install --user --name=qwikidata

Afterwards, refresh Jupyter lab F5 and you should be able to select qwikidata Kernel in the top-right corner.