Regional Energy Information in High Resolution

Collecting, analyzing and visualizing energy policy data from Swiss communities

#15

National energy policy goals are known and monitored, but how is the progress of the Swiss energy transition on regional level?

Working on our challenges will contributing to the current, highly relevant discussion on security of energy supply in Switzerland. You will support us to develop cutting-edge energy policy design.

Challenge slides

Step 1: Web scraping energy policy goals

Find and gather information about energy policy goals and activities of municipalities. Identify ambitious and less ambitious municipalities Optional: Cluster municipalities that follow same/close goals and benchmark

Step 2: regionalize renewable energy production

How much production in Swiss municipalities? Data wrangling and fusion

🧑🏽‍🏫 GeoImpact/BfE Digital Office


Project Team

Who has which skills and is in which team - policy (P), data (D), scraping (S)

Project Team

Challenge Team

Team 1 - Policy and Data

Which energy policy keywords and categories are relevant?

Step 1 - Brainstorming Keywords

Relevant Keywords

Step 2 - Structuring Keywords

Clustering brainstorming results mind map and converting to a structured table

Structured Keywords Table

Team 2 - Web Scraping

Finding energy policy information on Swiss municipality websites.

 Step 1 - Manual Samples

Manually retrieving text data from few sample municipalities

Community Text

Step 2 - Building a Scraper

As some research revealed, there is no service or implementation which could be used out of the box, so it was decided to go for a simple prototype implementation:

Semi automatic information gathering:

  • Loop over 64 municipalities as test set
  • For every municipality initiate a google search with municipality name and keywords «Energie Umwelt»
  • Take top 3 google results
  • Pick sentences where keywords from team 1 appear
  • Run sentiment analysis over retrieved sentences

Screen Scraper

Scraper result set

Scraper Result Set

Scraper result set - sentences and sentiment analysis detail

Scraper Results Sentences

Reconciliation and Validation

Bringing the pieces together, checking if it makes sense...

Results Visualization

Visualize results on map (interactive map)

Result Visualisation on Map

Showing municipalities by keywords on map (interactive)

Keywords on Map

Sample Municipality Aadorf

Validating Sentences

Sentences analysis

Aadorf sentences analysis

Map Visualization

Analyzing details (interactive map)

  • Thermal network - green
  • High solar capacity (kW/cap) - red
  • Energy city - blue

Details Aadorf

Final Pitch

By Jan-Philipp Sasse (with assistance of Thilo Weber)

Final Pitch

Challenge Data and Code

See links below for Github repository (click on «Source» icon).

Energy Hackdays 2022 - Regional Energy Information

Code and project files of challenge 15 about Regional Energy Information.

Information about the challenge and documentation: https://hack.opendata.ch/project/848

This content is a preview from an external site.
▲▲▲

Repository updated

19.09.2022 13:08 ~ loleg

Visualizations are part of documentation

19.09.2022 10:18 ~ PeterJanes

Analyse

19.09.2022 10:16

Data and code published on Github, link added as project information

19.09.2022 10:16 ~ PeterJanes

Clean

19.09.2022 10:14

Edited content version 107

19.09.2022 10:14 ~ PeterJanes

Edited content version 105

19.09.2022 10:14 ~ PeterJanes

Documentation completed, challenge pitched, documentation finalised

19.09.2022 10:09 ~ PeterJanes

Verify

19.09.2022 10:08

Edited content version 101

19.09.2022 10:08 ~ PeterJanes

Edited content version 99

19.09.2022 10:07 ~ PeterJanes

Edited content version 97

17.09.2022 19:55 ~ PeterJanes

Edited content version 95

17.09.2022 19:45 ~ PeterJanes

Event finished

17.09.2022 16:00

Get

17.09.2022 11:38

Edited content version 92

17.09.2022 11:38 ~ PeterJanes

Edited content version 90

17.09.2022 11:36 ~ PeterJanes

Edited content version 88

17.09.2022 11:36 ~ PeterJanes

Edited content version 86

17.09.2022 11:33 ~ PeterJanes

Edited content version 84

17.09.2022 11:30 ~ PeterJanes

Documentation mostly completed - hack portal, pitch deck

17.09.2022 10:43 ~ PeterJanes

Edited content version 80

17.09.2022 09:32 ~ PeterJanes

Edited content version 78

17.09.2022 09:27 ~ PeterJanes

Edited content version 76

17.09.2022 09:26 ~ PeterJanes

Edited content version 74

17.09.2022 09:23 ~ PeterJanes

Edited content version 72

17.09.2022 09:12 ~ PeterJanes

Edited content version 70

17.09.2022 09:00 ~ PeterJanes

Edited content version 68

17.09.2022 08:46 ~ PeterJanes

Edited content version 66

17.09.2022 08:45 ~ PeterJanes

Edited content version 64

17.09.2022 08:40 ~ PeterJanes

Edited content version 62

17.09.2022 08:37 ~ PeterJanes

Edited content version 60

17.09.2022 07:56 ~ PeterJanes

Edited content version 58

17.09.2022 07:42 ~ PeterJanes

Edited content version 56

17.09.2022 07:40 ~ PeterJanes

Edited content version 54

17.09.2022 07:34 ~ PeterJanes

Working plan for day 2 coordinated - approach for findings and documentation

17.09.2022 07:21 ~ PeterJanes

Edited content version 50

17.09.2022 06:59 ~ lukas_oberholzer

Edited content version 48

16.09.2022 18:31 ~ PeterJanes

Collaboration space set up, first cut project documentation completed

16.09.2022 18:25 ~ PeterJanes

Find

16.09.2022 18:18

Edited content version 44

16.09.2022 18:18 ~ PeterJanes

Edited content version 42

16.09.2022 18:14 ~ PeterJanes

Edited content version 40

16.09.2022 18:13 ~ PeterJanes

Edited content version 38

16.09.2022 18:10 ~ PeterJanes

Edited content version 36

16.09.2022 18:09 ~ PeterJanes

Edited content version 34

16.09.2022 18:08 ~ PeterJanes

Edited content version 32

16.09.2022 18:01 ~ PeterJanes

Edited content version 30

16.09.2022 17:52 ~ PeterJanes

Edited content version 28

16.09.2022 17:47 ~ PeterJanes

Edited content version 26

16.09.2022 17:46 ~ PeterJanes

Edited content version 24

16.09.2022 17:40 ~ PeterJanes

Edited content version 22

16.09.2022 17:33 ~ PeterJanes

Two teams - web scraping (to collect data), analysis (verify first examples manually)

16.09.2022 17:03 ~ PeterJanes

Ask

16.09.2022 17:00

Edited content version 17

16.09.2022 17:00 ~ PeterJanes

Edited content version 15

16.09.2022 16:56 ~ PeterJanes

Joined the team

16.09.2022 16:56 ~ lucas_tochtermann_bfe

2 teams - web scraping, data analysis - first PoCs being developed

16.09.2022 16:53 ~ PeterJanes

Joined the team

16.09.2022 10:41 ~ PeterJanes

Event started

16.09.2022 09:00

Ask

15.08.2022 15:02

Edited content version 5

15.08.2022 15:02 ~ darienne_hunziker

Edited content version 3

15.08.2022 15:01 ~ darienne_hunziker

Challenge posted

15.08.2022 15:01 ~ darienne_hunziker
▲▲▲
All attendees, sponsors, partners, volunteers and staff at our hackathon are required to agree with the Hack Code of Conduct. Organisers will enforce this code throughout the event. We expect cooperation from all participants to ensure a safe environment for everybody.

Creative Commons LicenceThe contents of this website, unless otherwise stated, are licensed under a Creative Commons Attribution 4.0 International License.

Energy Data Hackdays 2022