Challenge Project

Regional Energy Information in High Resolution

Collecting, analyzing and visualizing energy policy data from Swiss communities

#15

📂 Demo

National energy policy goals are known and monitored, but how is the progress of the Swiss energy transition on regional level?

Working on our challenges will contributing to the current, highly relevant discussion on security of energy supply in Switzerland. You will support us to develop cutting-edge energy policy design.

Challenge slides

Step 1: Web scraping energy policy goals

Find and gather information about energy policy goals and activities of municipalities. Identify ambitious and less ambitious municipalities Optional: Cluster municipalities that follow same/close goals and benchmark

Step 2: regionalize renewable energy production

How much production in Swiss municipalities? Data wrangling and fusion

🧑🏽‍🏫 GeoImpact/BfE Digital Office


Project Team

Who has which skills and is in which team - policy (P), data (D), scraping (S)

Project Team

Challenge Team

Team 1 - Policy and Data

Which energy policy keywords and categories are relevant?

Step 1 - Brainstorming Keywords

Relevant Keywords

Step 2 - Structuring Keywords

Clustering brainstorming results mind map and converting to a structured table

Structured Keywords Table

Team 2 - Web Scraping

Finding energy policy information on Swiss municipality websites.

 Step 1 - Manual Samples

Manually retrieving text data from few sample municipalities

Community Text

Step 2 - Building a Scraper

As some research revealed, there is no service or implementation which could be used out of the box, so it was decided to go for a simple prototype implementation:

Semi automatic information gathering:

  • Loop over 64 municipalities as test set
  • For every municipality initiate a google search with municipality name and keywords «Energie Umwelt»
  • Take top 3 google results
  • Pick sentences where keywords from team 1 appear
  • Run sentiment analysis over retrieved sentences

Screen Scraper

Scraper result set

Scraper Result Set

Scraper result set - sentences and sentiment analysis detail

Scraper Results Sentences

Reconciliation and Validation

Bringing the pieces together, checking if it makes sense...

Results Visualization

Visualize results on map (interactive map)

Result Visualisation on Map

Showing municipalities by keywords on map (interactive)

Keywords on Map

Sample Municipality Aadorf

Validating Sentences

Sentences analysis

Aadorf sentences analysis

Map Visualization

Analyzing details (interactive map)

  • Thermal network - green
  • High solar capacity (kW/cap) - red
  • Energy city - blue

Details Aadorf

Final Pitch

By Jan-Philipp Sasse (with assistance of Thilo Weber)

Final Pitch

Challenge Data and Code

See links below for Github repository (click on «Source» icon).

Energy Hackdays 2022 - Regional Energy Information

Code and project files of challenge 15 about Regional Energy Information.

Information about the challenge and documentation: https://hack.opendata.ch/project/848

This content is a preview from an external site.
 

Repository updated

19.09.2022 13:08 ~ loleg

Share

Visualizations are part of documentation

19.09.2022 10:18 ~ PeterJanes

Data and code published on Github, link added as project information

19.09.2022 10:16 ~ PeterJanes

Launch

Edited (version 107)

19.09.2022 10:14 ~ PeterJanes

Prototype

Edited (version 105)

19.09.2022 10:14 ~ PeterJanes

Documentation completed, challenge pitched, documentation finalised

19.09.2022 10:09 ~ PeterJanes

Edited (version 101)

19.09.2022 10:08 ~ PeterJanes

Sketching

Edited (version 99)

19.09.2022 10:07 ~ PeterJanes

Edited (version 97)

17.09.2022 19:55 ~ PeterJanes

Edited (version 95)

17.09.2022 19:45 ~ PeterJanes

Event finished

Edited (version 92)

17.09.2022 11:38 ~ PeterJanes

Research

Edited (version 90)

17.09.2022 11:36 ~ PeterJanes

Edited (version 88)

17.09.2022 11:36 ~ PeterJanes

Edited (version 86)

17.09.2022 11:33 ~ PeterJanes

Edited (version 84)

17.09.2022 11:30 ~ PeterJanes

Documentation mostly completed - hack portal, pitch deck

17.09.2022 10:43 ~ PeterJanes

Edited (version 80)

17.09.2022 09:32 ~ PeterJanes

Edited (version 78)

17.09.2022 09:27 ~ PeterJanes

Edited (version 76)

17.09.2022 09:26 ~ PeterJanes

Edited (version 74)

17.09.2022 09:23 ~ PeterJanes

Edited (version 72)

17.09.2022 09:12 ~ PeterJanes

Edited (version 70)

17.09.2022 09:00 ~ PeterJanes

Edited (version 68)

17.09.2022 08:46 ~ PeterJanes

Edited (version 66)

17.09.2022 08:45 ~ PeterJanes

Edited (version 64)

17.09.2022 08:40 ~ PeterJanes

Edited (version 62)

17.09.2022 08:37 ~ PeterJanes

Edited (version 60)

17.09.2022 07:56 ~ PeterJanes

Edited (version 58)

17.09.2022 07:42 ~ PeterJanes

Edited (version 56)

17.09.2022 07:40 ~ PeterJanes

Edited (version 54)

17.09.2022 07:34 ~ PeterJanes

Working plan for day 2 coordinated - approach for findings and documentation

17.09.2022 07:21 ~ PeterJanes

Edited (version 50)

17.09.2022 06:59 ~ lukas_oberholzer

Edited (version 48)

16.09.2022 18:31 ~ PeterJanes

Collaboration space set up, first cut project documentation completed

16.09.2022 18:25 ~ PeterJanes

Edited (version 44)

16.09.2022 18:18 ~ PeterJanes

Project

Edited (version 42)

16.09.2022 18:14 ~ PeterJanes

Edited (version 40)

16.09.2022 18:13 ~ PeterJanes

Edited (version 38)

16.09.2022 18:10 ~ PeterJanes

Edited (version 36)

16.09.2022 18:09 ~ PeterJanes

Edited (version 34)

16.09.2022 18:08 ~ PeterJanes

Edited (version 32)

16.09.2022 18:01 ~ PeterJanes

Edited (version 30)

16.09.2022 17:52 ~ PeterJanes

Edited (version 28)

16.09.2022 17:47 ~ PeterJanes

Edited (version 26)

16.09.2022 17:46 ~ PeterJanes

Edited (version 24)

16.09.2022 17:40 ~ PeterJanes

Edited (version 22)

16.09.2022 17:33 ~ PeterJanes

Two teams - web scraping (to collect data), analysis (verify first examples manually)

16.09.2022 17:03 ~ PeterJanes

Edited (version 17)

16.09.2022 17:00 ~ PeterJanes

Edited (version 15)

16.09.2022 16:56 ~ PeterJanes

Joined the team

16.09.2022 16:56 ~ lucas_tochtermann_bfe

Challenge

Event started

 
All attendees, sponsors, partners, volunteers and staff at our hackathon are required to agree with the Hack Code of Conduct. Organisers will enforce this code throughout the event. We expect cooperation from all participants to ensure a safe environment for everybody.

Creative Commons LicenceThe contents of this website, unless otherwise stated, are licensed under a Creative Commons Attribution 4.0 International License.