Repository updated
Regional Energy Information in High Resolution
Collecting, analyzing and visualizing energy policy data from Swiss communities
National energy policy goals are known and monitored, but how is the progress of the Swiss energy transition on regional level?
Working on our challenges will contributing to the current, highly relevant discussion on security of energy supply in Switzerland. You will support us to develop cutting-edge energy policy design.
Step 1: Web scraping energy policy goals
Find and gather information about energy policy goals and activities of municipalities. Identify ambitious and less ambitious municipalities Optional: Cluster municipalities that follow same/close goals and benchmark
Step 2: regionalize renewable energy production
How much production in Swiss municipalities? Data wrangling and fusion
🧑🏽🏫 GeoImpact/BfE Digital Office
Project Team
Who has which skills and is in which team - policy (P), data (D), scraping (S)
- Emilie Boillat
- Guiseppe Caponetto
- Peter Janes («Piet»)
- Baptiste Jordan
- Manuel Meyer
- Lukas Oberholzer
- Matthias Pauli
- Peter Robineau
- Jan-Philipp Sasse
- Alexander Thommen
- Lucas Tochtermann
- Thilo Weber
Team 1 - Policy and Data
Which energy policy keywords and categories are relevant?
Step 1 - Brainstorming Keywords
Step 2 - Structuring Keywords
Clustering brainstorming results mind map and converting to a structured table
Team 2 - Web Scraping
Finding energy policy information on Swiss municipality websites.
Step 1 - Manual Samples
Manually retrieving text data from few sample municipalities
Step 2 - Building a Scraper
As some research revealed, there is no service or implementation which could be used out of the box, so it was decided to go for a simple prototype implementation:
Semi automatic information gathering:
- Loop over 64 municipalities as test set
- For every municipality initiate a google search with municipality name and keywords «Energie Umwelt»
- Take top 3 google results
- Pick sentences where keywords from team 1 appear
- Run sentiment analysis over retrieved sentences
Scraper result set
Scraper result set - sentences and sentiment analysis detail
Reconciliation and Validation
Bringing the pieces together, checking if it makes sense...
Results Visualization
Visualize results on map (interactive map)
Showing municipalities by keywords on map (interactive)
Sample Municipality Aadorf
Validating Sentences
Sentences analysis
Map Visualization
Analyzing details (interactive map)
- Thermal network - green
- High solar capacity (kW/cap) - red
- Energy city - blue
Final Pitch
By Jan-Philipp Sasse (with assistance of Thilo Weber)
Challenge Data and Code
See links below for Github repository (click on «Source» icon).
Energy Hackdays 2022 - Regional Energy Information
Code and project files of challenge 15 about Regional Energy Information.
Information about the challenge and documentation: https://hack.opendata.ch/project/848
Visualizations are part of documentation
Analyse
Data and code published on Github, link added as project information
Clean
Edited content version 107
Edited content version 105
Documentation completed, challenge pitched, documentation finalised
Verify
Edited content version 101
Edited content version 99
Edited content version 97
Edited content version 95
Event finished
Get
Edited content version 92
Edited content version 90
Edited content version 88
Edited content version 86
Edited content version 84
Documentation mostly completed - hack portal, pitch deck
Edited content version 80
Edited content version 78
Edited content version 76
Edited content version 74
Edited content version 72
Edited content version 70
Edited content version 68
Edited content version 66
Edited content version 64
Edited content version 62
Edited content version 60
Edited content version 58
Edited content version 56
Edited content version 54
Working plan for day 2 coordinated - approach for findings and documentation
Edited content version 50
Edited content version 48
Collaboration space set up, first cut project documentation completed
Find
Edited content version 44
Edited content version 42
Edited content version 40
Edited content version 38
Edited content version 36
Edited content version 34
Edited content version 32
Edited content version 30
Edited content version 28
Edited content version 26
Edited content version 24
Edited content version 22
Two teams - web scraping (to collect data), analysis (verify first examples manually)
Ask
Edited content version 17
Edited content version 15
Joined the team
2 teams - web scraping, data analysis - first PoCs being developed
Joined the team
Event started
Ask
Edited content version 5
Edited content version 3
Challenge posted