|
|
| (97 intermediate revisions by 18 users not shown) |
| Line 1: |
Line 1: |
| | | | |
| − | This is work in progress and supposed to become a link list to sources of open energy related data. We focus on collecting links to data relevant for the modelling of energy and electricity systems and markets. You are welcome to fill in the missing spots and non-existing pages. Also, you are welcome to extend the list of relevant data that we should collect links to in the future.<br/> | + | This is work in progress and is supposed to become a link list to sources of open energy-related data. We focus on collecting links to data relevant for the modelling of energy and electricity systems and markets. You are welcome to fill in the missing spots and non-existing pages. Also, you are welcome to extend the list of relevant data (e.g. for a European energy system model) that we should collect links to in the future: |
| | | | |
| − | <br/>
| + | [[Data requirements for a European energy system model|Data requirements for a European energy system model]] |
| | | | |
| | <br/> | | <br/> |
| | | | |
| − | == Model relevant data<br/> == | + | = Open data sources for energy modelling = |
| | | | |
| − | Please note that red links indicate planned, but not yet filled pages. You are welcome to start creating the missing pages and breathe life into them.
| + | Open datasets related to energy are listed here by type. |
| | | | |
| − | *[[Electricity demand|Electricity demand]] | + | *Demand |
| − | *[[Wind profiles|Wind profiles]] | + | **[[Electricity demand|Electricity demand]] |
| − | *[[Power plant portfolios|Power plant portfolios]] | + | **[[Thermal demand|Thermal demand]] |
| − | *[[Transmission network datasets|Transmission network datasets]] | + | **[[Transport demand|Transport demand]] |
| − | *[[:File:Flexible demand and storage|Flexible demand and storage]] | + | **[[Industrial demand|Industrial demand]] |
| | + | **[[Demand response|Demand response]] |
| | | | |
| − | <br/>
| + | *Resources and Potentials |
| | + | **[[Weather data|Weather data and availability of renewable energy]] |
| | + | **[[Wind geographical potentials|Wind geographical potentials]] |
| | + | **[[Biomass potentials|Biomass potentials]] |
| | + | **[[Hydroelectricity data|Hydroelectricity potentials]] |
| | | | |
| | + | *Networks |
| | + | **[[Transmission network datasets|Electricity transmission network datasets]] |
| | + | **[[Distribution network datasets|Electricity distribution network datasets]] |
| | + | **[[Gas network datasets|Gas network datasets]] |
| | | | |
| − | == Various other open energy data sources ==
| + | *Technologies |
| | + | **[[Power plant portfolios|Power plants]] |
| | + | **[[Technology assumptions|Generation technology assumptions and projections]] |
| | + | **[[Storage technology assumptions and projections|Storage technology assumptions and projections]] |
| | + | **[[End-use technology assumptions and projections|End-use technology assumptions and projections]] |
| | | | |
| − | A small selection of high-quality datasets per type could be compiled here.
| + | *[[:File:Electricity prices|Electricity prices]] |
| | + | *[[Demographic and socio-economic data|Demographic and socio-economic data]] |
| | + | *[[Environmental data and regulations|Environmental data and regulations]] |
| | | | |
| − | === Fuel ===
| + | *[[Historical data and profiles|Historical data and profiles]] |
| | | | |
| − | === Transmission ===
| + | *[[Energy Scenarios|Energy scenarios]] |
| | | | |
| − | *[http://www.scigrid.de/ SciGRID], a European transmission grid model based on OpenStreetMap<br/> | + | *[[Country-specific targets and policies|Country-specific targets and policies]] |
| − | | + | |
| − | See also [[Transmission network datasets|Transmission network datasets]]<br/>
| + | |
| | | | |
| | <br/> | | <br/> |
| | | | |
| − | === Power Plants === | + | = Other lists of energy-related open datasets = |
| | | | |
| − | *[http://open-power-system-data.org/ Open Power System Data], a platform for European power system data such as power plants and load, wind and solar time series | + | *The Wikipedia article [https://en.wikipedia.org/wiki/Open_energy_system_databases open energy system databases] lists some of the portals serving open energy system datasets. |
| | + | *[http://datahub.io/dataset?q=energy datahub.io category "Energy"] has more datasets. |
| | + | *[http://energypedia.info/ Energypedia] is a wiki platform for collaborative knowledge exchange on renewable energy and energy access issues in the context of development cooperation. *[http://enipedia.tudelft.nl/ Enipedia (TU Delft)] is an active exploration into the applications of wikis and the semantic web for energy and industry issues. An extensive compilation of links to other data sources can be found at [http://enipedia.tudelft.nl/wiki/Energy_and_Industry_Data_Sets Energy and Industry Data Sets] and [http://enipedia.tudelft.nl/wiki/Electricity_Transmission_Network Electricity_Transmission Network]. |
| | | | |
| − | === Weather data<br/> ===
| + | *[https://www.europeandataportal.eu/ European Data Portal] |
| − | | + | *[http://www.iea-etsap.org/web/E-techDS.asp IEA ETSAP energy technology data source (E-Tech-DS)] is a series of four-page technology briefs similar to the [https://www.iea.org/publications/freepublications/ IEA Energy Technology Essentials (filter for "essentials")]. The page contains short technical descriptions of 29 energy-related technologies from power production, synthesised fuels, and fossil fuel production. |
| − | *[http://www.cosmo-model.org/ COSMO (Consortium for Small-scale Modelling)], simulation model for weather in Europe, used for the official weather forecasts. The model is free to use for research applications. You may have a chance to get data directly from national weather services or participating universities/institutions and thus avoid having to run it yourself. For Germany, the current resolution is 2.8 km. | + | *[http://www.iaee.org/en/EnergyDataLinks/ International Association of Energy Economists (IAEE) Energy Data Links (EDL)] provides a searchable database of energy-related resources |
| − | *[http://rda.ucar.edu/datasets/ds094.1/ National Centers for Environmental Prediction (NCEP) Climate Forecast System (CFS)], worldwide hourly reanalysis weather data, currently 0.2 deg spatial resolution. You need an account to get access to the downloads, but as the data is from a public US institution, it is free. | + | *[http://en.openei.org/ OpenEI] features a wiki of crowd-sourced energy information and a database of single-source data on buildings, energy, efficiency, consumption, demand, potential. |
| − | | + | *[[Openenergy-platform.org/|Open Energy Platform]] (OEP) and [https://openenergy-platform.org/dataedit/schemas Open Energy Database] (OEDB) was started with the requirements gathered in the first openmod meetings and was developed completely open with the support of the openmod community. Input and result data from research of energy system studies are available via an API. The OEDB provides some automated visualisations of the available data. |
| − | === Distribution ===
| + | *[http://open-power-system-data.org/data-sources Open Power System Data] has an extensive collection of links to data sources (Electricity consumption, Capacity and generation by fuel, Power plant data, Hydropower data, Prices and related data, Weather data, Wind and solar power time series, Country-specific data portals). |
| − | | + | *[http://www.ourenergypolicy.org/resource-library/ OurEnergyPolicy.org Resource Library] is a free online energy resource library updated weekly. |
| − | === Storage ===
| + | *[http://www.pfbach.dk/ PFBach.dk], a collection of wind and solar in-feed profiles |
| − | | + | *[https://www.reeep.org/reegle-clean-energy-information-portal reegle] is a data provider of country energy profiles, energy statistics and a directory of relevant stakeholders. It also offers the clean energy search and an extensive glossary. There is also an insightful clean energy blog with interesting and up-to-date background information. |
| − | === Consumption ===
| + | *The [https://globalenergymonitor.org/ Global Energy Monitor] is a project that maintains trackers on global (i) coal plants, (ii) fossil infrastructure, (iii) coal mines, (iv) steel plants, and (v) the European gas infrastructure with a global gas plant tracker in development. The data underlying the several trackers can be requested. |
| − | <ul style="font-size: 13.6px;">
| + | *[https://energydata.info/ https://energydata.info/] features a collection of almost 600 open energy-related datasets put together by the World Bank Group. |
| − | <li>[http://open-power-system-data.org/ Open Power System Data], a platform for European power system data such as power plants and load, wind and solar time series</li>
| + | *The [https://opendata.ffe.de/ FfE Open Data Portal] offers free datasets for modelling energy demand and generation with a regional focus on Europe and Germany. |
| − | </ul>
| + | *"[https://deep.eefig.eu/ DEEP] [De-risking Energy Efficiency Platform] is an open-source database for energy efficiency investments performance monitoring and benchmarking." |
| | | | |
| | <br/> | | <br/> |
| | | | |
| − | === Meta === | + | = Data sharing techniques = |
| | | | |
| − | This is a list to lists of datasets:
| + | The [https://okfn.org/ Open Knowledge Foundation] promotes the use of its '''[http://data.okfn.org/ data package]''' standard. It consists of using CSV for payload (data) and a file package.json to attach machine-readable metadata. The page links to many examples of existing, curated and maintained datasets that adhere to this standard. Additionally, they drive the creation of a software ecosystem that can create and digest this format. Due to its simplicity, using data packages does not depend on this ecosystem. |
| | | | |
| − | *[http://enipedia.tudelft.nl/ Enipedia (TU Delft)] is an active exploration into the applications of wikis and the semantic web for energy and industry issues.
| + | '''GitHub repositories''' are another pragmatic way of sharing "small" (up to about 10 MB) datasets. A fun example is the [http://bundestag.github.io/gesetze/ Bundesgit], a collection of all German federal laws under version control. New laws or modifications are tracked as commits, allowing to "see" how a dataset -- laws, in that case -- evolve over time. The repository [https://github.com/openmundi/world.db openmundi/world.db] shows a more data-focused way of using Git, or GitHub, for a collaborative collection of data. However, it clearly shows the limitations of using a version control system for code on data. |
| − | **An extensive compilation of links to other data sources can be found at [http://enipedia.tudelft.nl/wiki/Energy_and_Industry_Data_Sets Energy and Industry Data Sets] and [http://enipedia.tudelft.nl/wiki/Electricity_Transmission_Network Electricity_Transmission Network]
| + | |
| | | | |
| − | *[http://energypedia.info/ Energypedia] is a wiki platform for collaborative knowledge exchange on renewable energy and energy access issues in the context of development cooperation.<br/>
| + | An upcoming and (technically) promising project is '''[https://dat.foundation/ dat]''', which "is a version-controlled, decentralized data tool for collaboration between data people and data systems." Or, simply: Git for data. It is currently in public beta test but has come a long way already. |
| − | *[http://data.reegle.info/ reegle] is a data provider of country energy profiles, energy statistics and a directory of relevant stakeholders. It also offers the clean energy search and an extensive glossary. There is also an insightful clean energy blog with interesting and up-to-date background information.
| + | |
| − | *[http://www.iea-etsap.org/web/E-techDS.asp IEA ETSAP energy technology data source (E-Tech-DS)] is a series of four-page technology briefs similar to the [https://www.iea.org/publications/freepublications/ IEA Energy Technology Essentials (filter for "essentials")]. The page contains short technical descriptions of 29 energy related technologies from power production, synthesised fuels, and fossil fuel production.
| + | |
| − | *[http://en.openei.org/ OpenEI] features a wiki of crowd-sourced energy information and a database of single source data on buildings, energy, efficiency, consumption, demand, potential.
| + | |
| − | *[http://datahub.io/dataset?q=energy datahub.io category "Energy"] has more datasets
| + | |
| − | *[http://www.pfbach.dk/ PFBach.dk], a collection of wind and solar in-feed profiles<br/>
| + | |
| | | | |
| | <br/> | | <br/> |
| | | | |
| | + | = Help finding energy data = |
| | | | |
| − | == Data sharing techniques<br/> ==
| + | You can find a list of the latest questions on energy data sources on StackExchange: [http://opendata.stackexchange.com/questions/tagged/energy http://opendata.stackexchange.com/questions/tagged/energy] |
| | | | |
| − | The Open Knowledge foundation promotes the use of its '''[http://data.okfn.org/ data package]''' standard. It consists of using CSV for payload (data) and a file package.json to attach machine-readable metadata. The page links to many examples of existing, curated and maintained datasets that adhere to this standard. Additionally, they drive the creation of a software ecosystem that can create and digest this format. Due to its simplicity, using data packages does not depend on this ecosystem.<br/>
| + | <br/> |
| | | | |
| − | '''GitHub repositories''' are another pragmatic way of sharing "small" (up to about 10 MB) datasets. A fun example is the [http://bundestag.github.io/gesetze/ Bundesgit], a collection of all German federal laws under version control. New laws or modfications are tracked as commits, allowing to "see" how a dataset -- laws, in that case -- evolve over time. The repository [https://github.com/openmundi/world.db openmundi/world.db] shows a more data-focused way of using Git, or GitHub, for collaborative collection of data. However, it clearly shows the limitations of using a version control system for code on data.<br/>
| + | = Data extraction scripts = |
| | | | |
| − | An upcoming and (technically) promising project is '''[http://dat-data.com/ dat]''', which "is a version-controlled, decentralized data tool for collaboration between data people and data systems." Or, simply: Git for data. It is currently in public beta test, but has come a long way already.<br/>
| + | Feel free to add scripts here, by creating a new wiki page, or placing them on [https://gist.github.com/ Github Gists]. |
| | | | |
| − | == Help finding energy data<br/> ==
| + | *[https://gitlab.tubit.tu-berlin.de/electricity-modeling/crossborder-skript ENTSOE Cross-border Trading Flows Extraction Script by TU Berlin] |
| | + | *[http://www.open-power-system-data.org Open Power System Data] developed a data platform with open source scripts (based on Python and Jupyter Notebooks) for data on generation capacities, power plants, load time-series and weather data. Project running until 07/2017. The public version of the data platform was released 10/2016. |
| | + | *[https://www.electricitymap.org/ electricitymap.org] has parser scrips for various online data sources in its [https://github.com/tmrowco/electricitymap-contrib#real-time-electricity-data-sources GitHub/parsers page] under GPLv3 license. |
| | + | *[https://github.com/OpenEnergyPlatform/open-MaStR open_MaStR] develop to download BNetzA Marktstammdatenregister powerplant data for Germany (AGPL-3.0) |
| | | | |
| − | *[http://opendata.stackexchange.com/questions/tagged/energy http://opendata.stackexchange.com/questions/tagged/energy]<span style="font-size:14.666666666666666px; font-family:Arial; color:#000000; background-color:transparent; font-weight:400; font-style:normal; font-variant:normal; text-decoration:none; vertical-align:baseline">- latest questions on energy data sources</span>
| + | |
| | | | |
| − | <br/>
| + | = Interactive data visualizations = |
| | | | |
| − | == Data extraction scripts<br/> ==
| + | *[https://www.energy-charts.de/index.htm Energy Charts] by Fraunhofer ISE |
| − | | + | *[https://www.entsoe.eu/data/map/ Interactive Power Transmission Grid Map] by ENTSO-E |
| − | Feel free to add scripts here, by creating a new wiki page, or place them on [https://gist.github.com/ Github Gists].<br/>
| + | *[https://transparency.entsog.eu/ Interactive Gas Transmission Grid Map] by ENTSO-G |
| − | | + | *[https://www.electricitymap.org/?page=map&solar=false&remote=true&wind=false Electricity Map] shows the current carbon intensity of electricity consumed/produced. |
| − | *[https://gitlab.tubit.tu-berlin.de/electricity-modeling/crossborder-skript ENTSOE Cross-border Trading Flows Extraction Script by TU Berlin] | + | *[https://www.agora-energiewende.de/en/service/recent-electricity-data/chart/matrix/19.09.2017/19.09.2018/ Recent Electricity Data] by Agora Energiewende (English and German) |
| | + | *[https://globalwindatlas.info/ Global Wind Atlas] by Technical University of Denmark |
| | + | *[https://globalsolaratlas.info/ Global Solar Atlas] by the World Bank Group |
| | + | *[https://www.epexspot.com/en/market-data/dayaheadauction/chart/auction-chart/ EPEX Spot Day Ahead Auction] |
| | + | *[https://wam.rl-institut.de/ WAM - Web Applications & Maps] by Reiner Lemoine Institut |
| | | | |
| | <br/> | | <br/> |
| | | | |
| − | == Data organization ideas<br/> ==
| + | = Data organization ideas = |
| | | | |
| − | *<span id="docs-internal-guid-154daf3c-a714-76df-26f2-8a392ea8c33d" style="font-size:14.666666666666666px; font-family:Arial; color:#000000; background-color:transparent; font-weight:400; font-style:normal; font-variant:normal; text-decoration:none; vertical-align:baseline">A scheme similar to </span>[http://us-city.census.okfn.org/ http://us-city.census.okfn.org/]<span style="font-size:14.666666666666666px; font-family:Arial; color:#000000; background-color:transparent; font-weight:400; font-style:normal; font-variant:normal; text-decoration:none; vertical-align:baseline">might be useful for mapping out what types of data are available where</span>
| + | A scheme similar to [http://us-cities.survey.okfn.org/ http://us-cities.survey.okfn.org/] might be useful for mapping out what types of data are available where. |
This is work in progress and is supposed to become a link list to sources of open energy-related data. We focus on collecting links to data relevant for the modelling of energy and electricity systems and markets. You are welcome to fill in the missing spots and non-existing pages. Also, you are welcome to extend the list of relevant data (e.g. for a European energy system model) that we should collect links to in the future:
