|
|
(40 intermediate revisions by 10 users not shown) |
Line 1: |
Line 1: |
− | This is work in progress and supposed to become a link list to sources of open energy related data. We focus on collecting links to data relevant for the modelling of energy and electricity systems and markets. You are welcome to fill in the missing spots and non-existing pages. Also, you are welcome to extend the list of relevant data that we should collect links to in the future.<br/> | + | |
| + | This is work in progress and supposed to become a link list to sources of open energy related data. We focus on collecting links to data relevant for the modelling of energy and electricity systems and markets. You are welcome to fill in the missing spots and non-existing pages. Also, you are welcome to extend the list of relevant data that we should collect links to in the future. |
| + | |
| + | <br/> |
| | | |
| = Open data sources for energy modelling = | | = Open data sources for energy modelling = |
Line 5: |
Line 8: |
| Open datasets related to energy are listed here by type. | | Open datasets related to energy are listed here by type. |
| | | |
− | == Electricity demand ==
| + | *Demand |
| + | **[[Electricity demand|Electricity demand]] |
| + | **[[Thermal demand|Thermal demand]] |
| + | **[[Transport demand|Transport demand]] |
| + | **[[Industrial demand|Industrial demand]] |
| | | |
− | See [[Electricity demand|Electricity demand]]
| + | *Resources and Potentials |
| + | **[[Weather data|Weather data and availability of renewable energy]] |
| + | **[[Wind geographical potentials|Wind geographical potentials]] |
| + | **[[Biomass potentials|Biomass potentials]] |
| + | **[[Hydroelectricity data|Hydroelectricity potentials]] |
| | | |
− | == Thermal demand ==
| + | *Networks |
| + | **[[Transmission network datasets|Electricity transmission network datasets]] |
| + | **[[Distribution network datasets|Electricity distribution network datasets]] |
| + | **[[Gas network datasets|Gas network datasets]] |
| | | |
− | See [[Thermal demand|Thermal demand]]
| + | *Technologies |
| + | **[[Power plant portfolios|Power plants]] |
| + | **[[Technology assumptions|Generation technology assumptions and projections]] |
| + | **[[Storage technology assumptions and projections|Storage technology assumptions and projections]] |
| + | **[[End-use technology assumptions and projections|End-use technology assumptions and projections]] |
| | | |
− | == Transport demand ==
| + | *[[Demographic and socio-economic data|Demographic and socio-economic data]] |
| | | |
− | See [[Transport demand|Transport demand]]
| + | *[[Environmental data and regulations|Environmental data and regulations]] |
| | | |
− | == Power Plants ==
| + | *[[Historical data and profiles|Historical data and profiles]] |
| | | |
− | See [[Power plant portfolios|Power plant portfolios]]
| + | *[[Energy Scenarios|Energy scenarios]] |
| | | |
− | == Weather data ==
| + | *[[Country-specific targets and policies|Country-specific targets and policies]] |
| | | |
− | Weather data can be used to generate profiles for wind, solar and hydro power plants. Re-analysis or “hindcast” projects use state-of the art weather forecast models with long time series of weather observations. They create consistent series of weather data, often of decades and with a global scope. [http://reanalyses.org/ Reanalysis.org] and [https://climatedataguide.ucar.edu/climate-data NCAR] provide a helpful overview of re-analysis models. Data are usually provided in GRIB or NetCDF format and can be very large in size (100s of GB).
| + | <br/> |
− | | + | |
− | *OPSD compiled a [http://open-power-system-data.org/data-sources#9_Weather_data list of weather data sources] and provides [http://data.open-power-system-data.org/weather_data a script] to download customized subsets of the MERRA-2 dataset.
| + | |
− | *The Hans-Ertel-Centre for Weather Research has published two preliminary reanalysis [https://www.herz-tb4.uni-bonn.de/index.php/projectdata/reanalysisdata data sets] (COSMO-REA6 and COSMO-REA2) containing the most often requested variables that may be used without any restrictions.<br/>
| + | |
− | | + | |
− | *[http://www.cosmo-model.org/ COSMO (Consortium for Small-scale Modelling)], simulation model for weather in Europe, used for the official weather forecasts. The model is free to use for research applications. You may have a chance to get data directly from national weather services or participating universities/institutions and thus avoid having to run it yourself. For Germany, the current resolution is 2.8 km.<br/>
| + | |
− | *The Climate Data Center (CDC) of the German Weather Service (DWD) has puslished [ftp://ftp-cdc.dwd.de/pub/CDC/ a dataset] of<br/>
| + | |
− | **observed parameters from DWD stations,
| + | |
− | **derived parameters at the station locations,
| + | |
− | **gridded fields covering Germany,
| + | |
− | **regional averages for Germany and its federal states,
| + | |
− | **gridded fields covering Europe,
| + | |
− | **global climate station data
| + | |
− | **available at hourly, daily, monthly, annual or multi-annual resolution.
| + | |
− | **The data is useable without restrictions provided the source "Deutscher Wetterdienst" is mentioned.<br/>
| + | |
− | | + | |
− | *[http://rda.ucar.edu/datasets/ds094.1/ National Centers for Environmental Prediction (NCEP) Climate Forecast System (CFS)], worldwide hourly reanalysis weather data, currently 0.2 deg spatial resolution. You need an account to get access to the downloads, but as the data is from a public US institution, it is free.
| + | |
− | | + | |
− | === Projects to turn weather data into renewable power availability time series ===
| + | |
− | | + | |
− | Various projects exist that transform weather data into power availability time series for different solar/wind power plant model types, such as the [http://arxiv.org/abs/1409.3353 Aarhus University RE Atlas] or the [https://github.com/oemof/feedinlib oemof feedinlib] or [http://renewables.ninja/ renewables.ninja].
| + | |
− | | + | |
− | == Wind profiles ==
| + | |
− | | + | |
− | OPSD summaries [http://open-power-system-data.org/data-sources#Wind_and_solar_power_time_series European historical wind generation time series]
| + | |
− | | + | |
− | == Solar profiles ==
| + | |
− | | + | |
− | OPSD summaries [http://open-power-system-data.org/data-sources#Wind_and_solar_power_time_series European historical solar generation time series]
| + | |
− | | + | |
− | == Wind geographical potentials ==
| + | |
− | | + | |
− | See [[Wind geographical potentials|Wind geographical potentials]]
| + | |
− | | + | |
− | == Biomass potentials ==
| + | |
− | | + | |
− | See [[Biomass potentials|Biomass potentials]]
| + | |
− | | + | |
− | == Hydroelectricity data ==
| + | |
− | | + | |
− | See [[Hydroelectricity data|Hydroelectricity data]]
| + | |
− | | + | |
− | == Electricity transmission network datasets ==
| + | |
− | | + | |
− | See [[Transmission network datasets|Transmission network datasets]]
| + | |
− | | + | |
− | == Electricity distribution network datasets ==
| + | |
− | | + | |
− | See [[Distribution network datasets|Distribution network datasets]]
| + | |
− | | + | |
− | == Gas network datasets ==
| + | |
− | | + | |
− | See [[Gas network datasets|Gas network datasets]]
| + | |
− | | + | |
− | == Costs ==
| + | |
− | | + | |
− | "Recent cost estimates for distributed generation (DG) renewable energy technologies are available across capital costs, operations and maintenance (O&M) costs, and levelized cost of energy (LCOE). Use the tabs below to navigate the charts. The LCOE tab provides a simple calculator for both utility-scale and DG technologies that compares the combination of capital costs, O&M, performance, and fuel costs. If you are seeking utility-scale technology cost and performance estimates, please visit the Transparent Cost Database website for NREL's information regarding vehicles, biofuels, and electricity generation."
| + | |
− | | + | |
− | [http://www.nrel.gov/analysis/tech_cost_data.html http://www.nrel.gov/analysis/tech_cost_data.html]
| + | |
− | | + | |
− | TODO: capital, variable and fixed operational and maintenance costs of all generation, transmission etc. technologies.
| + | |
− | | + | |
− | Historical and projected GHG emission costs.
| + | |
− | | + | |
− | == Efficiencies and specific consumptions of end-use technologies<br/> ==
| + | |
− | | + | |
− | TODO: Efficiencies or specific consumptions of end-use technologies (e.g. vehicles [litres/km], etc.)
| + | |
− | | + | |
− | == Demographic and Socio-Economic Data ==
| + | |
− | | + | |
− | TODO: population trends, GDP trends, discount rates.
| + | |
− | | + | |
− | == Environmental data ==
| + | |
− | | + | |
− | TODO: biodiversity, health impacts, water extraction, water use, emission factors.
| + | |
− | | + | |
− | == Historical data ==
| + | |
− | | + | |
− | TODO: historical data on load, generation, emissions, market prices, etc.
| + | |
− | | + | |
− | == Country-specific policies and targets ==
| + | |
− | | + | |
− | TODO GHG emissions targets, renewable share targets, sector-specific targets, subsidy schemes for renewables, criteria for power plant siting (e.g. exclusion zones for wind turbines).
| + | |
− | | + | |
− | | + | |
− | = Wikipedia page =
| + | |
− | | + | |
− | The Wikipedia page [https://en.wikipedia.org/wiki/Open_energy_system_databases open energy system databases] lists some of the more developed portals serving open energy system datasets.
| + | |
| | | |
| = Other lists of energy-related open datasets = | | = Other lists of energy-related open datasets = |
| | | |
− | *[http://enipedia.tudelft.nl/ Enipedia (TU Delft)] is an active exploration into the applications of wikis and the semantic web for energy and industry issues. | + | *The Wikipedia article [https://en.wikipedia.org/wiki/Open_energy_system_databases open energy system databases] lists some of the portals serving open energy system datasets. |
− | **An extensive compilation of links to other data sources can be found at [http://enipedia.tudelft.nl/wiki/Energy_and_Industry_Data_Sets Energy and Industry Data Sets] and [http://enipedia.tudelft.nl/wiki/Electricity_Transmission_Network Electricity_Transmission Network]
| + | *[http://datahub.io/dataset?q=energy datahub.io category "Energy"] has more datasets. |
| + | *[http://energypedia.info/ Energypedia] is a wiki platform for collaborative knowledge exchange on renewable energy and energy access issues in the context of development cooperation. *[http://enipedia.tudelft.nl/ Enipedia (TU Delft)] is an active exploration into the applications of wikis and the semantic web for energy and industry issues. An extensive compilation of links to other data sources can be found at [http://enipedia.tudelft.nl/wiki/Energy_and_Industry_Data_Sets Energy and Industry Data Sets] and [http://enipedia.tudelft.nl/wiki/Electricity_Transmission_Network Electricity_Transmission Network]. |
| | | |
− | *[http://energypedia.info/ Energypedia] is a wiki platform for collaborative knowledge exchange on renewable energy and energy access issues in the context of development cooperation.<br/> | + | *[https://www.europeandataportal.eu/ European Data Portal] |
− | *[http://data.reegle.info/ reegle] is a data provider of country energy profiles, energy statistics and a directory of relevant stakeholders. It also offers the clean energy search and an extensive glossary. There is also an insightful clean energy blog with interesting and up-to-date background information.
| + | |
− | *[http://www.iaee.org/en/EnergyDataLinks/ International Association of Energy Economists (IAEE) Energy Data Links (EDL)] provides a searchable database of energy-related resources
| + | |
− | *[http://www.europeandataportal.eu/) European Data Portal]
| + | |
| *[http://www.iea-etsap.org/web/E-techDS.asp IEA ETSAP energy technology data source (E-Tech-DS)] is a series of four-page technology briefs similar to the [https://www.iea.org/publications/freepublications/ IEA Energy Technology Essentials (filter for "essentials")]. The page contains short technical descriptions of 29 energy related technologies from power production, synthesised fuels, and fossil fuel production. | | *[http://www.iea-etsap.org/web/E-techDS.asp IEA ETSAP energy technology data source (E-Tech-DS)] is a series of four-page technology briefs similar to the [https://www.iea.org/publications/freepublications/ IEA Energy Technology Essentials (filter for "essentials")]. The page contains short technical descriptions of 29 energy related technologies from power production, synthesised fuels, and fossil fuel production. |
| + | *[http://www.iaee.org/en/EnergyDataLinks/ International Association of Energy Economists (IAEE) Energy Data Links (EDL)] provides a searchable database of energy-related resources |
| *[http://en.openei.org/ OpenEI] features a wiki of crowd-sourced energy information and a database of single source data on buildings, energy, efficiency, consumption, demand, potential. | | *[http://en.openei.org/ OpenEI] features a wiki of crowd-sourced energy information and a database of single source data on buildings, energy, efficiency, consumption, demand, potential. |
− | *[http://datahub.io/dataset?q=energy datahub.io category "Energy"] has more datasets | + | *[https://oep.iks.cs.ovgu.de/dataedit/ OpenEnergy Database] was started with the requirements gathered in the first openmod meetings and was developed completely open with support of the openmod community. Input and result data from research of energy system studies are available via an API. The OEDB provides some automated visualisations of the available data. |
− | *[http://www.pfbach.dk/ PFBach.dk], a collection of wind and solar in-feed profiles<br/>
| + | |
| *[http://open-power-system-data.org/data-sources Open Power System Data] has an extensive collection of links to data sources (Electricity consumption, Capacity and generation by fuel, Power plant data, Hydro power data, Prices and related data, Weather data, Wind and solar power time series, Country-specific data portals). | | *[http://open-power-system-data.org/data-sources Open Power System Data] has an extensive collection of links to data sources (Electricity consumption, Capacity and generation by fuel, Power plant data, Hydro power data, Prices and related data, Weather data, Wind and solar power time series, Country-specific data portals). |
− | | + | *[http://www.ourenergypolicy.org/resource-library/ OurEnergyPolicy.org Resource Library] is a free online energy resource library updated weekly. |
| + | *[http://www.pfbach.dk/ PFBach.dk], a collection of wind and solar in-feed profiles |
| + | *[http://data.reegle.info/ reegle] is a data provider of country energy profiles, energy statistics and a directory of relevant stakeholders. It also offers the clean energy search and an extensive glossary. There is also an insightful clean energy blog with interesting and up-to-date background information. |
| | | |
| = Data sharing techniques = | | = Data sharing techniques = |
Line 138: |
Line 69: |
| = Help finding energy data = | | = Help finding energy data = |
| | | |
− | *[http://opendata.stackexchange.com/questions/tagged/energy http://opendata.stackexchange.com/questions/tagged/energy]<span style="font-size:14.666666666666666px; font-family:Arial; color:#000000; background-color:transparent; font-weight:400; font-style:normal; font-variant:normal; text-decoration:none; vertical-align:baseline">- latest questions on energy data sources</span>
| + | You can find a list of the latest questions on energy data sources on StackExchange: [http://opendata.stackexchange.com/questions/tagged/energy http://opendata.stackexchange.com/questions/tagged/energy] |
| | | |
| = Data extraction scripts = | | = Data extraction scripts = |
Line 146: |
Line 77: |
| *[https://gitlab.tubit.tu-berlin.de/electricity-modeling/crossborder-skript ENTSOE Cross-border Trading Flows Extraction Script by TU Berlin] | | *[https://gitlab.tubit.tu-berlin.de/electricity-modeling/crossborder-skript ENTSOE Cross-border Trading Flows Extraction Script by TU Berlin] |
| *[http://www.open-power-system-data.org Open Power System Data] developed a data platform with open source scripts (based on Python and Jupyter Notebooks) for data on generation capacities, power plants, load timeseries and weather data. Project running until 07/2017. The public version of the data platform was released 10/2016. | | *[http://www.open-power-system-data.org Open Power System Data] developed a data platform with open source scripts (based on Python and Jupyter Notebooks) for data on generation capacities, power plants, load timeseries and weather data. Project running until 07/2017. The public version of the data platform was released 10/2016. |
| + | *[https://www.electricitymap.org/ electricitymap.org] has parser scrips for various online data sources in its [https://github.com/tmrowco/electricitymap-contrib#real-time-electricity-data-sources GitHub/parsers page] under GPLv3 license. |
| + | |
| + | <br/> |
| + | |
| + | <br/> |
| + | |
| + | <br/> |
| + | |
| + | = Interactive data visualizations = |
| + | |
| + | *[https://www.energy-charts.de/index.htm Energy Charts] by Fraunhofer IEE |
| + | *[https://www.entsoe.eu/data/map/ Interactive Power Transmission Grid Map] by ENTSO-E |
| + | *[https://transparency.entsog.eu/ Interactive Gas Transmission Grid Map] by ENTSO-G |
| + | *[https://www.electricitymap.org/?page=map&solar=false&remote=true&wind=false Electricity Map] shows the current carbon intensity of electricity consumed/produced. |
| + | *[https://www.agora-energiewende.de/en/service/recent-electricity-data/chart/matrix/19.09.2017/19.09.2018/ Recent Electricity Data] by Agora Energiewende (English and German) |
| + | *[https://globalwindatlas.info/ Global Wind Atlas] by Technical University of Denmark |
| + | *[https://globalsolaratlas.info/ Global Solar Atlas] by the World Bank Group |
| + | *[https://www.epexspot.com/en/market-data/dayaheadauction/chart/auction-chart/ EPEX Spot Day Ahead Auction] |
| + | |
| + | <br/> |
| | | |
| = Data organization ideas = | | = Data organization ideas = |
| | | |
| A scheme similar to [http://us-city.census.okfn.org/ http://us-city.census.okfn.org/] might be useful for mapping out what types of data are available where. | | A scheme similar to [http://us-city.census.okfn.org/ http://us-city.census.okfn.org/] might be useful for mapping out what types of data are available where. |
This is work in progress and supposed to become a link list to sources of open energy related data. We focus on collecting links to data relevant for the modelling of energy and electricity systems and markets. You are welcome to fill in the missing spots and non-existing pages. Also, you are welcome to extend the list of relevant data that we should collect links to in the future.
Open datasets related to energy are listed here by type.