Database - Revision history

Lilly Schoen at 11:59, 6 November 2017

2017-11-06T11:59:06Z

Eva Wiechers: /* Finding and downloading data (web) */

2016-07-26T19:05:08Z

‎Finding and downloading data (web)

Ludwig Schneider at 15:00, 2 June 2016

2016-06-02T15:00:16Z

Ludwig Schneider at 13:47, 20 May 2016

2016-05-20T13:47:32Z

Berit Müller at 13:12, 18 February 2016

2016-02-18T13:12:13Z

Berit Müller at 13:11, 18 February 2016

2016-02-18T13:11:09Z

Berit Müller at 12:38, 18 February 2016

2016-02-18T12:38:26Z

Berit Müller at 12:32, 18 February 2016

2016-02-18T12:32:07Z

Eva Wiechers at 21:14, 8 February 2016

2016-02-08T21:14:37Z

Eva Wiechers at 21:14, 8 February 2016

2016-02-08T21:14:06Z

← Older revision		Revision as of 11:59, 6 November 2017
Line 1:		Line 1:
−		+	{{GlossaryTermTemp
		+	\|Abbreviation=DB
		+	\|Ambiguities=Data bank
		+	\|SubtermOf=Server, Data storage, Data preparation,Data security
		+	\|Definition=0. A database is an organized collection of data. It is the collection of schemas, tables, queries, reports, views and other objects
		+	1. A comprehensive collection of related data organized for convenient access, generally in a computer.
		+	2. A database is a collection of information that is organized so that it can be easily accessed, managed and updated
		+	\|Sources=Wikipedia https://en.wikipedia.org/wiki/Database (0.); http://www.dictionary.com/browse/database (1.); http://searchsqlserver.techtarget.com/definition/database (2.)
		+	}}
	(platform.openmod-initiative.org/data)<br/>		(platform.openmod-initiative.org/data)<br/>

@@ Line 17: / Line 17: @@
 <br/>
 == Finding and downloading data (web)<br/> ==
@@ Line 31: / Line 32: @@
 *evtl. feedin data (?)<br/>
-In addition, the user finds two schemata for every model (called app) in the working database (wdb) on the OpenEnergy Platform, which the model developers create and fill with model specific content.
+In addition, the user finds two schemata for every model (called app) <s>in the working database (wdb)</s> on the OpenEnergy Platform, which the model developers create and fill with model specific content.
 The user searches for certain data (using the data tags named above), assumptions or results (using provided tags like "scenario input" or "scenario results" etc.). She/he uses also the data view functions, which give an overview of the data itself in form of a table or in form of a map or graphs. Doing so, the user might limit the data to a certain geographical area using the map provided for the selection. The user might also download the (selected) data as a text, csv or json file.<br/>
@@ Line 40: / Line 41: @@
 On top of the PostgreSQL database CKAN with a faceted search will be installed.<br/>
 == Creating model specific schemata<br/> ==

@@ Line 2: / Line 2: @@
 (platform.openmod-initiative.org/data)<br/>
-'''Database Rules'''
+The name of the database is OpenEnergyDataBase '''oedb'''.
-[[DatabaseRules|Here]]. Work in progess, feel free to discuss.
+It contains original data (orig), processed data (calc) and results (res).
-'''Finding and downloading data (web)'''<br/>
+== Finding and downloading data (web)<br/> ==
 As before the user finds an overview (like e.g. [https://www.govdata.de/ here]) of data and their sources for different kind of data (replacing the [http://wiki._openmod-initiative.org/wiki/Data wiki page Data]) and an overview of data and their sources for grid data (replacing the [[Transmission network datasets|wiki page Transmission network datasets]]) including data models (sometimes also called grid models). The latter are understood as collections of all relevant information about a grid / network, separated from the tools for network calculations / analysis. ''Are theses tools models, that are presented in fact sheets?''<br/>
@@ Line 29: / Line 35: @@
 The user searches for certain data (using the data tags named above), assumptions or results (using provided tags like "scenario input" or "scenario results" etc.). She/he uses also the data view functions, which give an overview of the data itself in form of a table or in form of a map or graphs. Doing so, the user might limit the data to a certain geographical area using the map provided for the selection. The user might also download the (selected) data as a text, csv or json file.<br/>
-<u>'''Implementation:'''</u>
+=== Implementation: ===
 Assuming that the openmod database includes or links to all relevant data the lists on the wiki pages are no longer necessary. On overview pages the database entries might be ranked by access frequency in order to reduce the overwhelming effect of the database.
@@ Line 35: / Line 41: @@
 On top of the PostgreSQL database CKAN with a faceted search will be installed.<br/>
-'''Creating model specific schemata'''<br/>
+== Creating model specific schemata<br/> ==
 In addition to the activities of non-modellers, modellers of a model can make model specific configurations and store model specific data and assumptions in the xy_app_wd schema and results in the xy_app_res schema.
-'''Downloading data (API)'''
+== Downloading data (API) ==
 Modellers can use the API for data exchange between the databases (oedb and wdb) and the model. Models written in different languages can communicate with the database using a function or module that generates and sends http queries like for instance: “GET openmod.org/data/oedb/schema/table?fields=id,name,date”. Python model might for instance use a function based url lib.
-'''Uploading data (API and Web-GUI)'''
+== Uploading data (API and Web-GUI) ==
 A user can add new rows in the tables of the database as well as new tables. To upload the data she/he can use a function / module, sending PUT queries in http to the database API (see also “Downloading data (API)”), or the data importer (GUI) on the online presence. In the first step the importer can cope with data in csv fomat. The user can submit data as a batch and edit (or delete) the tables manually before she/he sends them. When the user accidentally leave the input site, while data is still pending, she/he will be warned and can cancel the exit.
@@ Line 49: / Line 55: @@
 Before theses changes will be available in the oedb, they are saved in a test databse and must pass tests, which a test server runs regularly. Data, that is made available by OpenPowerSystenData (OPSD), will in the same way be checked and saved in the database.
-'''Changing data'''
+== Changing data ==
 The documentation of changes of the data in the database means saving an enormous amount of information. Therefore, changing the data in the database must be organised very well.
-<u>'''Implementation:'''</u>
+== Implementation: ==
 The API and the data importer will be implemented following the RESTFul paradigms.
 Nightly tests check, whether the new data fulfil the requirements and work with the models and deliver plausible results.<br/>

← Older revision		Revision as of 13:47, 20 May 2016
Line 1:		Line 1:

	(platform.openmod-initiative.org/data)<br/>		(platform.openmod-initiative.org/data)<br/>
		+
		+
		+
		+	'''Database Rules'''
		+
		+	[[DatabaseRules\|Here]]. Work in progess, feel free to discuss.
		+
		+
		+
		+

	'''Finding and downloading data (web)'''<br/>		'''Finding and downloading data (web)'''<br/>

← Older revision		Revision as of 13:12, 18 February 2016
Line 47:		Line 47:
	The API and the data importer will be implemented following the RESTFul paradigms.		The API and the data importer will be implemented following the RESTFul paradigms.

−	Nightly tests check, whether the new data fulfil the requirements and work with the models and ~~delivers~~ plausible results.<br/>	+	Nightly tests check, whether the new data fulfil the requirements and work with the models and deliver plausible results.<br/>

@@ Line 4: / Line 4: @@
 '''Finding and downloading data (web)'''<br/>
-As before the user finds an overview (like e.g. [https://www.govdata.de/ here]; replacing the current open_eGo-project-internal Data Inventory in the Open Database Redmine) of data and their sources for different kind of data (replacing the [http://wiki._openmod-initiative.org/wiki/Data wiki page Data]) and an overview of data and their sources for grid data (replacing the [[Transmission network datasets|wiki page Transmission network datasets]]) including data models (sometimes also called grid models). The latter are understood as collections of all relevant information about a grid / network, separated from the tools for network calculations / analysis. ''Are theses tools models, that are presented in fact sheets?''
+As before the user finds an overview (like e.g. [https://www.govdata.de/ here]) of data and their sources for different kind of data (replacing the [http://wiki._openmod-initiative.org/wiki/Data wiki page Data]) and an overview of data and their sources for grid data (replacing the [[Transmission network datasets|wiki page Transmission network datasets]]) including data models (sometimes also called grid models). The latter are understood as collections of all relevant information about a grid / network, separated from the tools for network calculations / analysis. ''Are theses tools models, that are presented in fact sheets?''<br/>
-In contrast to the existing openmod wiki, the OpenEnergy Platform provides additionally a database, which links to and if possible includes data, which are of general interest for energy system modellers. This database is called oedb (open energy database) and includes original open data collected from different sources as well as processed data. The datasets might be grouped / tagged as follows:
+The OpenEnergy Platform provides additionally a database, which links to and if possible includes data, which are of general interest for energy system modellers. This database is called oedb (open energy database) and includes original open data collected from different sources as well as processed data. The datasets might be grouped / tagged as follows:<br/>
 *climate data
@@ Line 12: / Line 12: @@
 *demand data / energy consumption
 *transmission network
-*flexibility options
+*flexibility options<br/>
 In addition, the user finds two schemata for every model (called app) in the working database (wdb) on the OpenEnergy Platform, which the model developers create and fill with model specific content.
-The user searches for certain data (using the data tags named above), assumptions or results (using provided tag like scenario input or scenario results etc.). She/he uses also the data view functions, which give an overview of the data itself in form of a table or in form of a map or graphs. Doing so, the user might limit the data to a certain geographical area using the map provided for the selection. The user might also download the (selected) data as a text, csv or json file.
+The user searches for certain data (using the data tags named above), assumptions or results (using provided tags like "scenario input" or "scenario results" etc.). She/he uses also the data view functions, which give an overview of the data itself in form of a table or in form of a map or graphs. Doing so, the user might limit the data to a certain geographical area using the map provided for the selection. The user might also download the (selected) data as a text, csv or json file.<br/>
 <u>'''Implementation:'''</u>
@@ Line 22: / Line 23: @@
 Assuming that the openmod database includes or links to all relevant data the lists on the wiki pages are no longer necessary. On overview pages the database entries might be ranked by access frequency in order to reduce the overwhelming effect of the database.
-On top of the PostgreSQL database CKAN with a faceted search is installed.
+On top of the PostgreSQL database CKAN with a faceted search will be installed.<br/>
-<u>'''Use Case of model users and model developers:'''</u>
+'''Creating model specific schemata'''<br/>
-−
-'''Creating model specific schemata'''
 In addition to the activities of non-modellers, modellers of a model can make model specific configurations and store model specific data and assumptions in the xy_app_wd schema and results in the xy_app_res schema.

← Older revision		Revision as of 12:32, 18 February 2016
Line 1:		Line 1:

	(platform.openmod-initiative.org/data)<br/>		(platform.openmod-initiative.org/data)<br/>
−
−	~~<u>'''Use Case of all users:'''</u><br/>~~

	'''Finding and downloading data (web)'''<br/>		'''Finding and downloading data (web)'''<br/>