Demlas Data and Metadata management system

Data & Metadata Publishing Workflow | Python Metadata Viewer | GeoNetwork Metadata Viewer | Analysis | Geospatial Web Services | References

The data and metadata publishing workflow is based on the methodology for automatic creation, validation and publication of geospatial metadata proposed by Kliment et al (2013) and previously implemented in another project Bordogna et al. (2016). The overall idea is schematically represented in Figure 1. 

Figure 1. Schematic representation of the workflow designed to automate the process of GD and metadata publishing within an SDI (source: Bordogna et al, 2016).

Starting from the left hand side, the workflow inputs are historic / newly collected datasets provided by the producers (demlas stakeholders) by moving into predefined folder structure - THEMATIC DATA STORAGE - within the file system of the virtual machine deployed on the university cloud center (SRCE). Initially the historic data have been divided into the four thematic groups as follows: 

1. Cadastral parcels - CP

2. Land cover - LC

3. Orthoimagery - OI

4. Elevation - EL

In order to publish the datasets available within the thematic data folders into a Web GIS server, data publishing flows have to be implemented and incorporated into the workflow. The research reported in Bordogna et al (2016) used Java based open source tool GeoBatch, however they addressed some drawbacks among which was a statement that the project is obsolete and complex to be customized by a project specific needs. Since the needs of the DEMLAS project required additional functionality as support of other Geoserver data source types and automatic creation of projection files for raster images, we decided to develop an in-house DATA PUBLISHING FLOWS using PHP language and Apache web server. The current pilot version of data publication flow reads the TIF files in a predefined data folder (e.g. /opt/demlas/cp/georef), writes project file (tiffilename.prj), creates coverage stores of a worldimage type on Web GIS server deployed by GeoServer opensource application and executes a harvesting task assigned to the data category.

Once the data are published on GeoServer they are made available via open APIs as Web Map Service (WMS), Web Feature Service (WFS) and Web Coverage Service (WCS) to be portrayed, downloaded and used (Cetl et al, 2016). In order to automate the next step of the workflow, which is to create geospatial metadata, an SDI catalogue is introduced. Basic (mostly dynamic) metadata are collected in the catalogue by use of harvesting tasks applied on e.g. WMS service endpoint which bears basic metadata about the service and dataset visualized as WMS Layers via it's capabilities document. Other metadata as e.g. Lineage, abstract, keyword set, etc. are defined in so called metadata templates created for each data theme, e.g. cadastral parcels -> georeferenced maps; using an online excel sheet. These metadata are later integrated into individual records during the harvesting task execution.

Related resources as e.g. georeferenced cadastral and related scanned map sheets are linked via a single metadata record providing the geospatial metadata. In addition, any other related data sources as e.g. additional info to cadastral parcels (KDKO) is described by individual metadata records linked to the root geospatial dataset metadata. 

The metadata are finally made available for querying by users ofgeospatial web using Catalogue Service for Web (CSW) interface. Demlas has tested two pilot solutions, one using GeoNetwork opensource as a back-end catalogue and front-end client and pyCSW as only back-end CSW catalogue. 

Data & Metadata Publishing Workflow | Python Metadata Viewer | GeoNetwork Metadata Viewer | Analysis | References

Data & Metadata Publishing Workflow | Python Metadata Viewer | GeoNetwork Metadata Viewer | Analysis | References

The metadata portal is developed using openlayers 3 open API and provides basic functionalities to search, display, browse and evaluate metadata. 

Searching

User can use either full-text or spatial search defining the Point Of Interest by clicking on the map (Figure 1).

Figure 1 Searching options - full-text and spatial by POI identified by clicking on the map

Displaying, browsing and evaluating

Once the query has been sent to the back-end geonetwork server the modal window appears and display a list of results provided by the server (Figure 2).

Figure 2 Metadata result list

User can can click on a record and display detailed metadata. In addition, if any related metadata available, user can browse through them, e.g. DPKC (PDF) -> Georef dataset (GeoTIFF) -> Data service  (OGC WMS) and vice versa (Figure 3). 

Figure 3 Browsing related metadata records

Downloading data

The last step is to get the actual data. Each metadata record about georeferenced map contains link  to the original dataset in GeoTIFF format published via Web Coverage Service (WCS). In addition, respective scanned dataset can be downloaded via simple HTTP web link . Additional information about cadastral data can be downloaded as PDF files . Each georeferenced dataset can be used in any GIS client application which can work with OGC services.  

Figure 4 Links to download the data

Data & Metadata Publishing Workflow | Python Metadata Viewer | GeoNetwork Metadata Viewer | Analysis | References

SWOT Analysis

Data & Metadata Publishing Workflow | Python Metadata Viewer | GeoNetwork Metadata Viewer | Analysis | Geospatial Web Services | References

Web Map Service

Is a web service specification (WMS) developed by Open Geospatial Consortium (OGC) operating on both raster and vector geospatial data and provides their image representation in a map form.

The MLAS WMS endpoint for the georeferenced cadastral data is available here: http://www.pg.geof.unizg.hr:8080/geoserver/demlas_cp_georef/wms

Web Feature Service

Is a web service specification (WFS) developed by OGC operating on vector geospatial data and provides access to actual datasets features (geometries) and related data types (attributes).

The MLAS WFS endpoint for the homogenized vector cadastral data is available here: http://www.pg.geof.unizg.hr:8080/geoserver/demlas_cp_homo/ows 

The MLAS WFS endpoint for the vector cadastral data is available here: http://www.pg.geof.unizg.hr:8080/geoserver/demlas_cp_vect/ows

Web Coverage Service

Is a web service specification (WCS) developed by OGC operating on raster multidimensional gridded data, extending WMS by formats used for complex modelling and analysis.

The MLAS WCS GetCapabilities endpoint for the georeferenced cadastral raw data is available here: http://www.pg.geof.unizg.hr:8080/geoserver/demlas_cp_georef/ows?service=WCS&version=2.0.1&request=GetCapabilities 

Catalogue Service for Web

Is a web service specification (CSW) developed by OGC operating on collections of descriptive information (metadata) being created and maintained for geospatial data and services.

The MLAS CSW endpoint is available here: https://bolegweb.geof.unizg.hr/gnk_demlas/srv/eng/csw

Data & Metadata Publishing Workflow | Python Metadata Viewer | GeoNetwork Metadata Viewer | Analysis | References

Bordogna, G., Kliment, T., Frigerio, L., Brivio, P. A., Crema, A., Stroppiana, D., ... & Sterlacchini, S. (2016). A Spatial Data Infrastructure Integrating Multisource Heterogeneous Geospatial Data and Time Series: A Study Case in Agriculture. ISPRS International Journal of Geo-Information, 5(5), 73.

Cetl, V., Kliment, T., and Kliment, M.: BORDERLESS GEOSPATIAL WEB (BOLEGWEB), Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLI-B4, 677-682, doi:10.5194/isprs-archives-XLI-B4-677-2016, 2016. 

Kliment, T., Cetl, V., & Tuchyňa, M. Discovery of Geospatial Information Resources on the Web. SDI DAYS.