SDI Components

Spatial Data Infrastructures (SDI) are built from a collection of components, some
institutional and some technological. The configuration of an SDI is a network, but as you
have seen in the previous lesson, this network is not only informatic, but also a network of
people: providers of geodata, consumers of geodata, intermediaries (brokers) of geodata,
political guidance,...
Let us continue the introduction to SDI, by looking briefly at its major components. Another
lesson (SDI components 2) will go into further detail on some of the components mentioned
here.
Components of SDI
Below, you will read various opinions about which are the major components of SDI. As
always, the "truth" is somewhere in the intersection of these (and other) various viewpoints.
You will notice that many of the references in this course are from 10 years ago, more or less.
That was the period of huge growth in the field of SDIs, and the basic architecture and
principles have not changed much since then. Our last lesson in this course will take a critical
look at this.

Please take some time to read these in detail:

 Doug Nebert, ex-technical coordinator of the US NSDI, prepared a Powerpoint

presentation in 2004, showing his view on the major components. Do not worry if you are
unfamiliar with some of his terminology (we have not yet covered it in this course):
Nebert's definition is shown graphically in slide 35.
 Section 1.2 from a book on SDI and metadata, by Nogueras and others.
 Professor Ian Masser (ex-GSDI president) has a much more institutional viewpoint of
the 4 main components (see his slide number 8) in this presentation.

 European SDI (INSPIRE) Architecture and Standards Working Group position

paper (please read page 15, figure 2.1). This graphic --showing client software accessing
geodata resources via intermediate services-- has become the de facto standard view of
SDI technology components. You should make a note of it (draw it on a piece of paper).
You will see this diagram in other SDI documents around the world. 

Let us now look in some further detail, at some of the major SDI components in the INSPIRE
diagram.

Major SDI components

Before we go on to study the more tangible SDI components, let us remember one important
thing: the key (absolutely necessary) component in an SDI is collaboration. Without
collaboration, all the high technology in the world will not produce a successful SDI. Note
that in Nebert's slide 35, all the other components are surrounded by partnerships
(collaboration!). Without collaboration, geodata servers will be created and utilised in a
chaotic manner: each service setting and following its own rules, and users running around
trying to learn how each works in comparison to all the others. This was essentially the
situation 20 years ago: specialized map servers appeared on many websites, each server
expected users to download special visualization software to decode special data formats, and
each server worked using its own rules, protocols, and interfaces. This made it almost
impossible to access geodata from 2 different servers from the same visualization software (or
client). In short, there was very little interoperability. In the absence of interoperability,
work is duplicated and money is wasted.

Later we will look at the institutional and political aspects of collaboration, data sharing, etc.
and also at the international standards which serve to control and to guide collaboration. For
now, let us assume that collaboration exists among all the relevant parties.
We turn now to the more tangible, technological SDI components.
As you have seen in the various views on the topic, there are basically 4 technological SDI
components:

1. User access (software/service/system)

2. Metadata catalogues
3. Intermediary (geoprocessing) services
4. Geodata storage (databases)

This is the (top-down) order in which we encounter the components in the INSPIRE
architecture diagram. In reality, however, metadata catalogues should come first on the list.
Metadata (the topic of our Lesson 4) is documentation that describes the geodata source (or
resource), to facilitate its discovery and exploitation. We will see later that metadata is the
technological glue that holds together the SDI.

If we go back a moment, and think about the internal structure of a desktop GIS, we find a
local user interface (what the user sees on the screen), local geoprocessing routines or
procedures, and (mostly) local geodata storage, normally on the user's own hard disk. A
major difference between GIS and SDI is that SDI is, by definition,distributed rather than
local: the user, the data and even the geoprocessing can reside on different computers in
different parts of the world. In fact, as we saw in Lesson 1, the SDI does not compete against
or try to replace GIS, the SDI connects GIS users, to help make them more productive and
to minimize wasted expenditure.
Traditionally GIS users (often public sector workers with minimal budgets) have not utilized
metadata catalogues, for 2 reasons:

1. GIS users often worked within a limited environment (e.g. their own office)
 2. they often utilized a limited type and quantity of data: the geodata in their office,
which is more or less easy to find and organize.

These geodata may be outdated (old), or the wrong type of data, or be full of errors, but at
least they are more or less easy to find.    Or should we say  ?  Beware of using data that
are easily available but are not able to answer your questions!
What about finding (discovering) better, more appropriate, more up-to-date geodata? This
is where SDI, and especially the metadata catalogue, becomes an extremely useful tool.
Metadata is a necessary ingredient for successful SDI, and therefore the metadata catalogue
really should be at the TOP of our list of components. Unless geodata providers (anyone or
any organization that is willing to share data) publish documentation about their data--what it
is, where it can be found, what are its conditions for access and use-- then the user access
component cannot assist the user in discovering the geodata that is offered. Imagine trying to
find the phone number of a certain Chinese restaurant in your town, without a telephone
directory and not remembering its exact name! The metadata catalogue is somewhat like
theyellow pages directory: without it the only solution is normally phoning to colleagues,
friends, and colleagues' friends, to ask if they know where to find geodata of type A or B...
Unfortunately in many organizations this is still standard practice (word of mouth), in the
absence of proper SDIs.

As you read through the materials we have suggested, you will find (or have found already)
that the "experts" afirm that successful SDIs begin by creating metadata, and then by
publishing the metadata in Internet-accessible catalogue services.
Question: Isn't that last statement contradictory with the first sentence of this section,
that collaboration is the most important component (and therefore should come first)?
No. Many SDI experts will say that EVEN BEFORE reaching collaborative agreements (these
agreements may take several years to finalize), geodata providers (including individual users)
can, and should, begin creating metadata to describe (to document) those geodata that they
may wish to share in the future. Anytime you create geodata that MIGHT be useful to
someone else, you should also document it.
This is simply good practice: properly documenting data resources increases their
value! It makes them easier to understand, organize, categorize, and (ultimately) easier to
share or to sell on the marketplace. Remember that the person looking for that data next year
might be YOU, and so you want it to be documented and easy to find. You don't want to go
back to your geodata database and find a bunch of undocumented files called "testmap458".
Question: But what if there is not yet an agreement about how to create metadata, or on the
format of the metadata? Does everybody start creating metadata in his/her own format?
No! There are certain international standards that define geo-metadata: its format, its
elements (for example: title, date of creation, distribution format...), etc. EVEN BEFORE a
particular group (for example, a regional government) finalizes its collaborative agreements,
there are certain general rules that are unavoidable and unambiguous. In the area of geo-
 metadata the rule is clear (as of 2004): metadata should conform, as much as possible, to ISO-
19115 and related standards. The next lesson (Lesson 3) is dedicated to this, and other related,
international standards.
Reviewing: collaboration and metadata are VERY HIGH on our list of importance
among the SDI components. So high, in fact, that they have their own lessons dedicated
to them.

User Access Component

User Access component
You have seen in Nebert's NSDI Powerpoint presentation and in the position paper by the
INSPIRE working group, the overall architecture, or structure, of the SDI can be pictured as
having three tiers or levels. 

The concept of 3-tier architecture (client/ business logic /server) for information systems will
be very well known by students who work in informatics-related fields. If you are not
familiar with this term, then read more here. Later we will look briefly at a newer model, that
represents the same components as a triangle of services instead of as 3 tiers.

Let us look at the SDI application of the 3 tiers. At the top level we find the human user. We
assume that this user wants to discover and access geodata, for exploitation in some
professional work. This sounds quite simple and obvious, but we should stop and think here:
Are the geodata users working locally with their own data (in a verticalmanner) or are they
sharing data with other users (in a horizontal manner)? SDI permits geodata users to broaden
their horizons, and to work in a more horizontal manner, however this will only be possible
when, and if, their organization (their boss) allows them to do so. In many organizations there
are rules against formal sharing data, or there are network firewalls (barriers) in place that do
not allow users to access certain Internet sites (music/video download sites for example). In
many cases this forces users to do their data sharing unofficially, often from home.
(Remember that here we are not talking about sharing commercial material such as a music
CD; in general much of the geodata that is utilised has been collected and paid for thanks to
public tax money, and therefore should be available, in theory, without extra cost other than
perhaps the cost of distribution.) Data sharing is not as simple as it sounds: there are still
many institutional, legal, and even psychological barriers that do not allow sharing to happen
in many, many situations. Some SDI experts believe that this important problem will only be
solved when there is a generational change in management personnel: when the older
(analogue-mind) managers retire and younger (digital-mind) managers take over.  You
students are the new generation.
But for now, let us assume that users are allowed to share data. How, then, do typical
users access and work within the SDI?
Today, the most common methodology is to access an SDI by visiting a "geoportal"on the
web. You have already done this in Lesson 1: you visited and queried geodata on
the INSPIRE European geoportal, and on Data.gov.  There are many others around the
world; according to Professor Ian Masser already in 2002 more than half of the
180+ nations claimed* to have some type of a national SDI. And then at lower administrative
levels, many regional governments, provinces and even some cities have implemented SDIs,
many of them publicly accessible through a geoportal.
(* Note: We say claimed, because in many cases governments have attempted to rename
existing initiatives, such as for modernizing cartographic production, as SDI. You have seen
the definitions of SDI, and now you should be able to recognize and critically judge the key
differences. Therefore, perhaps Prof. Masser was a bit too generous in recognizing 120
national SDIs being created in 2002...however perhaps we have this many now in 2017.)
Back to user access via geoportals...

Geoportals have become quite a popular topic and even big business, as is the case for many
other applications on the web. The publisher ESRI Press produced a book dedicated to
"Spatial Portals". It is marketing-style book (not technical) that is very easy to read and which
describes several geoportal projects around the world.

But there is no real need to buy books on the topic: searching for "geoportal" on Google will
provide many links to real, on-line geoportals. Surprisingly, some of the first resources listed
by Google (depending on where you are when you search; Google search does work the same
way for everybody) are from Spain, and also from Romania, Colombia, New Zealand...and of
course international examples such as INSPIRE and GEOSS.
If you are interested in visiting two Spanish geoportals at national and regional
levels (user interfaces may be changed to English or regional languages) then please do visit
and test the capabilities of www.idee.es and www.geoportal-idec.net
When you visit these you will see how geoportals play a key role: attracting people to the
SDI project, or to the SDI community so to speak. The geoportal is the virtual forum, a
"place" where members of the geodata community learn what is new, exchange ideas,
advertise services, etc.

After you have visited these, or other similar, geoportals, let us now take a critical look at
geoportals in comparison to another methodology of accessing SDIs.
Many people have been told that the geoportal is THE single entrance into the SDI. This is
not true. Let us go back to an earlier assumption (at the beginning of this section): the typical
geodata "user wants to discover and access geodata, for exploitation in some professional
work". That is to say, the typical user is not so much interested in visiting the portal (like
visiting a shopping centre). The user is interested in getting results from his/her GIS, and in
most case one or more "data layers" (or themes) is missing. The user wants to access geodata,
not access geoportals per se. This is similar to the case of other infrastructures. We are not
interested in visiting a specialized water market website to purchase our water: we want to
open the tap in our home and receive water...and then at the end of every month or trimestre
we transparently pay our water bill via electronic transfer. That is how an infrastructure
should work.
Let's take a look at various methods for accessing the same geodata, taking as our example
climatological data:

1. Indirect access via data provider website.

2. Indirect access via Geoportal catalogue.
3. Indirect access via Geoportal map viewer.
4. Direct access from GIS client.

Accessing climate data

Example: Methods for user access to climate data

We begin with the assumption that a GIS user is analyzing some phenomenon within
the Iberian penninsula, and this user wishes to find a data theme (layer) about
climatology (perhaps also elevation), just to visualise these data in comparison to her other
data themes.

Google?

 These days, a first attempt to find these data (or nearly anything else) often begins by
querying Google! If we search  "datos climatologicos españa" for example, we encounter
more than 500.000 possible resources, however most of them do not bring us to useful
data (well, maybe some PDFs) even though we searched for "datos". This is typical,
because geodata are not normally stored in formats or locations that are recognized by
Google. KML is an exception but that is a graphic display format which is not of great
value to professional GIS users.

Indirect access via specific portal

 Perhaps a colleague informs us of a nice online Atlas website offering these data (if
we had known that it was called "Atlas of ..." then we might have found it with Google).
It is located at the Universitat Autonoma de Barcelona. Let's
visit:http://opengis.uab.es/wms/iberia/mms/index.htm
 There you will be able to choose a view of a specific month of the year, say
November, visualize pluviometria (rainfall), and zoom to a specific location; let's choose
the province of Valencia from the menu at the bottom of the page.
 If you click on the (M) button next to November you can download these data for use
in the MiraMon GIS. This is nice, if you are user of MiraMon... In my case the system
said I needed to install the Miramon viewer.

 For nonusers this specific web portal is ok but not so useful because they are
intending to visualize these climate data in their own GIS environment, not outside in a
separate window and in another format. So if I'm using ArcGIS or QGIS this does not get
me the data within my GIS.
Indirect access via general geoportal catalogue

 Geoportals normally include a catalogue search service, that perhaps the user can use
to search for "Atlas climatico iberica".
 You can return to www.idee.es and www.geoportal-idec.net and try these searches.
The data catalogues should offer you additional information (metadata) about these
geodata, and also perhaps offer to connect you to the map viewer to visualize the geodata.
 These catalogs do not have strong "free text" search capabilities (as do Google or
Bing); perhaps you´ll need to search on the atlas publisher "UAB". But what if you did
not know UAB was the publisher...? I searched in IDEE for "climatologico" and  for
"climatologia" and I got zero results. So on that geoportal I will not find the UAB Atlas.

Indirect access via general geoportal map viewer

 The user knows that the climate atlas was prepared by scientists in the Cataluña
region, so perhaps these same data will be available on the Cataluña SDI (IDEC). Let's
have a look: http://delta.icc.cat/idecwebservices/mapawms/index.jsp?
key=idec&language=en 
 We do not find the same climate data, but there is an option to Add WMS+ Click
there and then paste this server address: http://www.opengis.uab.es/cgi-
bin/iberia/MiraMon5_0.cgi
 Then you may select the digital terrain model, called "Models digitals de terreny".
You'll need to click on "Add" and then select the new layer (in the lefthand legend) to
visualize it.

 Accessing these data via the geoportal is nice, because it allows us to mix (overlay)
these data with other data themes in the same window. (note: the lack of transparency of
certain layers will complicate this task; some viewers allow transparency to be controlled
for each layer). WMS (web map service) is a special type of standard map service that we
will see in further detail in another lesson.
 There remains a problem however: these data are still not inside the GIS. They are
external, for separate viewing.

Using a more general map viewer

 Perhaps the user does not wish to utilise an entire SDI geoportal, but simply wishes to
add (and visualize) some selected geodata to other layers.
 First the user would access such a general-purpose viewer. Let's try the ArcGIS
Online map viewer. Go ahead and sign-in to your account (top right button).
 Now click on the Add+ button on the left. Then Add layer from Web. and then
the WMS option.
 Copy/paste the WMS address for the Climate Atlas:http://www.opengis.uab.es/cgi-
bin/iberia/MiraMon5_0.cgi
 It may take a few seconds for the server to transfer the image to your viewer. By
default it shows only municipal boundaries, but you can change the request to see other
layers.
 Select the new service from the Table of Contents on the left. There is a long list of
layers that it can show, but let's select the penultimate: "Models digitals de terreny"
(same one we saw earlier).
 Pass your mouse over the title of the service (Atlas climatic) and three blue dotswill
appear, offering you some options. Choose transparency and use the slider bar to play
with that property.
 Now you have seen how a generic viewer can access any valid WMS service coming
from anywhere around the world. We will learn more about these services in the near
future.

Direct access from GIS

 Let us remember again that the user of geodata is looking to discover new or better
geodata, for analysis (visual or otherwise) within his/her GIS. Well then why not use the
GIS software to visit remote servers and visualize the geodata directly?

 Start ArcGIS Desktop (ArcMap) and go to the Catalog window (on the right side).
There you click on GIS Servers and Add WMS Server.
 Paste the http://www.opengis.uab.es/cgi-bin/iberia/MiraMon5_0.cgi server address
into the box and click on Get Layers. Select the "Models digitals de terreny" as before.
 Right-click on that server in the list, and select Create Layer. Name it "Climate
Atlas DTM".
 Now the layer can be added using the Add Data button as usual in ArcMap.

Revisiting 3-tier architecture

Revisiting the 3-tier architecture, at the top we have seen two types of user application
clients: web browser access and desktop GIS access. Yes, a desktop GIS can be considered a
client.

At the bottom of the architecture diagram we have geodata storage/service: essentially servers

containing the raw geodata files (GeoTIFF, shapefiles, DGN, etc.) or spatial databases such
as Oracle Spatial, PostGres, or ArcGIS Server, storing geodata as features.

We will not say very much about the servers here. The key tier for the future of SDIs is
the middle tier of intermediary services, also called middleware. It is here in the middle of
the SDI, where technology assists the user in gaining access to data without needing to know
the specific details regarding where it is located, its format, etc. We might say that the Google
search engine is a type of middleware: in between the web browser and the huge collection of
web resources stored on hard disks across the world.
The intermediary tier of SDI was also described earlier as business logic (the main data
processing routines, in between the user interface and the servers) and some technology
people also call this the application tier.

Which are the principal intermediary services in the SDI? You have already seen and in fact
used some of them.

 Data catalogue
 Map server/viewer
 Geodata query filter and download

 Gazetteer
 Geodata transformation

 Geoprocessing

Data catalogue

The data catalogue service is a software application (middleware) that facilitates

queries from a single client software, to multiple databases dispersed around the world. Data
catalogues are created according to international standards, helping to assure that the geodata
are described (metadata) in a harmonized manner. Catalogue servers that are more
sophisticated can pass queries on to other associated (federated) catalogue services, creating
what are called cascading queries. This is like saying "I will look in my data collection but
also I will pass the query to other people who also might have what you are looking for." The
main catalogue service collects all the responses and then sends them back to the client as
though they were from a single service. Many GIS vendors are now offering catalogue
services, such as ESRI Geoportal Server (now open source), Ionic Red Spyder Catalogue,
Deegree iGeoPortal, CadCorp SIS, con-terra Catalog server, Galdos Indicio, GeoNetwork,
etc. 

For more details on data catalogue services you should read chapter 4 of the GSDI
Cookbook. Do not worry about the details; just have a look at where catalogues fit into the
SDI.

Map server/viewer

The most common SDI service, by far, is the web map viewer service, following the OGC
Web Map Service (WMS) interface specification. This specification (also an ISO
international standard) helps to assure that clients and servers from multiple vendors will be
able to interact with minimal problems (that is to say, with maximuminteroperability). All of
the geoportals you have visited so far contain user interfaces to access catalogue services and
also web map services. Web map services receive user queries of the type: "which layers are
available?" and "get me a map of these data, in this format". The servers then access the
geodata features from the data server, rasterize them, and then send a bitmap (normally GIF
or PNG formats) to the WMS client. Remember that WMS visualizes raster images (like
photos) of the data, not the underlying data themselves. These WMS data can be used for
context and as background images for other more operational data layers on top, or merely
for visual analysis of one layer on another.

For more details on web map viewers/services, read chapter 5 of the GSDI Cookbook.

Geodata query filter and download (WFS)

The Web Feature Service (WFS) interface allows user clients to send queries to feature
(vector geodata) databases and realize sub-selections. We should caution the reader: WFS is
not a vector data download service...it is a query filter service. WFS services reply with
XML-encoded geodata, in the GML format. It is the job of the client software (on the users
desktop or mobile device) to decode that GML and convert it to vector graphics on the
screen. GML is a verbose language, meaning that its contents can be quite large and
cumbersome to process. But for many applications GML's openness, in terms of being a
common standard format, outweighs this size/processing issue.

For more information on WFS see its entry in wikipedia.

Geodata transformation

In the path between data server and the client (when a user receives data) it is often necessary
to transform the data en route. The most common geodata transformation service is
the Coordinate Transformation Service (CTS) as defined by OGC. Transformation services
might also be used to reformat data while being passed from one service to another: we will
see more on this topic in the second lesson on SDI components. These days many SDI
projects and Online GIS products use the commercial Feature Manipulation Engine (FME) as
an internal transformation component.
Geoprocessing

The INSPIRE diagram shows that the intermediate (middleware) services are or may be
"geoprocessing" services. This assumes that geoprocessing routines (algorithms) can be
encapsulated in packages (components) in a standardized way, to allow for standard clients to
access these geoprocessing services, and for these services to access geodata servers.
Examples of geoprocessing services might be the calculation of buffers, overlay of two
layers, or shortest path calculation on a network.

For examples of how OGC's Web Processing Service (WPS) specification is being applied by

researchers, see this publications list. Can you find the reference by Laura Diaz and the team
at UJI? You will notice that the academic work on WPS more or less peaked in 2009, and not
much has happened since then.

In order to not overly elongate this lesson, we have left the discussion of intermediate
services at a basic level. In a coming lesson called "SDI components 2" we go into more
depth; this, after you have read about the relevant International Standards, in the next lesson.

END LESSON 2