0% found this document useful (0 votes)
50 views25 pages

ODIM H5 v23-1-25

Uploaded by

gorlicjakub
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
50 views25 pages

ODIM H5 v23-1-25

Uploaded by

gorlicjakub
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 25

EUMETNET OPERA 4

Work Package O4
9/01/2019

EUMETNET OPERA weather radar information model


for implementation with the HDF5 file format
Version 2.3

Original authors: Daniel B. Michelson1, Rafał Lewandowski2, Maciej Szewczykowski2, and Hans Beekhuis3

Additional authors: Günther Haase1, Theodor Mammen4, Dominique Faure5, Mark Simpson6, Hidde Leijnse3, and Daniel
Johnson1
1
Swedish Meteorological and Hydrological Institute, Norrköping, Sweden
2
Institute of Meteorology and Water Management, Warsaw, Poland
3
Royal Netherlands Meteorological Institute, De Bilt, Netherlands
4
German Weather Service, Hamburg, Germany
5
Météo France, Toulouse, France
6
Met Office, Exeter, United Kingdom

Page 1 of 1
EUMETNET OPERA weather radar information model for
implementation with the HDF5 file format

Version 2.3

Original authors:

Daniel B. Michelson1, Rafał Lewandowski2,


Maciej Szewczykowski2, and Hans Beekhuis3

Additional authors:

Günther Haase1 , Theodor Mammen4, Dominique Faure5 ,


Mark Simpson6, Hidde Leijnse3, and Daniel Johnson1
1
Swedish Meteorological and Hydrological Institute, Norrköping, Sweden
2
Institute of Meteorology and Water Management, Warsaw, Poland
3
Royal Netherlands Meteorological Institute, De Bilt, Netherlands
4
German Weather Service, Hamburg, Germany
5
Météo France, Toulouse, France
6
Met Office, Exeter, United Kingdom

on behalf of EUMETNET OPERA

January 9, 2019

Abstract

This document specifies an information model with which the encoding, decoding and management
of data and products from weather radar systems may be facilitated, primarily for the purposes of in-
ternational exchange in Europe. An implementation of this information model is also specified which
makes use of the HDF5 file format developed and maintained by the HDF Group. The result manifests
itself in the form of truly self-describing weather radar data files highly suitable for environments where
data exchange between radars from different manufacturers, different organizations, and/or different
countries is conducted. The ability to include quality information, in the forms of metadata and binary
arrays, is included in a powerful and flexible manner. This information model constitutes an official
second-generation European standard exchange format for weather radar datasets. Because the netCDF
file format is built on HDF5, we also try to ensure that our information model will be compliant with
netCDF.
OPERA Data Information Model for HDF5 i

RELEASE NOTES
Version 2.3, January 2019
The list of how Attributes has been splitted into highly desirable and recommended attributes. User-
definable subgroups (including sub-subgroups) may be added to existing how groups: e.g. how/rsp
(radar signal processor parameters), how/radar_system (general radar system parameters), how/bite
(BITE information), how/monitor (external monitoring information), how/schedule (additional
scheduling information), how/process_chain (processing chain), and how/noden, where n is the
index of a radar node included in a composite. User-definable attributes may be added to these subgroups.
Quantities that have not been subject to any filter or correction have been added. The WIGOS identifier is
now included in the list of source type identifiers.

Version 2.2, March 2014


Table 9 contains additional how Attributes for all objects while new quantities have been added to
Table 17. Some attributes (like TXloss, RXloss, SQI, SNR, VRAD, and WRAD) were assumed to be
horizontally-polarized only. For consistency and clarity attributes that exist in both H and V polarisations
are denoted accordingly. The new ODIM version also contains a minimum specification for a vertical profile
and a polar RHI representation. The latter resembles sector objects and scans. Table 2 includes a new file
object “ELEV” (Elevational object) to accommodate this polar representation. The ELEV object may only
contain the polar product how/product “RHI” as already specified in Table 15. Previously existing
quantities have been marked for deprecation. Optional Attributes with ambiguous polarities have been
removed where new attributes supercede them.

Version 2.1, 26 April 2011


The “simple array” type has been introduced and some attributes have been redefined to use it. The
/what/source attribute has undergone revision. Several new optional how metadata attributes have
been added. Section 7 has been reformulated to also identify prioritized optional how attributes in support
of various applications. Clarifications have been added where ambiguities have been found, and errors have
been corrected. Due to the scope of the changes introduced, this version has been incremented to the next
minor version number.

Version 2.0.1, 21 September 2010


The composite object and product are now officially accepted as OPERA standard. A couple of clarifications
have been made, without changing any of the contents of the information model itself. The UML annex
has been broken out to be its own working document. To mark this, and that we’ve released an updated
document, a minor-subversion has been introduced.

Version 2.0, 9 June 2009


A few inconsistancies between metadata in the tables and the diagrams in the UML representation have been
identified and corrected. These fixes do not motivate a version number increment.

Version 2.0, 1 June 2009


This is the first version of ODIM_H5 to be officially accepted as an OPERA standard. Notwithstanding this
status, the official parts are the definitions of polar scans and volumes, along with all associated metadata.

ODIM_H5 2.3
OPERA Data Information Model for HDF5 ii

All other objects restain their draft status, and will achieve official status in due course, subject to approval
in OPERA.

ODIM_H5 2.3
OPERA Data Information Model for HDF5 iii

Contents
1 Introduction and motivation 1

2 Information model concept 2

3 Definitions 7
3.1 Scalars (integers, real values, and strings) . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1.1 Booleans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1.2 Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1.3 Simple arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.2 Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.3 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.4 Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

4 Metadata attribute specification 9


4.1 Root Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.2 Top-level what Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.3 where Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.3.1 where for polar data Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.3.2 where for geographically referenced image Groups . . . . . . . . . . . . . . . . . 12
4.3.3 where for cross-section data Group . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.3.4 where for vertical profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.4 how Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.5 what Group for Dataset objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

5 Data specification 29
5.1 Polar Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.2 Image Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.3 RHIs, cross sections and side panels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.4 Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.5 Rays and sectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.6 Embedded graphical images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

6 Optional objects 32
6.1 Palettes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6.2 Legends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

7 Mandatory and prioritized optional metadata per product 35


7.1 Polar volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
7.2 Composite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
7.3 Vertical profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
7.4 RHI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

A Derivation of the radar calibration constant 43

ODIM_H5 2.3
OPERA Data Information Model for HDF5 iv

List of Tables
1 Mandatory top-level what header Attributes for all weather radar files. . . . . . . . . . 10
2 File object strings and their meanings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3 Source type identifiers and their associated values. . . . . . . . . . . . . . . . . . . . . . . . 11
4 where Attributes for polar data objects. . . . . . . . . . . . . . . . . . . . . . . . . . 12
5 where Attributes for geographical image data Groups. . . . . . . . . . . . . . . . . . 13
6 where Attributes for cross-section data. . . . . . . . . . . . . . . . . . . . . . . . . . 13
7 where Attributes for vertical profiles. . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
8 Highly desirable how Attributes for all objects. . . . . . . . . . . . . . . . . . . . . . . 15
9 Recommended how Attributes for all objects. . . . . . . . . . . . . . . . . . . . . . . . 16
10 Examples radar “places” and their node designations. . . . . . . . . . . . . . . . . . . . . . 21
11 Radar system abbreviations and their meanings. . . . . . . . . . . . . . . . . . . . . . . . . 22
12 Processing Software abbreviations and their meanings. . . . . . . . . . . . . . . . . . . . . 23
13 Method abbreviations and their meanings. . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
14 Dataset-specific what header Attributes. . . . . . . . . . . . . . . . . . . . . . . . . 24
15 Product abbreviations and their meanings. . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
16 Product parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
17 Quantity (variable) identifiers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
18 Eight-bit Image attributes. Note that these are part of a Dataset object. . . . . . . . . . . 29
19 Example for a hydrometeor classification legend. . . . . . . . . . . . . . . . . . . . . . . . 33
20 Polar volume. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
21 Cartesian image with palette. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
22 Vertical profile. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
23 Range-height indicator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

ODIM_H5 2.3
OPERA Data Information Model for HDF5 1

1 Introduction and motivation

During OPERA’s second incarnation, the goal of work package 2.1 “HDF5 exchange software development”
was to formulate a second-generation information model for use with weather radar data and the Hierarchical
Data Format version 5 (HDF5) file format. This information model has since been adopted and implemented
in operational software both inside and outside OPERA.

This document presents an information model developed for use with weather radar data and products.
Its implementation is also presented, which makes use of the HDF5 file format. HDF5 is developed and
maintained by the HDF Group. All references to attributes, data, types, and so on, are in relation to those
defined and used in the HDF5 documentation. The official HDF5 format documentation should be consulted
for details on this format. This information model is an elaboration of the model presented by COST 717
and used in real-time operations in the Nordic countries1 . Several enhancements have been made based on
operational experience, and based on data quality issues addressed in OPERA II and the EU Voltaire project.
The model presented here is not backwardly compatible with previous versions (version 1.2 and earlier), but
the advantages of its new structure outweigh this disadvantage.

The information model given here is designed from the point of view of trying to harmonize all relevant in-
formation independently of the radar manufacturer and organization from which the data originates. While
the information model is intended to enable the representation of the data and products agreed upon today
within the framework of EUMETNET OPERA, we also look ahead to future needs and have tried to en-
sure that they are appropriately met with this information model. This means that the known products are
supported, as are polarization diversity variables and virtually any quality-related information characteriz-
ing a given dataset. It is also vital to recognize the importance of being able to represent polar (spherical
coordinate space) data, and this is accomodated as well in a flexible manner.

This information model has become the modern European standard, and as such can be used in forthcoming
meteorological standard exchange mechanisms, ie. the WMO Information System (WIS) and its infrastruc-
ture. There is also an important link to OPERA’s operational data centre (Odyssey), in that quality informa-
tion in the information model is being used by Odyssey, thus supporting efforts to improve the quality of
operational products covering the European continent.

In this way, weather radar data data and products are well-organized for data exchange infrastructure, and
we have tried to ensure that the use of the modern technology will facilitate access to and use of European
radar data both within and outside the meteorological community.

1
Michelson D.B., Holleman I., Hohti H., and Salomonsen M., 2003: HDF5 information model and implementation for weather
radar data. COST 717 working document WDF_02_200204_1. version 1.2.

ODIM_H5 2.3
OPERA Data Information Model for HDF5 2

2 Information model concept

The information model attempts to achieve a general-purpose model for storing both individual scans, im-
ages and products, while also allowing series (time and/or space) of these types of information to be stored
using the same building-blocks. The heirarchical nature of HDF5 can be described as being similar to di-
rectories, files, and links on a hard-drive. Actual metadata are stored as so-called “attributes”, and these
attributes are organized together in so-called “groups”. Binary data are stored as so-called “datasets”. The
challenge in formulating an information model is in defining the way in which these general building-blocks
are organized. This section illustrates the information model defined in detail later in this document, in an
attempt to present and clarify its structure.

The first example is given in Figure 1, which shows how a simple cartesian product, for example a CAPPI,
is represented. At the top level, the HDF5 file contains the so-called “root” group. This group is al-
ways there. Following that, three groups contain metadata; these are called “what” (object, information
model version, and date/time information), “where” (geographical information), and “how” (quality and op-
tional/recommended metadata). The data is organized in a group called “dataset1” which contains another
group called “data1” where the actual binary data are found in “data”. This organization may seem overly
complicated at first, but we will see shortly how powerful and necessary this organization is.

Figure 1: Cartesian product.

In terms of the analogy with a file system on a hard-disk, the HDF5 file containing this simple cartesian
product is organized like this:

/
/what
/where
/how
/dataset1
/dataset1/data1
/dataset1/data1/data

In Figure 2, the exact same structure is used to represent a polar scan of data.

When we use this structure to represent a complete scan of data containing three parameters, the seemingly
complicated organization used to store the binary data becomes easier to understand. Figure 3 shows how
“dataset1” now contains three “data” groups, each containing a binary array of data. What we also see is
the way in which the metadata groups “what” and “where” can be used recursively to hold metadata which
are unique to their place in the hierarchy. In this case, “where” in “dataset1” contains the elevation angle of
the scan used to collect the three parameters, and the three local “what” groups contain the information on
which parameter is stored in each. In this way, “dataset1” can contain Z, V, and W in this case, but it can
also contain an arbitrary number of other parameters.

ODIM_H5 2.3
OPERA Data Information Model for HDF5 3

Figure 2: Simple polar scan.

Figure 3: A complete polar scan containing three parameters.

The use of dataset-specific metadata groups is illustrated in Figure 4. Both methods of using “what”, “where”
and “how” are acceptable. In the example on the left, the metadata groups contain attributes which are valid
for every dataset in that group. In the case on the right, the metadata groups contain attributes which are
valid only for that single dataset. In Figure 3, “where” is valid for all three datasets in the group, whereas
“what” is only valid locally. The local metadata always have top priority, so if a mistake is made where a
file contains the same metadata at different levels, the most local level will always take precedence.

Figure 4: Use of dataset-specific metadata groups.

Continuing from the single scan in Figure 3, we can easily extend the same structure every time we
want to add a new scan. This is illustrated in Figure 5. Here, the new elevation angle will be stored in
/dataset2/where, and the information on the parameters stored in the datasets are found in each local

ODIM_H5 2.3
OPERA Data Information Model for HDF5 4

“what”. Note that optional metadata can be added in a “how” group either directly under “dataset2” or
together with each parameter, but this hasn’t been included in this example. A complete volume containing
an arbitrary number of scans, each containing an arbitrary number of parameters, is organized using this
structure. A cartesian volume can be constructed in exactly the same way. Time series of polar or cartesian
products can be constructed in the same way too.

Figure 5: A polar volume containing two scans.

This information model uses the same logic to include the representation of quality information. Figure 6
illustrates how quality data can be added to polar data in a scan containing two parameters. In the same way
that metadata can be applicable to all parameters in the scan, so too can quality information be representative
either generally or locally. In this example, /dataset1/quality1 can contain quality based on beam
blockage since this is applicable to all parameters collected at the same elevation angle. However there may
be different quality metrics which are applicable to each individual radar parameter, and these are stored
locally. In this case, “data1” contains two such local quality metrics which are unique to that parameter,
whereas “data2” only contains one. And we also see that each quality metric can be described using local
metadata groups. The quality metrics should follow the guidelines set out in OPERA II2 .

Using this hierarchical structure, it becomes clear that we now have an information model capable of rep-
resenting a volume containing an arbitrary number of scans/images, each of which can contain an arbitrary
number of parameters which, in turn, can be characterized by an arbitrary number of quality metrics.

The examples provided in this discussion have focused on polar data, but they can also be applied to all the
objects supported in this information model. These objects are polar scans, cartesian images, profiles, RHIs,
cross-sections, side-panels, individual rays, sectors, and even embedded graphics images in an industrial
2
Holleman I., Michelson D., Galli G., Germann U., and Peura M., 2006: Quality information for radars and radar data. OPERA
II deliverable OPERA_2005_19.

ODIM_H5 2.3
OPERA Data Information Model for HDF5 5

Figure 6: A polar scan containing two parameters and associated quality metrics.

graphics file format.

There are additional objects which are complementary to this information model and which are included
since they are quite useful in various situations. One of them is the ability to add a color palette to an 8-bit
dataset. There are standard mechanisms for this in the HDF5 library, and these are used without modifica-
tion. Another object is a legend. This is a useful object when the data in a dataset is classified, discrete, or
just level-sliced. The legend provides the ability to describe which quantitative values are represented by
each qualitative value in the dataset, or which qualitative class is represented by a given value. The legend
is used to tell the user that a certain value in the data represents e.g. “0.1–0.5 mm/h” in the case of a rainrate
product, and e.g. “sleet” in the case of a hydrometeor classification. Both the palette and the legend objects
are illustrated in Figure 7.

Finally, this information model enables the exchange of graphics representations of data. Using the same
structure as that shown in Figure 1, Figure 8 illustrates this.

In the rest of this document, the detail specification of the information model is presented. With this speci-
fication, it shall be possible to read, write, and understand HDF5 files using this information model.

ODIM_H5 2.3
OPERA Data Information Model for HDF5 6

Figure 7: A polar scan containing a color palette and a legend.

Figure 8: An embedded Slovak picture in an industrial graphics file format.

ODIM_H5 2.3
OPERA Data Information Model for HDF5 7

3 Definitions

HDF5 allows data to be stored as Attributes, Datasets, Groups, and user-defined Compound types.
For use here, all types except user-defined Compound are permitted for use. Only Compound types defined
in this document are permitted, in practise the optional legend object described in Section 6.2. In practice,
all of these types are manipulated through the use of general-purpose Node objects, i.e., a Node may be
any of the above mentioned types.

3.1 Scalars (integers, real values, and strings)

Scalar values are stored in Attribute objects and may be strings, integers, or real (floating-point) values.
For the sake of consistency and simplicity, integer values shall be represented as 8-byte long, and real
values shall be represented as double. These can be written to file as native types. They will be read and
automatically returned as the corresponding native type on another platform. Endianness is therefore no
cause for concern.

A clarification is however necessary; a long on one machine may have a different length than that on
another machine. This is because a native long could, perhaps, be returned as an int in some cases. To
prevent this, you should ensure that all integer scalar attributes are written with a length of 8 bytes. In some
cases, this may require writing values of type long long, but it can also be enforced on POSIX-compliant
systems by using the int64_t typedef.

Strings shall be encoded in ASCII or the ASCII representation of UTF-8. Strings should never be terminated
by the application, because they shall be automatically null-terminated by the HDF5 library. This is done by
setting the string termination to STRPAD=H5T_STR_NULLTERM. Other methods of termination available
in the HDF5 library shall not be used. It is also necessary to specify the length of each string (STRSIZE).
Even when H5T_STR_NULLTERM is used, the value of STRSIZE must be incremented manually by one
to ensure that it includes the NULL character. Other methods of specifying the length of the string provided
by the HDF5 library (ie. H5T_VARIABLE) shall not be used.

3.1.1 Booleans

A string is used to store truth value information. The string “True” is used to represent true, and “False”
is used to represent false.

3.1.2 Sequences

A special kind of string may be used to store sequences. A sequence contains comma-separated scalar values
in string notation. For example, a sequence is useful for storing the radar stations contributing to a composite
image, and in storing the elevation angles used in a polar volume of data.

ODIM_H5 2.3
OPERA Data Information Model for HDF5 8

3.1.3 Simple arrays

Sometimes it is necessary to store numeric metadata whose geometry corresponds with the number of az-
imuth gates in a scan, or bins along a ray. The simple array provides this ability. Instead of using a complete
dataset object provided by the HDF5 library (H5D dataset interface), the simple array uses the H5S datas-
pace interface. To simplify this further, only one-dimensional simple arrays are allowed, and to remain
consistent with scalar integer and real value attributes, the simple arrays shall contain either “long” integers
or “double” floats, as defined above.

3.2 Attributes

An Attribute is an HDF5 object used to store a header attribute or data. For our purposes, it is only used
to store header attributes. In order to facilitate the management of so-called “atomic” attributes, ie. individual
values, we use double precision for both integers (long) and floating-point (double) values. Note that the
specification of strings is intended to be case sensitive.

3.3 Datasets

An HDF5 Dataset is a self-describing data object containing an n-dimensional binary array and attributes
describing it. The array type may be any of char, schar, uchar, short, ushort, int, uint, long,
ulong, llong, ullong, float, double on a given platform.

In this document the text formatting of Dataset (in Courier font) means an HDF5 dataset, whereas the
text formatting of dataset (in bold face) refers to the binary data in that dataset.

3.4 Groups

A Group is a top-level object which is used to organize other objects. For example, a Group may contain
a collection of header Attributes, a collection of Datasets, or be the root object for the complete
contents of an HDF5 file.

ODIM_H5 2.3
OPERA Data Information Model for HDF5 9

4 Metadata attribute specification

Header attributes are collected in three Group objects, all of which may be used recursively. Attributes
which describe a given file’s contents and the time and place for which it represents are collected in a Group
called what. Attributes which describe a given file’s geographical characteristics (projection, corner
coordinates, dimensions, scales) are collected in a Group called where. The what and where Groups
both contain mandatory Attributes only. Those Attributes which collectively describe additional
data/product characteristics, such as radar system, chosen algorithm, and quality-related metadata, are stored
in a Group called how. All Attributes found in how may be considered optional, although the use of
those given in this document is recommended. Additional Attributes not specified in this document may
only be stored in the how Group.

Top-level header attributes are collected in what, where and how Group objects located directly under
the root Group ’/’. Each attribute is stored as an Attribute object containing a scalar value. Some
data/products may have Dataset-specific header attributes, in which case a the Dataset and its header
Attributes are collected in lower-level what, where and how Groups which are associated with that
Dataset.

The concept used here is that each Attribute is identified with a string, the idea being that this is a more
intuitive means of organizing data compared to numeric descriptors. It also means that a file may be queried
using the h5dump utility to easily see and understand the contents of a file.

Mandatory header attributes for all files are given in Table 1. Note that all date and time information is for
the nominal time of the data, ie. the time for which the data are valid. (The nominal time is not the exact
acquisition time which is found elsewhere in the file.)

Note that all scans belonging to the same volume must have the same nominal date and time.

Note that all geographical longitude/latitude coordinates are specified with positive easting values east of
the Greenwich meridian and positive northing values north of the equator.

4.1 Root Group

No HDF5 file can exist without this Group, since it is the starting point of the hierarchy. However, in order
to take the first steps towards interoperatility with Climate and Forecast (CF) Conventions, we require an
Attribute in the root Group. This Attribute is called “Conventions” and its value is a string
containing the acronym of the information model being used, and its version, in a format which lends itself
to a directory structure. This is done to comply with the documentation policy maintained by the community
managing CF Conventions3 .

This information model is called the “OPERA Data Information Model for HDF5” which gives the acronym
“ODIM_H5”. Recognizing the history of this information model, we start at version 2.0. The resulting value
for “/Conventions” is “ODIM_H5/V2_0”, and this Attribute must be present in all ODIM_H5 files.

The present version is 2.3, which translates to “ODIM_H5/V2_3”.


3
http://cfconventions.org/

ODIM_H5 2.3
OPERA Data Information Model for HDF5 10

4.2 Top-level what Group

In this section the content of the top-level what is described.

This Group contains mandatory Attributes only which collectively describe a given file’s contents.
These Attributes are given in Tables 1-15.

Table 1: Mandatory top-level what header Attributes for all


weather radar files.

Name Type Format Description


object string - According to Table 2
version string H5rad M.m Format or information model version. “M” is the major version.
“m” is the minor version. Software is encouraged to warn if it
receives a file stored in a version which is different from the one
it expects. The software should, however, proceed to read the file,
ignoring Attributes it does not understand.
date string YYYYMMDD Nominal Year, Month, and Day of the data/product
time string HHmmss Nominal Hour, Minute, and Second, in UTC of the data/product
source string TYP:VALUE Variable-length string containing pairs of identifier types and their
values, separated by a colon. Several pairs can be concatenated,
separated by commas, in the form TYP:VALUE,TYP:VALUE,
etc. At least one identifier according to Table 3 must be specified.
All identifiers assigned to a given radar shall be provided.

Object strings may be any of those given in Table 2. For each Dataset, there may be an accompanying
what Group with information specific to that Dataset, according to Table 14.

Table 2: File object strings and their meanings.

String Description
PVOL Polar volume
CVOL Cartesian volume
SCAN Polar scan
RAY Single polar ray
AZIM Azimuthal object
ELEV Elevational object
IMAGE 2-D cartesian image
COMP Cartesian composite image(s)
XSEC 2-D vertical cross section(s)
VP 1-D vertical profile
PIC Embedded graphical image

Note that a file containing a single scan of polar data may not be represented using object type PVOL; the
object type in these cases must be SCAN. Note also that an RHI polar volume can be represented using a
number of Datasets and the ELEV object.

ODIM_H5 2.3
OPERA Data Information Model for HDF5 11

Table 3: Source type identifiers and their associated values.

Identifier Description Example


WIGOS WIGOS (WMO Integrated Global Observing System) identifier
WMO Combined WMO block and station number in the form A1 bw nnnnn, WMO:02954
or 0 if none assigned. The first two digits represent the block number,
where the first digit A1 is the regional association area and the second
digit bw is the sub-area. Remaining digits are the station number. (Ac-
cording to the WMO, numbers in the form A1 bw nnn are considered
equivalent to the form A1 bw 00nnn). One mandatory way to identify
single-site data. .
RAD Radar site as indexed in the OPERA database. One mandatory way to RAD:FI44
identify single-site data.
PLC Place according to the left column of Table 10 of this document PLC:Anjalankoski
NOD Node according to the right column of Table 10 of this document. One NOD:fianj
mandatory way to identify single-site data.
ORG Originating centre according to BUFR descriptor 0 01 033. One ORG:86
mandatory way to identify composites.
CTY Country according to BUFR descriptor 0 01 101 CTY:613
CMT Comment: allowing for a variable-length string CMT:Suomi tutka

4.3 where Group

In this section the Attributes for the where Group are described. These are different for polar or
cartesian Datasets, i.e. containing Dataset and Dataset Groups respectively.

Note that the use of the where Group is mandatory but that its placement will be at the top level of a
given file, and at a lower level associated with a given Dataset. This is because some attributes are valid
globally (e.g. the coordinates of a radar) whereas others are local (e.g. the elevation angle used for a given
sweep).

4.3.1 where for polar data Groups

This where Group contains mandatory Attributes only which collectively describe geographical and
geometrical characteristics of a given Dataset dataset. Note that the dataset itself is the object containing
the binary data and that where in this context describes that dataset but is located in the corresponding
Dataset which contains all these objects. Section 2 illustrates how these two objects are related to each
other.

Polar data, i.e. raw radar data as a function of azimuth and range, are stored in a Dataset Group, and
polar volume data are stored as a stack of these Dataset Groups. Each Dataset Group contains
azimuthal data from a single elevation.

ODIM_H5 2.3
OPERA Data Information Model for HDF5 12

Table 4: where Attributes for polar data objects.

Name Type Description


lon double Longitude position of the radar antenna (degrees), normalized to the
WGS-84 reference ellipsoid and datum. Fractions of a degree are given
in decimal notation.
lat double Latitude position of the radar antenna (degrees), normalized to the WGS-
84 reference ellipsoid and datum. Fractions of a degree are given in dec-
imal notation.
height double Height of the centre of the antenna in meters above mean sea level.
Dataset specific
elangle double Antenna elevation angle (degrees) above the horizon.
nbins long Number of range bins in each ray
rstart double The range (km) of the start of the first range bin
rscale double The distance in meters between two successive range bins
nrays long Number of azimuth or elevation gates (rays) in the object
a1gate long Index of the first azimuth gate radiated in the scan
Sector specific
startaz double The azimuth angle of the start of the first gate in the sector (degrees)
stopaz double The azimuth angle of the end of the last gate in the sector (degrees)
startel double The elevation angle of the start of the first gate in the sector (degrees)
stopel double The elevation angle of the end of the last gate in the sector (degrees)

4.3.2 where for geographically referenced image Groups

This where Group contains mandatory Attributes only which collectively describe a given
Dataset’s geographical and geometrical characteristics. Note that the dataset is the object containing
the binary data and that where in this context describes that dataset but is located in the corresponding
Dataset which contains all these objects. Section 2 illustrates how these two objects are related to each
other.

The PROJ.4 cartographic projections library4 is a comprehensive means of managing geographically ref-
erenced information which has become a de facto standard. PROJ.4 is being used increasingly throughout
Europe and the world. As a result, the most straightforward way of representing projection information in
radar files is by means of the projection definition string which is used with the library itself. For example,
the arguments used with PROJ.4 to define the Google Maps projection are +proj=merc +lat_ts=0
+lon_0=0 +k=1.0 +R=6378137.0 +nadgrids=@null +no_defs so this is what should be
found as an Attribute in the where Group associated with the Dataset used to store the data geo-
located using this projection. Similarly, the standard “longitude, latitude” projection, also known as EPSG
4326, is defined as +proj=latlong +ellps=WGS84 +datum=WGS84 +no_defs.

Note that PROJ.4 contains a complete set of arguments for specifying a given projection, including false
easting/northing, rescaling of coordinates, choice of ellipsoid (or defining your own), choice of geodetic
datum, and defining oblique (rotated) projections.

4
Originally from the United States Geological Survey, now from MapTools

ODIM_H5 2.3
OPERA Data Information Model for HDF5 13

Table 5: where Attributes for geographical image data


Groups.

Name Type Description


projdef string The projection definition arguments, described above, which can be
used with PROJ.4. See the PROJ.4 documentation for usage. Longi-
tude/Latitude coordinates are normalized to the WGS-84 ellipsoid and
geodetic datum.
xsize long Number of pixels in the X dimension
ysize long Number of pixels in the Y dimension
zsize long Number of pixels in the Z dimension
zstart double Height in meters above mean sea level of the lowest pixel in the Z dimen-
sion
xscale double Pizel size in the X dimension, in projection-specific coordinates (often
meters)
yscale double Pixel size in the Y dimension, in projection-specific coordinates (often
meters)
zscale double Pixel size in the Z dimension (meters)
LL_lon double Longitude of the lower left corner of the lower left pixel
LL_lat double Latitude of the lower left corner of the lower left pixel
UL_lon double Longitude of the upper left corner of the upper left pixel
UL_lat double Latitude of the upper left corner of the upper left pixel
UR_lon double Longitude of the upper right corner of the upper right pixel
UR_lat double Latitude of the upper right corner of the upper right pixel
LR_lon double Longitude of the lower right corner of the lower right pixel
LR_lat double Latitude of the lower right corner of the lower right pixel

4.3.3 where for cross-section data Group

RHI and cross-sections are treated as a special form of cartesian image. The x-dimension of the image
represents the coordinate in the x/y-plane, while the y-dimension describes the vertical coordinate of the
RHI or cross-section. To describe the geographical orientation and extend of a RHI or cross-section a
dedicated set of Attributes in the where Group has been defined. The geographical location of cross-
sections is just given by longitudes and latitudes of start and stop positions (Figure 9). The cross-sections are
thus assumed to be taken along great-circles. In case they are taken along a line in a plane of a geographical
projection, the deviation from the great-circle will be negligible for visualization purposes.

Table 6: where Attributes for cross-section data.

Name Type Description


Common attributes
xsize long Number of pixels in the horizontal dimension
ysize long Number of pixels in the vertical dimension
xscale double Horizontal resolution in m
yscale double Vertical resolution in m
minheight double Minimum height in meters above mean sea level
continued on next page

ODIM_H5 2.3
OPERA Data Information Model for HDF5 14

continued from previous page


Name Type Description
maxheight double Maximum height in meters above mean sea level
RHI specific
lon double Longitude position of the radar antenna (degrees). Fractions of a degree
are given in decimal notation.
lat double Latitude position of the radar antenna (degrees). Fractions of a degree are
given in decimal notation.
az_angle double Azimuth angle
angles simple array Elevation angles, in degrees, in the order of acquisition. Marked for DEP-
of doubles RECATION.
range double Maximum range in km
Cross section and side panel specific
start_lon double Start position’s longitude
start_lat double Start position’s latitude
stop_lon double Stop position’s longitude
stop_lat double Stop position’s latitude

Figure 9: Cartesian MAX product with horizontal (HSP) and vertical side panels (VSP). The red boxes
indicate the start position for each panel.

4.3.4 where for vertical profiles

This where Group contains mandatory Attributes only which collectively describe the geographical
and geometrical characteristics of vertical profiles of horizontal winds and/or radar reflectivity.

Table 7: where Attributes for vertical profiles.

Name Type Description


lon double Longitude position of the radar antenna (degrees). Fractions of a degree
are given in decimal notation.
continued on next page

ODIM_H5 2.3
OPERA Data Information Model for HDF5 15

continued from previous page


Name Type Description
lat double Latitude position of the radar antenna (degrees). Fractions of a degree are
given in decimal notation.
height double Height of the centre of the antenna in meters above mean sea level.
levels long Number of points in the profile
interval double Vertical distance (m) between height intervals, or 0.0 if variable
minheight double Minimum height in meters above mean sea level
maxheight double Maximum height in meters above mean sea level

4.4 how Group

This Group contains recommended and other Attributes which provide additional and complimentary
information which can be used to describe a given Dataset object, for example information related to an
object’s quality. Attributes in how can be added, but they won’t offially constitute the OPERA standard until
they are added to the tables in this information model. Note that the use of the how Group is optional and
that its placement may be either at the top level of a given file, or at a lower level associated with a given
Dataset. If how is found at the top level, then it is assumed that its contents apply to all Datasets in the
file. If how is found at a lower level, then it must be located in the Dataset Group to which its contents
apply. If how exists at both the top and lower levels, then the contents of the local how override the contents
of the top level how.

Note user-definable subgroups (including sub-subgroups) may be added to existing how groups: e.g.
how/rsp (radar signal processor parameters), how/radar_system (general radar system parameters),
how/bite (BITE information), how/monitor (external monitoring information), how/schedule (ad-
ditional scheduling information), how/process_chain (processing chain), and how/noden, where n is
the index of a radar node included in a composite. User-definable attributes may be added to these subgroups
as long as they do not conflict with each other or with existing attributes.

Note as well that there can often be cases where some Attributes apply for all Datasets in a file and
others may be different for each Dataset. In such cases, the Attributes which apply to all Datasets
may be held in a top level how Group and those that change may be held in local how Groups.

For clarity, Tables 8 and 9 containing highly desirable and recommended how Attributes, respectively,
are partitioned such that different partitions contain different product-specific Attributes.

Table 8: Highly desirable how Attributes for all objects.

Name Type Description


Data from individual radars
beamwidth double The radar’s half-power beamwidth (degrees)
beamwH double Horizontal half-power (-3 dB) beamwidth in degrees
beamwV double Vertical half-power (-3 dB) beamwidth in degrees
wavelength double Wavelength in cm
RXbandwidth double Bandwidth in MHz that the receiver is set to when operating the radar
with the above mentioned pulsewidth
continued on next page

ODIM_H5 2.3
OPERA Data Information Model for HDF5 16

continued from previous page


Name Type Description
RXlossH double Total loss in dB in the receiving chain for horizontally-polarized signals,
defined as the losses that occur between the antenna reference point and
the receiver, inclusive.
RXlossV double Total loss in dB in the receiving chain for vertically-polarized signals,
defined as the losses that occur between the antenna reference point and
the receiver, inclusive.
antgainH double Antenna gain in dB for horizontally-polarized signals
antgainV double Antenna gain in dB for vertically-polarized signals
radconstH double Radar constant in dB for the horizontal channel. For the precise defini-
tion, see Appendix A
radconstV double Radar constant in dB for the vertical channel. For the precise definition,
see Appendix A
NI double Unambiguous velocity (Nyquist) interval in ±m/s
Polar data
scan_index long Which scan this is in the temporal sequence (starting with 1) of the total
number of scans comprising the volume.
scan_count long The total number of scans comprising the volume.
astart double Azimuthal offset in degrees (◦ ) from 0◦ of the start of the first ray in the
sweep. This value is positive where the gate starts clockwise after 0◦ ,
and it will be negative if it starts before 0◦ . In either case, the value must
be no larger than half a ray’s width.
startazA simple array Azimuthal start angles (degrees) used for each gate in a scan. The num-
of doubles ber of values in this array corresponds with the value of where/nrays
for that dataset.
stopazA simple array Azimuthal stop angles (degrees) used for each gate in a scan. The num-
of doubles ber of values in this array corresponds with the value of where/nrays
for that dataset.
Quality
NEZH double The total system noise expressed as the horizontally-polarized reflectiv-
ity (dBZ) it would represent at one km distance from the radar.
NEZV double The total system noise expressed as the vertically-polarized reflectivity
(dBZ) it would represent at one km distance from the radar.
LOG double Security distance above mean noise level (dB) threshold value.

Table 9: Recommended how Attributes for all objects.

Name Type Description


General
extensions string Name of the extensions of /what/version
task string Name of the acquisition task or product generator
task_args string Task arguments
continued on next page

ODIM_H5 2.3
OPERA Data Information Model for HDF5 17

continued from previous page


Name Type Description
data_origin sequence If a quantity or quality field has been modified, the
originating quantity or quality field together with the
applied quantity or quality field(s) should be provided,
e.g. [/datasetM/dataN, /datasetM/dataN/qualityP] or [DBZH,
se.smhi.detector.beamblockage].
startepochs double Seconds after a standard 1970 epoch for which the starting time
of the data/product is valid. A compliment to “date” and “time”
in Table 1, for those who prefer to calculate times directly in
epoch seconds.
endepochs double Seconds after a standard 1970 epoch for which the ending time
of the data/product is valid. A compliment to “date” and “time”
in Table 1, for those who prefer to calculate times directly in
epoch seconds.
system string According to Table 11
TXtype string Transmitter type [magnetron; klystron; solid state]
poltype string Polarization type of the radar [single; simultaneous-dual;
switched-dual]
polmode string Current polarity mode [LDR-H; single-H; LDR-V; single-V;
simultaneous-dual; switched-dual]
software string According to Table 12
sw_version string Software version in string format, e.g. “5.1” or “8.11.6.2”
zr_a double Z-R constant a in Z = a Rb , applicable to any product con-
taining reflectivity or precipitation data
zr_b double Z-R exponent b in Z = a Rb , applicable to any product con-
taining reflectivity or precipitation data
zr_a_A simple array Z-R constant a in Z = a Rb , applicable to any product con-
of doubles taining reflectivity or precipitation data
zr_b_A simple array Z-R exponent b in Z = a Rb , applicable to any product con-
of doubles taining reflectivity or precipitation data
kr_a double Kdp -R constant a in R = a K dp b
kr_b double Kdp -R exponent b in R = a K dp b
kr_a_A simple array Kdp -R constant a in R = a K dp b
of doubles
kr_b_A simple array Kdp -R exponent b in R = a K dp b
of doubles
simulated boolean “True” if data are simulated, otherwise “False”
Data from individual radars
rpm double The antenna speed in revolutions per minute, positive for
clockwise scanning, negative for counter-clockwise scanning.
Marked for DEPRECATION.
elevspeed double Antenna elevation speed (RHI mode) in degrees/s, positive val-
ues ascending, negative values descending. Marked for DEP-
RECATION.
antspeed double Antenna speed in degrees/s (positive for clockwise and ascend-
ing, negative for counter-clockwise and descending)
continued on next page

ODIM_H5 2.3
OPERA Data Information Model for HDF5 18

continued from previous page


Name Type Description
pulsewidth double Pulsewidth in µs
lowprf double Low pulse repetition frequency in Hz
midprf double Intermediate pulse repetition frequency in Hz
highprf double High pulse repetition frequency in Hz
TXlossH double Total loss in dB in the transmission chain for horizontally-
polarized signals, defined as the losses that occur between the
calibration reference plane and the feed horn, inclusive.
TXlossV double Total loss in dB in the transmission chain for vertically-
polarized signals, defined as the losses that occur between the
calibration reference plane and the feed horn, inclusive.
injectlossH double Total loss in dB between the calibration reference plane and the
test signal generator for horizontally-polarized signals.
injectlossV double Total loss in dB between the calibration reference plane and the
test signal generator for vertically-polarized signals.
radomelossH double One-way dry radome loss in dB for horizontally-polarized sig-
nals
radomelossV double One-way dry radome loss in dB for vertically-polarized signals
gasattn double Gaseous specific attenuation in dB/km assumed by the radar
processor (zero if no gaseous attenuation is assumed)
nomTXpower double Nominal transmitted peak power in kW at the output of the
transmitter (magnetron/klystron output flange)
TXpower simple array Transmitted peak power in kW at the calibration reference
of doubles plane. The values given are average powers over all transmitted
pulses in each azimuth gate. The number of values in this array
corresponds with the value of where/nrays for that dataset.
powerdiff double Power difference between transmitted horizontally and
vertically-polarized signals in dB at the the feed horn.
phasediff double Phase difference in degrees between transmitted horizontally
and vertically-polarized signals as determined from the first
valid range bins
Vsamples long Number of samples used for radial velocity measurements
Polar data
scan_optimized string Scan optimized for quantity [DBZH; VRADH; etc.]
azmethod string How raw data in azimuth are processed to arrive at the given
value, according to Table 13
elmethod string How raw data in elevation are processed to arrive at the given
value, according to Table 13
binmethod string How raw data in range are processed to arrive at the given value,
according to Table 13
binmethod_avg long How many original data elements in range are averaged to ar-
rive at the given value.
elangles simple array Elevation angles (degrees above the horizon) used for each az-
of doubles imuth gate in an “intelligent” scan that e.g. follows the horizon.
The number of values in this array corresponds with the value of
where/nrays for that dataset. Marked for DEPRECATION.
continued on next page

ODIM_H5 2.3
OPERA Data Information Model for HDF5 19

continued from previous page


Name Type Description
startazT simple array Acquisition start times for each azimuth gate in the sector or
of doubles scan, in seconds past the 1970 epoch. The number of values
in this array corresponds with the value of where/nrays
for that dataset. The required precision is to the millisecond.
Marked for DEPRECATION.
stopazT simple array Acquisition stop times for each azimuth gate in the sector or
of doubles scan, in seconds past the 1970 epoch. The number of values
in this array corresponds with the value of where/nrays
for that dataset. The required precision is to the millisecond.
Marked for DEPRECATION.
startelA simple array Elevational start angles (degrees) used for each gate in a scan.
of doubles The number of values in this array corresponds with the value
of where/nrays for that dataset.
stopelA simple array Elevational stop angles (degrees) used for each gate in a scan.
of doubles The number of values in this array corresponds with the value
of where/nrays for that dataset.
startelT simple array Acquisition start times for each elevation gate in the sector or
of doubles scan, in seconds past the 1970 epoch. The number of values
in this array corresponds with the value of where/nrays
for that dataset. The required precision is to the millisecond.
Marked for DEPRECATION.
stopelT simple array Acquisition stop times for each elevation gate in the sector or
of doubles scan, in seconds past the 1970 epoch. The number of values
in this array corresponds with the value of where/nrays
for that dataset. The required precision is to the millisecond.
Marked for DEPRECATION.
startT simple array Acquisition start times for each gate in the sector or scan, in
of doubles seconds past the 1970 epoch. The number of values in this array
corresponds with the value of where/nrays for that dataset. The
required precision is to the millisecond.
stopT simple array Acquisition stop times for each gate in the sector or scan, in
of doubles seconds past the 1970 epoch. The number of values in this array
corresponds with the value of where/nrays for that dataset. The
required precision is to the millisecond.
Cartesian images including composites
top_heights simple array Layer top heights (meter) above mean sea level
of doubles
bottom_heights simple array Layer bottom heights (meter) above mean sea level
of doubles
angles simple array Elevation angles in the order in which they were acquired, used
of doubles to generate the product
arotation simple array Antenna rotation speed in degrees/s (positive for clockwise,
of doubles negative for counter-clockwise). The number of values in this
array corresponds with the values of how/angles described
above.
continued on next page

ODIM_H5 2.3

You might also like