HEC-DSS

USACE Hydrologic Engineering Center

Davis, CA

Hydrologic Engineering Center

 Topics
 Purpose of HEC-DSS
 What is HEC-DSS and example uses
 Comparison of HEC-DSS to other
database systems
 Data Conventions
 Catalogs (pathname lists)
 HEC-DSSVue primary features
 Getting data into HEC-DSS
HEC-DSS Purpose
 First developed in 1978
for Jacksonville District’s
Kissimmee River Study
and National Hydro-power
Study
 Models for those studies
had to share a large
amount of time series data
 HEC-DSS developed to
manage serial data for
water resources
engineering
 Designed so that both
programs and users could
easily exchange and
 HEC-DSS
Characteristics
 Designed to store and
retrieve serial
engineering/scientific data,
including:
 Time series
 Curve (paired data)
 Gridded (including radar
precipitation data)
 Textual
 Others
 Optimized for storing and
retrieving, not for data
searches. Includes
compression
 Not a relational database
management system

What is HEC-DSS?
 The HEC Data Storage System is
a database system designed to
efficiently store and retrieve
scientific data that is
characteristically sequential and
is typically used in hydrologic
modeling applications.
 A “model-oriented” or “working”
database, written for programs to
exchange data, as well as users.
 Numerous applications program
are interfaced with HEC-DSS,
including the models used in
CWMS and HEC-WAT.
 Interfaces for Fortran, C#, C/C+
+, Visual Basic and Java.
What is HEC-DSS?
 Optimized for serial data, such
as time-series, compared to
relational databases which are
of a tabular style.
 HEC-DSS has been installed on
may types of computers. It is
fully supported on MS
Windows, Linux and Sun
Solaris.
 There are no licenses or fees.
The software is on github.
https://github.com/HydrologicE
ngineeringCenter/hec-dss

 HEC-DSS does not use a

database dictionary or
definition file. There is no
database setup – database files
What is HEC-DSS?
 HEC-DSS consists of
 Libraries of functions, e.g.,
 ztsRetrieve (retrieve time
series)
 HecTimeSeries (Java)
Why HEC-DSS?
 Not a relational database
management system
 HEC-DSS designed for
engineering/scientific
applications
 Optimized for storing and
retrieving data, not for data
searches
 Commercial databases require
data dictionary / data definition
setup (can be quite extensive)
 HEC-DSS has no database setup.
(Just use it - HEC-DSS files
created automatically.)
 Commercial database requires
training and certain level of
“expertise”
 HEC-DSS is simple to use.
Training is optional
 Example Use of HEC-DSS:
CWMS
 CWMS - Corps Water Management System
 For a “Forecast” (modeling run):
 Pertinent observed data copied from the CWMS
Oracle database into a modeling DSS file.
 MetVue reads observed gridded precipitation from
DSS, writes future gridded precipitation back to DSS
 HMS reads gridded precipitation and observed flows,
writes forecasted in-flows and local flows
 ResSim reads forecasted flows, writes regulated flows
 RAS reads regulated flows, writes stages
 FIA reads stages, computes damages
 Results (above data sets) are stored back to CWMS
Oracle database
 Data for plots and tables read from DSS
 Same plots and tables as in HEC-DSSVue
 HEC-DSS Organization
 Data is stored within the file in “blocks”, for
example:
 Time Series – Hourly data stored in months
 Curve (paired) – Stage/Damage w/ single stage axis
and multiple damage axes
 Gridded – Single radar scan
 Multiple blocks may make up a single “data
set”, e.g., 50 years of hourly data is one data
set
 Each block is called a “record”
 A HEC-DSS file can have many records –
 Version 6 file size limit is 8 GB
 Version 7 is unlimited (tested to 250 GB).
 Name of a record is called a “pathname”
 Each pathname within a file must be unique
 Pathnames
 Pathname self-documents the data
 Consists of 6 parts, separated by
forward slashes “/”
 Parts are labeled A – F:
“/A/B/C/D/E/F/”
 Each part can be from 0 to 128
characters long
 A single pathname can be up to 391
characters long
 Example:
 /Sacramento/Red Bluff/Flow/01Mar2015/1Hour/Observed/
Regular-interval time series
pathnames
/A/B/C/D/E/F/
Part Description
A Group, basin, river, region or
study name
B Location or gage name
C Data parameter
D Starting date for block (not 1st
data)
E Time interval (standard)
F Version or additional information
Regular-interval time series
intervals
Interval Block
Length
1Second, 2Second, 3Second, 4Second, 5Second,
6Second, 10Second, 15Second, 20Second, 30Second,
1Minute, 2Minute, 3Minute, 4Minute , 5Minute, One day
6Minute, 10Minute, 12Minute
15Minute, 20Minute, 30Minute ,1Hour One
month
2Hour, 3Hour, 4Hour ,
6Hour, 8Hour, 12Hour
1Day One year
1Week, Tri-Month, One decade
Semi-Month, 1Month
1Year One century
 Regular-interval time
series
 Each record contains a “header”
 Data Units (e.g., Feet, CFS)
 Data Type:
 PER-AVER Period Average (daily average flows)
 INST-VAL Instantaneous (measured stage)
 PER-CUM Period Cumulative (precipitation mass
curve)
 INST-CUM Instantaneous Cumulative (Incremental
precipitation )
 Time offset (e.g., daily data read at 8:00 am)
 Blocks are “standard size” (there are always 365
or 366 values for one year of daily data)
 Missing data flags are used as a place holder
 DSS-6: -901.0
 DSS-7: -3.402823466e+38F (Negative of largest float
 Irregular-interval time
series
 Same as regular-interval, except:
 Date and time stored with each data value
(make data sets much larger)
 Blocks (E parts) are:
 IR-Day
 IR-Month
 IR-Year
 IR-Decade
 IR-Century
 Block sizes are (user) variable length. Try
to limit sizes between 100 and 5000 values
per block
 Pseudo Regular-interval
time series
 Irregular-interval data that looks like
regular-interval, with an regular-interval E
part, except has a “~” at the start.
 For example:
 “Daily” data that will report more frequently
during an event
“~1Day”, uses an IR-Year block size
 Date and time still stored with each data
value (make data sets much larger)
 Paired Data (General
Curves)
Examples: Rating tables, stage-damage
curves

Part Description
A Group, basin, river, region or study
name
B Location
C Two part parameter, separated by a
dash (-)
D Optional Descriptor
E Time Descriptor (often year of
measurement)
F General Descriptor
 Paired Data
 Parameter (Part C) must always have 2
parts:
 First part is the “independent variable”
 Second is the “dependent variable”
 Dependent (second) variable may contain
multiple curves, with each curve having a
“label”
 Example for “Elevation-Damage”, one set
of elevations, and a set of damage amounts
for:
 Residential
 Commercial
Paired Data – Frequency
Curves
 Frequency curves can be stored
using paired data conventions
 First parameter in C part is “Freq”,
e.g., “Freq-Flow”
 Select “Probability” as the X Type
Paired Data – Frequency
Curves
 Other Data Types
 Gridded Data
 Text Data
 General File
 HEC-DSS Catalogs
 A catalog is a list of the pathnames in a
HEC-DSS file.
 DSS Version 6 uses catalog files
 “.dsc” extension for a regular catalog
 “.dsd” extension for a condensed catalog
 A condensed catalog combines time
series D (date) parts to see the date span
for a dataset.
 DSS Version 7 uses an internal sorted
catalog
Pathname Parts (Regular
Catalog) in
HEC-DSSVue
Condensed Catalog in HEC-
DSSVue
 Summary
 HEC-DSS provides an efficient
database for storing and
retrieving serial data for
application and utility
programs.
 It has been incorporated into
the current hydrologic and
hydraulic modeling programs
developed by HEC, as well as
programs developed by other
organizations.