Relational Data to XML:

A Users Unexpected Journey

Paul Murray
Westpac New Zealand Ltd
Session Code: D02
Wednesday, September 11 13:00 to 14:00 | Platform: Cross Platform

Click to edit Master title style

OBJECTIVES
Describe reasons why conversion of relational data to XML may
be considered.
Detail situations when conversion appears to be a candidate
for consideration.
Describe how to convert relational data to XML for

A single column on a table

For multiple columns on a table
For data on multiple tables whether single or multiple columns
How to handle data where nulls exists

Detail what has been discovered / achieved so far.

Click to edit Master title style

Acknowledgements
Phil Grainger of BMC Software
From his education seminar, DB2 9 for z/OS In Depth presented at
IDUG Australia 2011, came the idea about using XML.
And for his introduction of me to

Dr Matthias Nicola from IBM

His presentation entitled - Publishing Relational Data as XML, from
IDUG North America 2006. This gave me the base knowledge to
develop the solution used in the situation described in this
presentation. (The presentation is available in the IDUG Website)

Robert Mala an Independent Database contractor

For his presentation entitled - Getting to grips with Complex SQL, from
IDUG Australia 2008. Again assisted with developing the solution with
relation to using and coding Common Table expressions.

Click to edit Master title style

Disclaimer
The following presentation is the views and opinions of the
author / presenter. It in no way represents the views of
Westpac NZ Ltd or IBM.
The facts presented as part of this presentation were observed
in a specific environmental configuration that existed within
Westpac NZ Ltd and are factual only to that environment.
They should by no means constitute a guarantee of results
achievable in any other environment and as such are therefore
indicative only.
Other observations are factual events that the author /
presenter encountered.

Click to edit Master title style

Agenda

Why convert to XML.

When you may wish to convert.
How to convert to XML. (The Basics)
Dealing with NULLS.
Outcomes achieved in Westpac.
Additional Suggestions / Observations and XML
syntax.
Conclusion.
Questions.

Click to edit Master title style

WHY?
6

Click to edit Master title style

Curiosity at work
7

Click to edit Master title style

Reasons Why convert Relational Data to XML.

Understanding.
Pseudo Standard.
Flexibility.
Re-use.
Performance.

Click to edit Master title style

Why convert to XML - Understanding

This reason may come into play in two ways;
1. XML has been widely taught from several years now and as
such has numerous people who can read / write / develop it.
This is turn has lead to a desire from developers and
architects who have this understanding, to see it utilised
within the organisation they work for.
2. When XML is well formed it can be a self describing format.
E.G.
<AccountBalance>100.00</AccountBalance> is more
easily understood than having 100.00 sitting somewhere
within a record in a file.
E.G. Record 1ACCT 1234567890
100.00O

Click to edit Master title style

Why convert to XML Pseudo Standard

Due to widespread understanding of and potential easy of
understanding of the meaning of the data within XML, XML is
being accepted as a Pseudo Standard as a data transmission
format between applications / platforms and even
organisations.
Even to the stage whereby external organisations to yours may
dictate its usage as the format they must receive in cross
organisation data communications.

10

Click to edit Master title style

Why convert to XML - Flexibility

XML is a very flexible data format.
Need a new piece of data just add a new tag in.
If a data element is no longer needed then the tag can be removed by
not populating it or removing it from showing in the XML generated.
XML can also be versioned which means multiple versions of the same
XML document can be maintained to allow for changes in the elements
that the document contains.
From a database point of view changing the elements within an XML
Document, which is akin to changing columns in a table, can be done
without requiring an actual table change being made, if stored as XML
within an XML column on the database.

11

Click to edit Master title style

Why convert to XML Re-use.

By converting current relational data used in data selection
processes to transform data to XML first this can allow current
processes that create and maintain the data to still function as
is, thus avoiding mass changes to the current environment.
This of course can allow for a slower more incremental set of
changes to be made, reducing risk and allowing a more
targeted approach to be taken.
It also allows for the adoption of a hybrid methodology where
relational data and XML coexist in the same database(s) and
are utilised where they are most suited.

12

Click to edit Master title style

Why convert to XML - Performance

Transforming relational data to XML can provide performance
gains by
1.
2.
3.

Reducing Calls to the database.

Reducing CPU consumption.
Reducing Elapsed Time taken.

13

Click to edit Master title style

WHEN
14

Click to edit Master title style

Conversion to XML may be an option when

The data is of a hierarchical nature. E.G.
A Family Tree.
An Organisational Chart.

The nature or structure of the data looks like a

A squat pyramid.
A Star or Snow Flake schema.

15

Click to edit Master title style

HOW The Basics

16

Click to edit Master title style

The nature / structure of XML

For an XML Document
to be well formed it has
to relate back to a single
point or root element.
i.e. You are trying to
create Mount Everest
(from peak to base) not
the whole Himalayan
Mountain range.

17

18

Click to edit Master title style

How to convert to XML - (Base Data Definition)

EMP
EMPNO

FIRSTNME

LASTNAME

SEX

HIREDATE

CHAR(6)

VARCHAR(12)

VARCHAR(15)

CHAR(1)

DATE

000010

CHRISTINE

HAAS

1965-01-01

000020

MICHEAL

THOMPSON

1973-10-10

EMPPROJACT
EMPNO

PROJNO

EMSTDATE

EMENDATE

CHAR(6)

CHAR(6)

DATE with Nulls

DATE with Nulls

000010

MA2100

1982-01-01

1982-11-01

000010

MA2110

1982-01-01

1983-02-01

000010

AD3100

1982-01-01

1982-07-01

000020

PL2100

1982-01-01

1982-09-05

Click to edit Master title style

How to convert to XML

The basics functions
XMLELEMENT
Creates a single XML element within an XML Document.
Note that every XML document you create needs at least 1 of these.

XMLFOREST
Allows the creation of multiple XML elements within a XML Document
with one XML command.

XMLAGG (XML Aggregation)

Combines XML Elements or XML Documents together.

19

Click to edit Master title style

XMLELEMENT The syntax

20

Click to edit Master title style

How to convert to XML

(Single Column from a Single Table)

SELECT XMLELEMENT(NAME First, FIRSTNME)

FROM DSN8910.EMP
WHERE EMPNO = 00010
;

Result
<FIRST>CHRISTINE</FIRST>

21

Click to edit Master title style

How to convert to XML

(Single Column for multiple rows from 1 table)
SELECT XMLELEMENT(NAME First, FIRSTNME)
FROM DSN8910.EMP
;

Result
<FIRST>CHRISTINE</FIRST>
<FIRST>MICHAEL</FIRST>

22

Click to edit Master title style

How to convert to XML

(Multiple Columns for 1 row from 1 table Option 1)
SELECT XMLELEMENT(NAME Name,
XMLELEMENT(NAME First, FIRSTNME),
XMLELEMENT(NAME Last, LASTNAME))
FROM DSN8910.EMP
WHERE EMPNO = 000010
;

Result
<NAME><FIRST>CHRISTINE</FIRST><LAST>HAAS</LAST></NAME>

23

Click to edit Master title style

XMLFOREST The syntax

24

Click to edit Master title style

How to convert to XML

(Multiple Columns for 1 row from 1 table Option 2)
SELECT XMLELEMENT(NAME Name,
XMLFOREST(FIRSTNME as First,
LASTNAME as Last))
FROM DSN8910.EMP
WHERE EMPNO = 000010
;

Result
<NAME><FIRST>CHRISTINE</FIRST><LAST>HAAS</LAST></NAME>

25

Click to edit Master title style

XMLAGG The syntax

26

Click to edit Master title style

How to join XML together - XMLAGG

WITH PROJECT(EMPNO, PROJECTXML) as
(SELECT PROJ1.EMPNO,
XMLELEMENT(NAME ProjectDetails,
XMLFOREST(PROJNO as ProjectID,
EMSTDATE as StartDate,
EMENDATE as EndDate))
FROM DSN8910.EMPPROJACT PROJ1)
SELECT XMLELEMENT(NAME Name,
XMLFOREST(FIRSTNME as First,
LASTNAME as Last),
(SELECT XMLAGG(PROJ.PROJECTXML)
FROM PROJECT PROJ
WHERE PROJ.EMPNO = EMP.EMPNO))
FROM DSN8910.EMP EMP
;

27

Click to edit Master title style

How to join XML together XMLAGG

The Result
Row 1

Row 2

<NAME>
<FIRST>CHRISTINE</FIRST>
<LAST>HAAS</LAST>
<PROJECTDETAILS>
<PROJECTID>MA2100</PROJECTID>
<STARTDATE>1982-01-01</STARTDATE>
<ENDDATE>1982-11-01</ENDDATE>
</PROJECTDETAILS>
<PROJECTDETAILS>
<PROJECTID>MA2110</PROJECTID>
<STARTDATE>1982-01-01</STARTDATE>
<ENDDATE>1983-02-01</ENDDATE>
</PROJECTDETAILS>
<PROJECTDETAILS>
<PROJECTID>AD3100</PROJECTID>
<STARTDATE>1982-01-01</STARTDATE>
<ENDDATE>1982-07-01</ENDDATE>
</PROJECTDETAILS>
</NAME>

<NAME>
<FIRST>MICHAEL</FIRST>
<LAST>THOMPSON</LAST>
<PROJECTDETAILS>
<PROJECTID>PL2100</PROJECTID>
<STARTDATE>1982-01-01</STARTDATE>
<ENDDATE>1982-09-05</ENDDATE>
</PROJECTDETAILS>
</NAME>

28

Click to edit Master title style

The power of XMLAGG

It was discovered by using an additional construct in the SQL
statement, used to generate the XML, that is was possible to
generate all the data in a single select statement.
This was achieved using Common Table Expressions and a final
outer query which utilised the XMLAGG command to string all
the previously created result set rows into one row.

29

Click to edit Master title style

The power of XMLAGG - Example

WITH PROJECT(EMPNO, PROJECTXML) AS
(SELECT PROJ1.EMPNO, XMLELEMENT(NAME ProjectDetails,
XMLFOREST(PROJNO AS ProjectID, EMSTDATE AS StartDate,
EMENDATE AS EndDate))
FROM DSN8910.EMPPROJACT PROJ1),
EMPLOYEEDETAIL(EMPLOYEEXML) AS
(SELECT XMLELEMENT(NAME Name,
XMLFOREST(FIRSTNME AS First, LASTNAME AS Last),
(SELECT XMLAGG(PROJ.PROJECTXML)
FROM PROJECT PROJ WHERE PROJ.EMPNO = EMP.EMPNO))
FROM DSN8910.EMP EMP)
SELECT XMLELEMENT(NAME EmployeeDetails,
(SELECT XMLAGG(EMPLOYEEXML) FROM EMPLOYEEDETAIL))
FROM SYSIBM.SYSDUMMY1
WITH UR
;
30

Click to edit Master title style

The power of XMLAGG - The Result

<EMPLOYEEDETAILS>
<NAME>
<FIRST>CHRISTINE</FIRST>
<LAST>HAAS</LAST>
<PROJECTDETAILS>
<PROJECTID>MA2100</PROJECTID>
<STARTDATE>1982-01-01</STARTDATE>
<ENDDATE>1982-11-01</ENDDATE>
</PROJECTDETAILS>
<PROJECTDETAILS>
<PROJECTID>MA2110</PROJECTID>
<STARTDATE>1982-01-01</STARTDATE>
<ENDDATE>1983-02-01</ENDDATE>
</PROJECTDETAILS>
<PROJECTDETAILS>
<PROJECTID>AD3100</PROJECTID>
<STARTDATE>1982-01-01</STARTDATE>
<ENDDATE>1982-07-01</ENDDATE>
</PROJECTDETAILS>
</NAME>
<NAME>
<FIRST>MICHAEL</FIRST>
<LAST>THOMPSON</LAST>
<PROJECTDETAILS>
<PROJECTID>PL2100</PROJECTID>
<STARTDATE>1982-01-01</STARTDATE>
<ENDDATE>1982-09-05</ENDDATE>
</PROJECTDETAILS>
</NAME>
</EMPLOYEEDETAILS>

31

Click to edit Master title style

DEALING WITH NULLS

32

Click to edit Master title style

Options for Dealing with NULLS

1.
2.
3.

Three Choices
Do nothing specific and just create XML as you would.
Specifically ask to create an empty element.
Specifically dont create an element.

33

Click to edit Master title style

Create an Element (Treat as part of the data)

One option is to just create the XML as you would normally.
Therefore if we were to assume the EMP table had another
column called LEAVEDATE which allowed nulls and for Christine
the column contained a null, using the following SQL
SELECT XMLELEMENT(NAME Name,
XMLFOREST(FIRSTNME as First,
LASTNAME as Last
LEAVEDATE as LeaveDate))
FROM DSN8910.EMP
WHERE EMPNO = 000010
;

Would result in
<NAME><FIRST>CHRISTINE</FIRST><LAST>HAAS</LAST><LEAVEDATE/></NAME>

34

Click to edit Master title style

Specifically Create an Empty Element

Another option is to specify that an EMPTY element is created
as part of the XML.
SELECT XMLELEMENT(NAME Name,
XMLFOREST(FIRSTNME as First,
LASTNAME as Last
LEAVEDATE as LeaveDate
OPTION EMPTY ON NULL))
FROM DSN8910.EMP
WHERE EMPNO = 000010
;
<NAME><FIRST>CHRISTINE</FIRST><LAST>HAAS</LAST><LEAVEDATE/></NAME>

35

Click to edit Master title style

Specifically do not Create an Element

Another option is to specify that an element is not created as
part of the XML if the data in the element does not exist
SELECT XMLELEMENT(NAME Name,
XMLFOREST(FIRSTNME as First,
LASTNAME as Last
LEAVEDATE as LeaveDate
OPTION NULL ON NULL))
FROM DSN8910.EMP
WHERE EMPNO = 000010
;
<NAME><FIRST>CHRISTINE</FIRST><LAST>HAAS</LAST></NAME>

36

Click to edit Master title style

Dealing with NULLS - Recap

1.
2.
3.

3 Options
Create XML as is
Specifically create an EMPTY tag
Specifically do not create a tag.
Default behaviour if nothing specifically specified i.e. option 1
is followed, is to create an EMPTY tag.
Specification of an option is allowed in both the XMLELEMENT
and XMLFOREST syntax.

37

Click to edit Master title style

WHAT CAN BE ACHIEVED

38

Click to edit Master title style

What was achieved in Westpac

In a particular situation
Significant reduction in SQL calls to DB2
Significant reduction in average elapsed times to extract data
Significant reduction in CPU consumption to extract data.

39

40

Click to edit Master title style

What was achieved in Westpac - Context

Situation found was a COBOL program executing in the following
manner
Driver
Simpleton
Select

Cursor

Child
Cursor 10

Child
Cursor 9

Child
Cursor 8

Child
Cursor 1

Result set for

Driver Cursor
contained 64 rows.
Child
Cursor 7

Child
Cursor 4

Child
Cursor 6

Child
Cursor 5

Child
Cursor 2

Child
Cursor 3

Click to edit Master title style

Initial Redesign Option

Following suggested best practice it was decided to attempt to
rewrite all the cursors and the select statement into 1 cursor
using joins.
Result found that
Average CPU consumed increased 2000%
Average Elapsed Time increased 2000%
Calls to DB2 increased over 1000 %

41

Click to edit Master title style

What was achieved in Westpac

Initial Results.
Within a Test Environment Current State Cobol Program
Consumed on average 0.1630 seconds of CPU in DB2 and 0.1693
overall.
Executed 4588 Calls to DB2.
This includes 1 commit statement.

Took on average 0.1808 seconds elapsed time in DB2.

After using XML Transformation

Consumed on average 0.1510 seconds of CPU in DB2 and 0.1516
overall.
Executed 2 Calls to DB2.
This includes 1 commit statement.

Took on average 0.1660 seconds elapsed time in DB2.

42

Click to edit Master title style

What was achieved in Westpac

Final Results.
After using an addition to the XML Transformation Technique it
was managed to achieve the following results.
Consumed on average 0.0028 seconds of CPU in DB2 and 0.0033
overall.
Again executed 2 Calls to DB2.
This still included 1 commit statement.

Took 0.0034 seconds elapsed time on average in DB2.

43

Click to edit Master title style

What was achieved in Westpac

Final Results - Explained.
The final solution was able to select the converted relational
data from a table as an already generated XML document.
This approach was taken due to the fact that the data being
read was static in its nature so therefore did not need to be
generated every time it was requested.
Therefore the initial XML transformation was inserted into
another table as XML in an XML Column with a relational data
identifier and the process was modified to extract this instead
of regenerating the data every time it was read.

44

Click to edit Master title style

Other Suggestions /
Observations /
XML Syntax

45

Click to edit Master title style

Other useful XML Syntax - XMLSERIALIZE

Allows the serialisation of XML into a string type.
SELECT XMLSERIALIZE(XMLELEMENT(NAME First, FIRSTNME) AS
CLOB(100))
FROM DSN8910.EMP
WHERE EMPNO = 000010
;

Results
<FIRST>CHRISTINE</FIRST>
Note be careful in z/OS as you cannot serialise to a VARCHAR, in version 9 at least.

46

Click to edit Master title style

Other useful XML Syntax - XMLATTRIBUTE

SELECT XMLELEMENT(NAME EmployeeDetails,
XMLATTRIBUTE(SELECT CASE WHEN SEX = F THEN FEMALE
WHEN SEX = M THEN MALE
ELSE OTHER
END AS SEX
FROM DSN8910.EMP
WHERE EMPNO = 000010),
XMLFOREST(FIRSTNME as FirstName,
LASTNAME as LastName))
FROM DSN8910.EMP
WHERE EMPNO = 000010
;

47

Click to edit Master title style

XMLATTRIBUTE The Result

<EMPLOYEEDETAILS SEX="FEMALE">
<FirstName>CHRISTINE</FirstName>
<LastName>HAAS</LastName>
</EMPLOYEEDETAILS>

48

Click to edit Master title style

How to convert to XML - XMLCONCAT

SELECT XMLCONCAT(XMLELEMENT(NAME First, FirstName),
XMLELEMENT(NAME Last, LastName))
FROM DSN8910.EMP
WHERE EMPNO = 000010
;

Result

NOT XML

<FIRST>CHRISTINE</FIRST><LAST>HAAS</LAST>

49

Click to edit Master title style

How to convert to XML XMLCONCAT (Proper usage)

SELECT XMLELEMENT(NAME Name,
XMLCONCAT(XMLELEMENT(NAME First, FirstName),
XMLELEMENT(NAME Last, LastName)))
FROM DSN8910.EMP
WHERE EMPNO = 000010
;

Result
<NAME><FIRST>CHRISTINE</FIRST><LAST>HAAS</LAST></NAME>

50

Click to edit Master title style

Other Observations
Be careful of open Pandoras box.
People treating this a one size fits all solution or
A magic silver bullet.

System Configuration especially memory settings

In z/OS specifically zPARMs
XMLVALA The amount of memory a single agent is allowed to use while
processing XML
XMLVALS The amount of memory the sub system has allocated in total to
handle all XML processing at any point in time.

51

Click to edit Master title style

Conclusion What you colleagues might initially think

(Especially those of you who work with DB2 for z/OS)

52

Click to edit Master title style

Conclusion
Conversion from relational data to XML can provide benefits.
In certain circumstances these could be significant.
The benefits are not derived in all circumstances. Treat each
situation as its own and develop a solution accordingly. XML
conversion is not of a one size fits all approach so please be
sensible when applying.
The conversion process is not overly onerous and requires little
specialist knowledge of DB2 XML functionality. Just a good
understand of really only three SQL XML functions and
Common Table Expressions.

53

Click to edit Master title style

QUESTIONS
54

Click to edit Master title style

THANK YOU
55

Paul Murray
Westpac New Zealand Ltd
[email protected]
Session D02
From Relational Data to XML :
A Users unexpected journey.