Introduction to Database Systems
CSE 414
Lecture 16: XPath, XQuery, JSON
CSE 414 - Spring 2013 1
Announcements
• Next webquiz out now, due Friday night
• Homework 5 (XML/XQuery) out now, due
Wednesday night
• Midterm
– Returned today. Please hold off questions until
tomorrow after you’ve had a chance to review your
work, compare to sample solution, etc.
• (Although we can correct arithmetic bugs right away)
– If we goofed, we’ll fix it!! (But let’s be sure it’s a
goof first)
CSE 414 - Spring 2013 2
Querying XML Data
• XPath = simple navigation
• XQuery = the SQL of XML
• XSLT = recursive traversal
– will not discuss in class
CSE 414 - Spring 2013 3
Sample Data for Queries
<bib>
<book> <publisher> Addison-Wesley </publisher>
<author> Serge Abiteboul </author>
<author> <first-name> Rick </first-name>
<last-name> Hull </last-name>
</author>
<author> Victor Vianu </author>
<title> Foundations of Databases </title>
<year> 1995 </year>
</book>
<book price=“55”>
<publisher> Freeman </publisher>
<author> Jeffrey D. Ullman </author>
<title> Principles of Database and Knowledge Base Systems </title>
<year> 1998 </year>
</book>
</bib>
4
Data Model for XPath
XPath returns a sequence of items. An item is either:
• A value of primitive type, or
• A node (doc, element, or attribute)
The root
bib
The root element
book book
publisher author . . . .
Addison-Wesley Serge Abiteboul
5
XPath: Simple Expressions
/bib/book/year
Result: <year> 1995 </year>
<year> 1998 </year>
/bib/paper/year
Result: empty (there were no papers)
/bib What’s the difference ? /
CSE 414 - Spring 2013 6
XPath: Restricted Kleene Closure
//author
Result:<author> Serge Abiteboul </author>
<author> <first-name> Rick </first-name>
<last-name> Hull </last-name>
</author>
<author> Victor Vianu </author>
<author> Jeffrey D. Ullman </author>
/bib//first-name
Result: <first-name> Rick </first-name>
CSE 414 - Spring 2013 7
XPath: Attribute Nodes
/bib/book/@price
Result: “55”
@price means that price has to be an attribute
CSE 414 - Spring 2013 8
XPath: Wildcard
//author/*
Result: <first-name> Rick </first-name>
<last-name> Hull </last-name>
* Matches any element
@* Matches any attribute
CSE 414 - Spring 2013 9
XPath: Text Nodes
/bib/book/author/text()
Result: Serge Abiteboul
Victor Vianu
Jeffrey D. Ullman
Rick Hull doesn’t appear because he has first-name, last-name
Functions in XPath:
– text() = matches the text value
– node() = matches any node (= * or @* or text())
– name() = returns the name of the current tag
CSE 414 - Spring 2013 10
XPath: Predicates
/bib/book/author[first-name]
Result: <author> <first-name> Rick </first-name>
<last-name> Hull </last-name>
</author>
CSE 414 - Spring 2013 11
XPath: More Predicates
/bib/book/author[first-name][address[.//zip][city]]/last-name
Result: <last-name> … </last-name>
<last-name> … </last-name>
How do we read this ?
First remove all qualifiers (predicates):
/bib/book/author/last-name
Then add them one by one:
/bib/book/author[first-name][address]/last-name
CSE 414 - Spring 2013 12
XPath: More Predicates
/bib/book[@price < 60]
/bib/book[author/@age < 25]
/bib/book[author/text()]
CSE 414 - Spring 2013 13
XPath: Position Predicates
/bib/book[2] The 2nd book
/bib/book[last()] The last book
/bib/book[@year = 1998] [2] The 2nd of all
books in 1998
/bib/book[2][@year = 1998] 2nd book IF it
is in 1998
CSE 414 - Spring 2013 14
XPath: More Axes
. means current node /bib/book[.//review]
/bib/book[./review] Same as /bib/book[review]
/bib/author/. /first-name Same as /bib/author/first-name
CSE 414 - Spring 2013 15
XPath: More Axes
.. means parent node
/bib/author/.. /author/zip Same as /bib/author/zip
/bib/book[.//review/../comments]
Same as
/bib/book[.//*[comments][review]] Hint: don’t use ..
CSE 414 - Spring 2013 16
A Few Extra Examples
Run these examples on the sample xml posted on course website
Follow hw5 instructions
Each line is a separate example:
doc("sample-xml.xml")//book/price
doc("sample-xml.xml")//book[editor]/price
doc("sample-xml.xml")//book[price/text() > 100]/title
CSE 414 - Spring 2013 17
XPath: Summary
bib matches a bib element
* matches any element
/ matches the root element
/bib matches a bib element under root
bib/paper matches a paper in bib
bib//paper matches a paper in bib, at any depth
//paper matches a paper at any depth
paper|book matches a paper or a book
@price matches a price attribute
bib/book/@price matches price attribute in book, in bib
bib/book[@price<“55”]/author/last-name matches…
bib/book[@price<“55” or @price>”99”]/author/last-name matches…
CSE 414 - Spring 2013 18
XQuery
• Standard for high-level querying of databases
containing data in XML form
• Based on Quilt, which is based on XML-QL
• Uses XPath to express more complex queries
CSE 414 - Spring 2013 19
FLWR (“Flower”) Expressions
Zero or more
FOR ...
LET... Zero or more
WHERE...
RETURN... Zero or one
Exactly one
CSE 414 - Spring 2013 20
FOR-WHERE-RETURN
Find all book titles published after 1995:
FOR $x IN doc("bib.xml")/bib/book
WHERE $x/year/text() > 1995
RETURN $x/title
Result:
<title> abc </title>
<title> def </title>
<title> ghi </title>
CSE 414 - Spring 2013 21
FOR-WHERE-RETURN
Equivalently (perhaps more geekish)
FOR $x IN doc("bib.xml")/bib/book[year/text() > 1995] /title
RETURN $x
And even shorter:
doc("bib.xml")/bib/book[year/text() > 1995] /title
CSE 414 - Spring 2013 22
COERCION
The query:
FOR $x IN doc("bib.xml")/bib/book[year > 1995] /title
RETURN $x
Is rewritten by the system into:
FOR $x IN doc("bib.xml")/bib/book[year/text() > 1995] /title
RETURN $x
CSE 414 - Spring 2013 23
FOR-WHERE-RETURN
• Find all book titles and the year when they
were published:
FOR $x IN doc("bib.xml")/ bib/book
RETURN <answer>
<title>{ $x/title/text() } </title>
<year>{ $x/year/text() } </year>
</answer>
Result:
<answer> <title> abc </title> <year> 1995 </ year > </answer>
<answer> <title> def </title> < year > 2002 </ year > </answer>
<answer> <title> ghk </title> < year > 1980 </ year > </answer>
24
FOR-WHERE-RETURN
• Notice the use of “{“ and “}”
• What is the result without them ?
FOR $x IN doc("bib.xml")/ bib/book
RETURN <answer>
<title> $x/title/text() </title>
<year> $x/year/text() </year>
</answer>
<answer> <title> $x/title/text() </title> <year> $x/year/text() </year> </answer>
<answer> <title> $x/title/text() </title> <year> $x/year/text() </year> </answer>
<answer> <title> $x/title/text() </title> <year> $x/year/text() </year> </answer>
CSE 414 - Spring 2013 25
Nesting
• For each author of a book by Morgan
Kaufmann, list all books he/she published:
FOR $b IN doc(“bib.xml”)/bib,
$a IN $b/book[publisher /text()=“Morgan Kaufmann”]/author
RETURN <result>
{ $a,
FOR $t IN $b/book[author/text()=$a/text()]/title
RETURN $t
}
</result>
In the RETURN clause comma concatenates XML fragments
26
Result
<result>
<author>Jones</author>
<title> abc </title>
<title> def </title>
</result>
<result>
<author> Smith </author>
<title> ghi </title>
</result>
CSE 414 - Spring 2013 27
Aggregates
Find all books with more than 3 authors:
FOR $x IN doc("bib.xml")/bib/book
WHERE count($x/author)>3
RETURN $x
count = a function that counts
avg = computes the average
sum = computes the sum
distinct-values = eliminates duplicates
CSE 414 - Spring 2013 28
Aggregates
Same thing:
FOR $x IN doc("bib.xml")/bib/book[count(author)>3]
RETURN $x
CSE 414 - Spring 2013 29
Eliminating Duplicates
Print all authors:
FOR $a IN distinct-values($b/book/author/text())
RETURN <author> { $a } </author>
Note: distinct-values applies ONLY to values, NOT elements
CSE 414 - Spring 2013 30
The LET Clause
Find books whose price is larger than average:
FOR $b in doc(“bib.xml”)/bib
LET $a:=avg($b/book/price/text())
FOR $x in $b/book
WHERE $x/price/text() > $a
RETURN $x
LET enables us to declare variables
CSE 414 - Spring 2013 31
Flattening
Compute a list of (author, title) pairs
Input:
<book>
<title> Databases </title>
<author> Widom </author>
<author> Ullman </author> FOR $b IN doc("bib.xml")/bib/book,
</book> $x IN $b/title/text(),
Output: $y IN $b/author/text()
<answer> RETURN <answer>
<title> Databases </title> <title> { $x } </title>
<author> Widom </author> <author> { $y } </author>
</answer>
</answer>
<answer>
<title> Databases </title>
<author> Ullman </author>
</answer> CSE 414 - Spring 2013 32
Re-grouping
For each author, return all titles of her/his books
Result:
FOR $b IN doc("bib.xml")/bib, <answer>
$x IN $b/book/author/text() <author> efg </author>
RETURN <title> abc </title>
<answer> <title> klm </title>
<author> { $x } </author> ....
{ FOR $y IN $b/book[author/text()=$x]/title </answer>
RETURN $y } What about
</answer> duplicate
authors ?
CSE 414 - Spring 2013 33
Re-grouping
Same, but eliminate duplicate authors:
FOR $b IN doc("bib.xml")/bib
LET $a := distinct-values($b/book/author/text())
FOR $x IN $a
RETURN
<answer>
<author> $x </author>
{ FOR $y IN $b/book[author/text()=$x]/title
RETURN $y }
</answer>
CSE 414 - Spring 2013 34
Re-grouping
Same thing:
FOR $b IN doc("bib.xml")/bib,
$x IN distinct-values($b/book/author/text())
RETURN
<answer>
<author> $x </author>
{ FOR $y IN $b/book[author/text()=$x]/title
RETURN $y }
</answer>
CSE 414 - Spring 2013 35
SQL and XQuery Side-by-side
Product(pid, name, maker, price) Find all product names, prices,
sort by price
SELECT x.name, FOR $x in doc(“db.xml”)/db/Product/row
x.price ORDER BY $x/price/text()
FROM Product x RETURN <answer>
ORDER BY x.price { $x/name, $x/price }
</answer>
SQL
XQuery
CSE 414 - Spring 2013 36
XQuery’s Answer
<answer>
<name> abc </name>
<price> 7 </price>
</answer>
<answer>
<name> def </name>
<price> 23 </price> Notice: this is NOT a
</answer> well-formed document !
.... (WHY ???)
CSE 414 - Spring 2013 37
Producing a Well-Formed Answer
<aQuery>
{ FOR $x in doc(“db.xml”)/db/Product/row
ORDER BY $x/price/text()
RETURN <answer>
{ $x/name, $x/price }
</answer>
}
</aQuery>
CSE 414 - Spring 2013 38
XQuery’s Answer
<aQuery>
<answer>
<name> abc </name>
Now it is well-formed !
<price> 7 </price>
</answer>
<answer>
<name> def </name>
<price> 23 </price>
</answer>
....
</aQuery>
CSE 414 - Spring 2013 39
SQL and XQuery Side-by-side
Product(pid, name, maker, price)
Company(cid, name, city, revenues) Find all products made in Seattle
FOR $r in doc(“db.xml”)/db,
$x in $r/Product/row,
SELECT x.name $y in $r/Company/row
FROM Product x, Company y WHERE
WHERE x.maker=y.cid $x/maker/text()=$y/cid/text()
and y.city=“Seattle” and $y/city/text() = “Seattle”
RETURN { $x/name }
SQL XQuery
FOR $y in /db/Company/row[city/text()=“Seattle”],
Cool $x in /db/Product/row[maker/text()=$y/cid/text()]
XQuery RETURN { $x/name } 40
<product>
<row> <pid> 123 </pid>
<name> abc </name>
<maker> efg </maker>
</row>
<row> …. </row>
…
</product>
<product>
...
</product>
....
CSE 414 - Spring 2013 41
SQL and XQuery Side-by-side
For each company with revenues < 1M count the products over $100
SELECT y.name, count(*)
FROM Product x, Company y
WHERE x.price > 100 and x.maker=y.cid and y.revenue < 1000000
GROUP BY y.cid, y.name
FOR $r in doc(“db.xml”)/db,
$y in $r/Company/row[revenue/text()<1000000]
RETURN
<proudCompany>
<companyName> { $y/name/text() } </companyName>
<numberOfExpensiveProducts>
{ count($r/Product/row[maker/text()=$y/cid/text()][price/text()>100])}
</numberOfExpensiveProducts>
</proudCompany> 42
SQL and XQuery Side-by-side
Find companies with at least 30 products, and their average price
SELECT y.name, avg(x.price)
FROM Product x, Company y
WHERE x.maker=y.cid
GROUP BY y.cid, y.name An element
HAVING count(*) > 30
FOR $r in doc(“db.xml”)/db,
$y in $r/Company/row
LET $p := $r/Product/row[maker/text()=$y/cid/text()]
WHERE count($p) > 30
A collection RETURN
<theCompany>
<companyName> { $y/name/text() }
</companyName>
<avgPrice> avg($p/price/text()) </avgPrice>
</theCompany> 43
XML Summary
• Stands for eXtensible Markup Language
1. Advanced, self-describing file format
2. Based on a flexible, semi-structured data model
• Query languages for XML
– XPath
– XQuery
CSE 414 - Spring 2013 44
Beyond XML: JSON
• JSON stands for “JavaScript Object Notation”
– Lightweight text-data interchange format
– Language independent
– “Self-describing" and easy to understand
• JSON is quickly replacing XML for
– Data interchange
– Representing and storing semi-structure data
CSE 414 - Spring 2013 45
JSON
Example from: http://www.jsonexample.com/
myObject = {
"first": "John",
"last": "Doe",
"salary": 70000,
"registered": true,
"interests": [ "Reading", “Biking”, "Hacking" ]
}
CSE 414 - Spring 2013 46
