0% found this document useful (0 votes)

2 views

Module 9 Lecture

The document outlines advanced concepts in business intelligence and big data, focusing on reporting systems, RFM analysis, data mining techniques, and decision trees. It details the importance of timely report delivery, various report types, and the methodologies for unsupervised and supervised data mining. Additionally, it introduces the concept of Big Data and its significance in modern data analysis.

Uploaded by

samerelking3

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

2 views

Module 9 Lecture

Uploaded by

samerelking3

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 137

Database Concepts

Ninth Edition

Extension C
Advanced Business Intelligence
and Big Data

• Learn the basic concepts of reporting systems

• Learn the basic concepts of RFM reporting
• Learn the basic concepts of data mining
• Learn the basic concepts of market basket analysis
• Learn the basic concepts of decision trees
• Learn the basic concepts of Big Data
• Learn the basic concepts of nonrelational database management
systems
• Learn about replicated, partitioned data stores for supporting clusters
• Understand the importance of XML
• Understand the role of XSLT in materializing XML documents

• Learn the elements of XML documents

• Understand how to describe and validate XML document structure
using XML Schema
• Understand how XML can be supported in a DBMS
• See examples of XPath and XQuery for queries of XML documents
• Learn the basic concepts of key-value NoSQL databases
• Learn the basic concepts of column family NoSQL databases
• Learn the basic concepts of graph NoSQL databases
• Learn some advanced concepts of document NoSQL databases
• Learn the basic concepts of the Microsoft Azure cloud environment for
creating and using relational databases

Learn the basic concepts of reporting systems

• The purpose of a reporting system is to create
meaningful information from disparate data sources and to
deliver that information to the proper users on a timely
basis.
• Reporting systems summarize the current status of
business activities and compare that status with past or
predicted future activities.
• Report delivery is crucial.
– Reports must be delivered to the proper users on a
timely basis in the appropriate format.

Learn the basic concepts of RFM reporting

• RFM analysis analyzes and ranks customers according to their
purchasing patterns.
• RFM analysis is a simple customer classification technique that
considers how recently (R) a customer has ordered, how
frequently (F) a customer orders, and how much money (M) the
customer spends per order.
• To produce an RFM score, we need only two things: customer
data and sales data for each purchase (including the date of
the sale and the total amount of the sale) made by each
customer.
• If you look at the SALES_FOR_RFM table and its associated
CUSTOMER and TIMELINE dimension tables in Figure C-1,
you see that we have exactly those data:
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Figure C.1 The HSD-DW Dimensional
6

Database with RFM Data

SQL Server 2017, SQL Server Management Studio, Windows 10, Microsoft Corporation
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Calculating the R Score in RFM
7

Learn the basic concepts of RFM reporting

• To calculate an R score:
1. Sort the customer purchase records by the date of the
most recent (R) purchase—note that only the most
recent purchase for each customer is used in this
calculation.
2. The customers are then divided into five groups, and a
score of 1 to 5 is given to customers in each group.
§ The 20 percent of the customers having the most
recent orders are given an R score of 1, the 20
percent of the customers having the next most recent
orders are given an R score of 2, and so forth, down
to the last 20 percent, who are given an R score of 5.

Learn the basic concepts of RFM reporting

• To calculate an F score:
1. Re-sort the customers on the basis of how
frequently they order.
2. As before, the customers are again divided into five
groups.
§ The 20 percent of the customers who order most
frequently are given an F score of 1, the next 20
percent most frequently ordering customers are
given a score of 2, and so forth, down to the least
frequently ordering customers, who are given an
F score of 5.

Learn the basic concepts of RFM reporting

• To calculate an M score:
1. Re-sort the customers according to the average
amounts spent on their orders.
2. The 20 percent who have placed the largest orders
on average are given an M score of 1, the next 20
percent are given an M score of 2, and so forth,
down to the 20 percent who spend the least, who
are given an M score of 5.

- Tables for the RFM Analysis

- RFM_Analysis Stored Procedure

- Calculate_R Stored Procedure

- Example RFM Results

Learn the basic concepts of reporting systems

• A reporting system maintains a database of reporting
metadata.
• The metadata describe reports, users, groups, roles,
events, and other entities involved in the reporting activity.
• The reporting system uses the metadata to prepare and
deliver appropriate reports to the proper users in the
correct format on a timely basis as shown in Figure C-3,

Reporting System

Learn the basic concepts of reporting systems

• Report types include:
– Static reports are prepared once from the underlying data,
and they do not change. A report of the past year’s sales,
for example, is a static report.
– Dynamic reports are created when the reporting system
reads the latest, most current data and generates the report
using those fresh data. Reports on today’s sales and on
current stock prices are dynamic reports.
– Query reports are prepared in response to information
entered by users. Within a specific organization, such as
Heather Sweeney Designs, a query report could be
generated to show current inventory levels. The user would
enter item numbers, and the reporting system would
respond with inventory levels of those items.
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Figure C.4 Report Characteristics
18

Learn the basic concepts of reporting systems

• Reports can be delivered by printed paper, sent
electronically, delivered via Web portals, and by other
means.
• A digital dashboard is an electronic display that is
customized for a particular user. Companies such as
Google, MSN, and Yahoo! offer digital dashboard services
that you might have seen or used.
– Users can define the content they want to see (a local
weather forecast, a list of stock prices) and the vendor
constructs a customized display for each user. (My
Yahoo, for example)
– Reports can also be delivered via alerts.

- RFM Report Dashboard

Learn the basic concepts of reporting systems

• Most reporting systems serve the following three functions:
– Report management consists of defining who receives what reports,
when, and by what means. Most report management systems enable the
report system administrator to define user accounts and user groups and
to assign users to one or more groups
– Report authoring involves connecting to the required data sources,
creating the report structure, and formatting the report. Reports created
using a report authoring system are then assigned to groups and users.
Assigning reports to groups saves the administrator work; when a report
is created, changed, or removed, the administrator need only change the
report assignments of the group, and all the users in the group will inherit
the changes.
– The report delivery function of a reporting system pushes reports or
allows them to be pulled based on the report management metadata.
Reports can be delivered by hand or via an email server, a Web portal,
XML Web Services, or other program-specific means.

Learn the basic concepts of data mining

• Data mining represents the convergence of several
methodologies with techniques emerging from the
statistical and mathematics disciplines and from the
artificial intelligence and machine-learning communities.
• Data mining techniques take advantage of developments
for processing enormous databases that have emerged in
the past two decades.
• Almost all data mining techniques require specialized
software such as Enterprise Miner from SAS Corporation,
SPSS Modeler from IBM, and HP Vertica Analytics
Platform from Hewlett-Packard.

Disciplines for Data Mining

Learn the basic concepts of data mining

• When using unsupervised data mining techniques,
analysts do not create a model or hypothesis prior to
beginning the analysis.
• Instead, the data mining technique is applied to the data
and results are observed. After the analysis, explanations
and hypotheses are created to explain the patterns found.
• Two commonly used unsupervised technique is:
1. Cluster analysis uses statistical techniques to
identify groups of entities that have similar
characteristics.
2. Market basket analysis is another form of
unsupervised data mining.

Learn the basic concepts of market basket analysis

• Market Basket Analysis is a data mining technique for
determining patterns and rules and can be readily
implemented with pure SQL.
– Uses association rules to upsell customers
– Based on conditional probabilities

Example

Analysis
Learn the basic concepts of market basket analysis
• Support as the probability that two items will be
purchased together.
• Confidence is defined as the probability of a customer
buying one product, given that he or she is buying another
product.
• Lift is defined as the ratio of confidence divided by the
base probability of an item purchase.

Learn the basic concepts of data mining

• When using supervised data mining techniques, data miners
develop a model prior to the analysis and then apply statistical
techniques to the data to estimate parameters of the model.
– For example, suppose that marketing experts at a
communications company believe that the use of cell phone
weekend minutes is determined by the age of the customer and
the number of months the customer has had the cell phone
account. A data mining analyst could then run a statistical
analysis technique known as regression analysis to determine the
coefficients of the equation of that model. A possible result is:

CellPhoneWeekendMinutes = 12 + (17.5 * CustomerAge) + (23.7 *

NumberMonthsOfAccount)

Learn the basic concepts of data mining

• Supervised data mining techniques include:
– Regression analysis uses statistical analysis techniques to
determine the coefficients of an equation model.
– Decision tree analysis classifies customers or other entities of
interest into two or more groups, according to history.
– Logistic regression produces equations that offer probabilities
that particular events will occur.
§ An example is using donor characteristics to predict the
likelihood of a donation in a given period and using customer
characteristics to predict the likelihood that customers will
switch to another vendor.
– Neural networks are complex statistical prediction techniques
using very complex mathematical functions for making
predictions.

Decision Trees
• A decision tree represents a set of rules that can
be easily expressed in English or in SQL.
• A decision tree is created in two main phases:
– 1. Training phase – the decision tree is
created based on data found (such as in a data
warehouse).
– 2. Testing phase – the tree is given some new
data points (for which we already know the
answer) and then the results are judged.

Root

Leaf

Example

Decision Tree

Oracle Database 12c, SQL Developer 18.4, Oracle Corporation

Decision Trees for the Ice Skating

Example

Oracle Database 12c, SQL Developer 18.4, Oracle Data Miner, Oracle Corporation
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Figure C.11 A Decision Tree for the
35

Skating Data Set

Figure C.12 Oracle Data Mining Partial Decision

Tree and Rule for Poisonous Mushroom Data

Oracle Database 12c, SQL Developer 18.4, Oracle Data Miner, Oracle Corporation
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
What is Big Data?
37

Learn the basic concepts of Big Data

• Big Data is defined as a cultural, technological, and
scholarly phenomenon that rests on the interplay of:
1. Technology: maximizing computation power and
algorithmic accuracy to gather, analyze, link, and compare
large data sets.
2. Analysis: drawing on large data sets to identify patterns
in order to make economic, social, technical, and legal
claims.
3. Mythology: the widespread belief that large data sets
offer a higher form of intelligence and knowledge that can
generate insights that were previously impossible, with
the aura of truth, objectivity, and accuracy.

Learn the basic concepts of Big Data

• The 3V framework discussed Big Data in terms of:
– Volume: the extremely large amount of data that
needs to be stored.
– Velocity: the speed and continuous nature of data
acquisition.
– Variety: the many different forms of data being
acquired and stored.

- Volume

• The growth of the volume of big data:

– Twitter needed to store more than 7 TB of data per
day(in 2012)
– Facebook needed to store more than 10 TB of data per
day(in 2012)
– Projected the need to store more than 35 ZB by 2020
– 2.3 trillion GB of data are currently created each day
– Hard drives currently handle 1-10 TBs (Jan, 2018)

- Velocity and Variety

• With the increasing volume of data there is an

increasing need of velocity (measure of speed).
• Consider just 7 terabytes per day and what that
equates to:
– 7 TB/day = 290 GB/hour = 4.8 GB/minute and that
amounts to 81 MB per second that must be stored and
processed!
• Data variety indicates that it is not just data but
also pictures, music streaming and all the other
types of content that need to be stored!
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
41

- The 3 Vs plus 1

• IBM added veracity, having to do with the quality

and accuracy of data. Data needs to be correct
and useful.

Learn the basic concepts of nonrelational database management

systems

• NoSQL systems (referred to as the Not only SQL

movement) are nonrelational DBMSs. A NoSQL DBMS is
often a distributed, replicated (i.e., cloud) database and is
used where this type of DBMS is needed to support large
data sets. Examples of NoSQL systems include:
– Key-Value—Examples are DynamoDB and MemcacheDB
– Document—Examples are ArangoDB, Couchbase Server,
MongoDB, and Microsoft Azure Cosmos DB
– Column Family—Examples are Apache Cassandra and
Hbase
– Graph—Examples are Neo4j and AllegroGraph

Learn the basic concepts of nonrelational database management

systems
• Sharding is a set of approaches for taking portions of a database and moving
them to a different server in a cluster.
• Replication is accomplished by making one or more copies of a set of data
and maintaining it on a different server.
– Each copy is called a replica set.
– Replication increases the processing capacity of our system by allowing
several servers to access the data simultaneously (helps continuity if a
server fails).

Learn the basic concepts of nonrelational database management

systems
• Aggregates refers to databases that can store complex data using the data
modeling languages XML and JSON.
– The first three types of NoSQL systems (Key-value, Document, and Column
Family), as well as some NoSQL Graph databases, can store aggregates
– For aggregate NoSQL databases, we have to think of “transactions” in a
different way. Although operations on a single aggregate are atomic,
there is usually not built-in support for operations across more than one
aggregate data object.

Understand the importance of XML

• SGML (Standard Generalized Markup Language) was clearly as
important to document processing as the relational model was to
database processing and was the predecessor to a series of standards
that describes a language called XML.
• XML (Extensible Markup Language) technology is a hybrid of document
processing and database processing. XML provides a standardized yet
customizable way to describe the content of documents.
– XML documents can automatically be generated from database data.
– As a markup language, XML is significantly better than HTML in several
ways.
§ XML provides a clean separation between document structure, content, and
Materialization—the way the document is displayed on a particular device.
§ XML has facilities for dealing with each, and they cannot be confounded
(mixed together), as they are with HTML.

Understand the importance of XML

• XML makes a clear separation between structure, content,
and materialization.
– Structure is defined by either a DTD or an XML Schema
document.
– Content is expressed in an XML document, and the
– Materializations of a document are expressed in an XSL
document.

Materializing XML Documents with XSLT

Understand the role of XSLT in materializing XML documents
• XSLT (Extensible Style Language: Transformations) is
a powerful and robust transformation language.
– It can be used to materialize XML documents into
HTML as well as for other purposes.
– One common application of XSLT is to transform an
XML document in one format into a second XML
document in another format.
§ For example, a company can use XSLT to
transform an XML order document in its own format
into an equivalent XML order document in its
customer’s format.

Reference to an XSLT Stylesheet

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="DBC-e09-
SeminarListStyleSheet.xsl"?>
<SeminarList>
<Seminar>
<SeminarID>1</SeminarID>
<SeminarDate>10/10/2018</SeminarDate>
<SeminarTime>11:00</SeminarTime>
<Location>San Antonio Convention Center</Location>
<SeminarTitle>Kitchen on a Budget</SeminarTitle>
</Seminar>
<Seminar>
<SeminarID>2</SeminarID>
<SeminarDate>10/26/2018</SeminarDate>
<SeminarTime>16:00</SeminarTime>
<Location>Dallas Convention Center</Location>
<SeminarTitle>Kitchen on a Big D Budget</SeminarTitle>
</Seminar>
</SeminarList>
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Figure C.14 The Heather Sweeney
49

Designs Seminar Stylesheet (1 of 2)

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet
version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns="http://www.w3.org/1999/xhtml">
<xsl:output method="xml" encoding="UTF-8"/>
<xsl:template match="/">
<html>
<head> <title>Seminar Data</title> </head>
<style type="text/css">
h1 {text-align:center; color:blue}
body {font-family:ariel,sans-serif;font-size:12pt;
background-color:#FFFFFF}
div.seminardata {font-weight:bold;background-color:#3399
FF;color:#FFFFFF;padding:4px}
</style>
<body>

Designs Seminar Stylesheet (2 of 2)

<p>
<h1>HSD Seminar Data</h1>
</p><hr />
<div class="seminardata">
<xsl:for-each select="SeminarList/Seminar">
<br/>
<xsl:value-of select="Location"/> <br/>
<xsl:value-of select="SeminarDate"/> <br/>
<xsl:value-of select="SeminarTime"/>
<br/><br/>
</xsl:for-each>
</div>
</body>
</html>
</xsl:template>
</xsl:stylesheet>

Application of Seminar Stylesheet

Windows 10, Microsoft Corporation

XML Document Type Declarations and XML Schema

(1/2)
Understand how to describe and validate XML document structure using XML
Schema

• DTDs (Document Type Declarations) were the XML community’s first attempt at
developing a document structure specification language recognized by the keyword
DOCTYPE in the document. DTD documents are not XML documents.
• XML Schema is the preferred method for defining and validating document structure.
XML Schemas are XML documents meaning that you use the same language to define
an XML Schema as you would use to define any other XML document.
• XML Schemas consist of elements. Elements are either:
– A simple element has a single data item. In Figure C-16(a), the elements
SeminarID, SeminarDate, SeminarTime, Location, and SeminarTitle are all simple
elements.
– A complex element can contain other elements that can be either simple or
complex. In Figure C-16(a), Seminar is a complex element and is followed by a
section with five simple elements.

XML Document Type Declarations and XML Schema

(2/2)
Understand how to describe and validate XML document structure using XML
Schema

• XML Schema is the preferred method for defining and validating document structure.
XML Schemas are XML documents meaning that you use the same language to define
an XML Schema as you would use to define any other XML document.
– Schemas also contain Attributes, which are usually used to provide additional
information about elements. For example, we could choose to store the
SeminarID as an attribute of Seminar by coding: <Seminar SeminarID=”1”>
– A good rule of thumb is that elements represent data and attributes represent
metadata, though this is not part of any XML standard.

Document (1 of 3)
(a) XML Schema

<?xml version="1.0" encoding="UTF-8"?>

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:element name="SeminarList">
<xsd:annotation>
<xsd:documentation>
This is an XML Schema describing a Seminar
List at Heather Sweeney Designs.
</xsd:documentation>
</xsd:annotation>
<xsd:complexType>
<xsd:sequence>
<xsd:element name="Seminar"
maxOccurs="unbounded">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="SeminarID"
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Figure C.16 Example XML Schema
55

Document (2 of 3)
(a) XML Schema

type="xsd:integer"/>
<xsd:element name="SeminarDate"
type="xsd:string"/>
<xsd:element name="SeminarTime"
type="xsd:string"/>
<xsd:element name="Location"
type="xsd:string"/>
<xsd:element name="SeminarTitle"
type="xsd:string"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Figure C.16 Example XML Schema
56

Document (3 of 3)

(b) Graphical Representation of XML Schema

Database Data
Several relational database systems, such as SQL Server, Oracle Database,
and MySQL have facilities for generating XML documents from database
data using the FOR XML clause in an SQL query. Consider the following
SQL statement,

/* * SQL-Query-ExtC-01 * */

SELECT *
FROM SEMINAR
FOR XML RAW;

The expression FOR XML RAW places the values of the columns as attributes in
the attribute of element named row.

XML Document
<row SeminarID="1" SeminarDate="2018-10-12" SeminarTime="11:00:00"
Location="San Antonio Convention Center" SeminarTitle="Kitchen on a
Budget" />
<row SeminarID="2" SeminarDate="2018-10-26" SeminarTime="16:00:00"
Location="Dallas Convention Center" SeminarTitle="Kitchen on a Big D
Budget" />
<row SeminarID="3" SeminarDate="2018-11-02" SeminarTime="08:30:00"
Location="Austin Convention Center" SeminarTitle="Kitchen on a Budget" />
<row SeminarID="4" SeminarDate="2019-03-22" SeminarTime="11:00:00"
Location="Dallas Convention Center" SeminarTitle="Kitchen on a Big D
Budget" />
<row SeminarID="5" SeminarDate="2019-03-23" SeminarTime="11:00:00"
Location="Dallas Convention Center" SeminarTitle="Kitchen on a Big D
Budget" />
<row SeminarID="6" SeminarDate="2019-04-05" SeminarTime="08:30:00"
Location="Austin Convention Center" SeminarTitle="Kitchen on a Budget" />

Database Data
It is also possible to cause SQL Server to place the values of the columns
into elements rather than attributes by using FOR XML AUTO, ELEMENTS.
For example, we can display the data in the SEMINAR table using the SQL
query:

/* * SQL-Query-ExtC-02 * */

SELECT *
FROM SEMINAR
FOR XML AUTO, ELEMENTS;

Figure C.18 FOR XML AUTO, ELEMENTS Results in XML

Document (1 of 2)
<SEMINAR>
<SeminarID>1</SeminarID>
<SeminarDate>2018-10-12</SeminarDate>
<SeminarTime>11:00:00</SeminarTime>
<Location>San Antonio Convention Center</Location>
<SeminarTitle>Kitchen on a Budget</SeminarTitle>
</SEMINAR>
<SEMINAR>
<SeminarID>2</SeminarID>
<SeminarDate>2018-10-26</SeminarDate>
<SeminarTime>16:00:00</SeminarTime>
<Location>Dallas Convention Center</Location>
<SeminarTitle>Kitchen on a Big D Budget</SeminarTitle>
</SEMINAR>
<SEMINAR>
<SeminarID>3</SeminarID>
<SeminarDate>2018-11-02</SeminarDate>
<SeminarTime>08:30:00</SeminarTime>
<Location>Austin Convention Center</Location>
<SeminarTitle>Kitchen on a Budget</SeminarTitle>
</SEMINAR>

Figure C.18 FOR XML AUTO, ELEMENTS Results in XML

Document (2 of 2)
<SEMINAR>
<SeminarID>4</SeminarID>
<SeminarDate>2019-03-22</SeminarDate>
<SeminarTime>11:00:00</SeminarTime>
<Location>Dallas Convention Center</Location>
<SeminarTitle>Kitchen on a Big D Budget</SeminarTitle>
</SEMINAR>
<SEMINAR>
<SeminarID>5</SeminarID>
<SeminarDate>2019-03-23</SeminarDate>
<SeminarTime>11:00:00</SeminarTime>
<Location>Dallas Convention Center</Location>
<SeminarTitle>Kitchen on a Big D Budget</SeminarTitle>
</SEMINAR>
<SEMINAR>
<SeminarID>6</SeminarID>
<SeminarDate>2019-04-05</SeminarDate>
<SeminarTime>08:30:00</SeminarTime>
<Location>Austin Convention Center</Location>
<SeminarTitle>Kitchen on a Budget</SeminarTitle>
</SEMINAR>

Using XPath and XQuery to Query XML

Documents (1/2)
See examples of XPath and XQuery for queries of XML documents
• There are two basic approaches to performing queries on XML documents.
1. The XPath standard renders the document as a hierarchical structure, much like
files are stored in a directory structure, to navigate and select elements in an
XML file. Here is an example XPath query:
/* *** XML-Query-ExtC-01 *** */
.//SeminarList/Seminar/SeminarTitle[ends-with(., 'Budget’)]
The result of using this query is:
<SeminarTitle>Kitchen on a Budget</SeminarTitle>
<SeminarTitle>Kitchen on a Big D Budget</SeminarTitle>

Using XPath and XQuery to Query XML

Documents (2/2)
See examples of XPath and XQuery for queries of XML documents
• There are two basic approaches to performing queries on XML documents.
2. The XQuery standard, which uses a programming-language style FOR loop to
iterate through the document as follows:
/* *** XML-Query-ExtC-02 *** */
for $seminar in SeminarList/Seminar
let $location := $seminar/Location
let $date := $seminar/SeminarDate
where matches($location, "Dallas*")
order by ($date)[1]
return $seminar

This query structure is called FLWOR (For/Let/Where/Order by/Return).

NoSQL Database Management Systems

Learn the basic concepts of nonrelational database management
systems

• NoSQL database management systems can be divided

into four major categories as previously described (key-
value, column family, graph, and document).
• No matter which of the four categories they belong to, they
all have one thing in common: There is often no schema
for the database, as there is in a relational database.
– Results in fewer restrictions on the way data can be
structured.
– These systems are also often called semi-structured
databases or unstructured databases.

Key-Value Databases
Learn the basic concepts of key-value NoSQL databases
• Key-value databases use a simple key and value pairing.
– Each key (similar to a relational DBMS primary key) appears only
once in each database.
– A key-value database is extremely simple and allows for easy
distribution of the key value pairs in a networked cluster of
servers.
– Key-value databases are ideal for large amounts of data that
need fast storage and retrieval of simple objects but do not need
the full complexity of SQL queries.
– The basic operations available to query a key-value database are:
§ get (key): retrieves the value associated with a key
§ set (key, value): creates or updates a key-value pair
§ delete (key): removes a key-value pair
• Examples of Key-Value databases include DynamoDB and
MemcacheDB.
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Key-Value Database Examples
• Sample data from a key-value database:

("[email protected]", 172.13.233.1)
("[email protected]", 177.10.254.1)

Column Family Databases

Learn the basic concepts of column family NoSQL databases
• The basis for much of the development of column family
databases was a structured storage mechanism named
Bigtable.
– The smallest unit of storage is called a column.
– Columns can be grouped into sets referred to as super columns.
– Columns and super columns are grouped to create column
families.
§ Each row of a grouped column has a RowKey (similar to a primary
key)
§ All column families are contained within a keyspace.

• Examples of Column Family databases include Apache

Cassandra and Hbase.

Figure C.19 A Generalized Column Family

Database Storage System (1 of 4)

(a) A Column
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
69

Figure C.19 A Generalized Column Family

Database Storage System (2 of 4)

(b) A Super Column

Figure C.19 A Generalized Column Family

Database Storage System (3 of 4)

(c) A Column Family

Figure C.19 A Generalized Column Family

Database Storage System (4 of 4)

(d) A Super Column Family

Learn the basic concepts of graph NoSQL databases

• Based on mathematical graph theory, graph databases are composed
of three elements:
1. Nodes are equivalent to entities in E-R data modeling and
tables (or relations) in database design. They represent the
things that we want to keep track of or about which we want to
store data.
2. Properties are equivalent to attributes in E-R data modeling
and columns (or fields) in database design. They represent the
data items that we want to store for each node.
3. Edges are similar to, but not identical to, the relationships in E-
R data models and database designs. They are similar because
they connect nodes as relationships connect entities, but they
are different because they also store data. Edges can also have
a “direction.”
• Examples of Graph Databases include Neo4j and AllegroGraph
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Figure C.20 A Partial HSD Graph
73

Database

ArangoDB

Courtesy of ArangoDB GmbH

AQL Query Example

• AQL, the ArangoDB Query Language, will give us the emails of all people who are
known to someone who is known to Ralph Able or known to someone who is known to
him:

/* * AQL-Query-ExtC-01 * */

FOR Cust IN 1..2 OUTBOUND
"CUSTOMERS/RalphAble" GRAPH "KnowsGraph"
RETURN DISTINCT Cust.EmailAddress

IE: Starting with customer Ralph Able, and following the “Knows” edges either 1 or 2
steps, return the email addresses of every customer encountered.

[
"[email protected]",
"[email protected]",
"[email protected]"
]

Learn some advanced concepts of document NoSQL databases

• Document databases store data according to a
document-oriented format.
• Examples include ArangoDB, Couchbase Server,
MongoDB, and Microsoft Azure Cosmos DB.
• The CAP Theorem deals with network partitioning:
– Consistency
– Availability
– Partition tolerance
– For example: ArangoDB has chosen C and P over A in
the terminology of the CAP theorem.

- Document Databases
• Document databases store data according to a document-
oriented format (such as XML), and JSON (Java Script
Object Notation).
• Typical commands available in a document database:
– insert (doc, collection): put document "doc" into the collection
– update (collection, doc_specifier, update_action): within the
collection, update all documents matching the "doc_specifier" (e.g.
all students with last name 'Smith'). The "update_action" can alter
the values of multiple fields within each document.
– delete (collection, doc_specifier): removes all documents from
collection that match the "doc_specifier"
– find (collection, doc_specifier): retrieves all documents in a
collection matching the "doc_specifier"

- Document Databases
• JSON, has some similarities to XML, but there is no notion
of any schema external to the data.
• A document (“object” in JSON terminology) consists of a
set of (field, value) pairs. Values can be simple, other
objects, arrays of values or objects. The number, order,
types, and (in some systems) names of columns are less
restricted in a semistructured database; in addition, fields
can be multi-valued.
• Figure C-22 shows the entire Art Course database as an
array of documents in ArangoDB.

Database in JSON (1 of 6)
[
{
"_key": "558",
"_id": "Courses/558",
"_rev": "_XU2oaVC--F",
"CourseNumber": 4,
"Course": "Beg Oils",
"CourseDate": "10/15/2019",
"Fee": 350,
"Enrollments": [
{
"CustomerNumber": 7,
"CustomerLastName": "Myers",
"CustomerFirstName": "Lynda",
"Phone": "360-789-5678",
"AmountPaid": 0
}

Database in JSON (2 of 6)
]
},
{
"_key": "560",
"_id": "Courses/560",
"_rev": "_XU2oaVC--H",
"CourseNumber": 5,
"Course": "Adv Pastels",
"CourseDate": "11/15/2019",
"Fee": 500,
"Enrollments": [
{
"CustomerNumber": 6,
"CustomerLastName": "Kyle",
"CustomerFirstName": "Leah",
"Phone": "425-678-7654",
"AmountPaid": 250
}
]
},

Database in JSON (3 of 6)
{
"_key": "550",
"_id": "Courses/550",
"_rev": "_XU2oaVC--_",
"CourseNumber": 1,
"Course": "Adv Pastels",
"CourseDate": "10/1/2019",
"Fee": 500,
"Enrollments": [
{
"CustomerNumber": 1,
"CustomerLastName": "Johnson",
"CustomerFirstName": "Ariel",
"Phone": "206-567-1234",
"AmountPaid": 250
},
{
"CustomerNumber": 3,
"CustomerLastName": "Jackson",
"CustomerFirstName": "Charles",
"Phone": "360-789-3456",
"AmountPaid": 500
},

Database in JSON (4 of 6)
{
"CustomerNumber": 4,
"CustomerLastName": "Pearson",
"CustomerFirstName": "Jeffery",
"Phone": "206-567-2345",
"AmountPaid": 500
}
]
},
{
"_key": "554",
"_id": "Courses/554",
"_rev": "_XU2oaVC--B",
"CourseNumber": 2,
"Course": "Beg Oils",
"CourseDate": "9/15/2019",
"Fee": 350,
"Enrollments": [
{
"CustomerNumber": 2,
"CustomerLastName": "Green",

Figure C.22(a) The Art Course Database in JSON

(5 of 6)
"CustomerFirstName": "Robin",
"Phone": "425-678-8765",
"AmountPaid": 350
},
{
"CustomerNumber": 5,
"CustomerLastName": "Sears",
"CustomerFirstName": "Miguel",
"Phone": "360-789-4567",
"AmountPaid": 350
}
]
},
{
"_key": "556",
"_id": "Courses/556",
"_rev": "_XU2oaVC--D",
"CourseNumber": 3,
"Course": "Int Pastels",
"CourseDate": "3/15/2019",
"Fee": 350,
"Enrollments": [

Figure C.22(a) The Art Course Database in JSON

(6 of 6)
{
"CustomerNumber": 1,
"CustomerLastName": "Johnson",
"CustomerFirstName": "Ariel",
"Phone": "206-567-1234",
"AmountPaid": 350
}
]
}
]

AQL Query Example

/* *** AQL-QUERY-ExtC-02 *** */

The query searches for course docs that match a criteria, the last line has a nested query
that creates a new document for each customer (student) enrolled in the course.

FOR C in Courses
FILTER C.Fee >400
RETURN {CourseName: C.Course,
CourseDate: C.CourseDate,
StudentNames: (FOR S in C.Enrollments
RETURN {Fname: S.CustomerFirstName,
Lname: S.CustomerLastName})}

Figure C.23 Results of AQL-

QUERYExtC-02 in JSON Format (1 of 2)
[
{
"CourseName": "Adv Pastels",
"CourseDate": "11/15/2019",
"StudentNames": [
{
"Fname": "Leah",
"Lname": "Kyle"
}
]
},
{
"CourseName": "Adv Pastels",
"CourseDate": "10/1/2019",
"StudentNames": [
{
"Fname": "Ariel",

Figure C.23 Results of AQL-

QUERYExtC-02 in JSON Format (2 of 2)
"Lname": "Johnson"
},
{
"Fname": "Charles",
"Lname": "Jackson"
},
{
"Fname": "Jeffery",
"Lname": "Pearson"
}
]
}
]

AQL Query Example

/* *** AQL-QUERY-ExtC-03 *** */

The AQL equivalent of GROUP BY is called COLLECT. The AQL COLLECT . . .

INTO syntax allows us to group the documents of a collection based on an expression

FOR C in Courses
COLLECT CourseFee = C.Fee INTO FeeGroup
RETURN {FeeCharged: CourseFee,
TotalRevenue:
SUM(FLATTEN(FeeGroup[*].C.Enrollments[*].AmountPaid))}

Figure C.24 Results of AQL-

QUERYExtC-03 in JSON Format
[
{
"FeeCharged": 350,

"TotalRevenue": 1050
},
{
"FeeCharged": 500,
"TotalRevenue": 1500
}
]

Microsoft Azure
Learn the basic concepts of the Microsoft Azure cloud environment
for creating and using relational databases
• Azure is Microsoft’s public cloud service that we will
sometimes refer to simply as Azure.
• The SQL Database server on Azure is very similar to, but
not exactly the same as, SQL Server.
– Some of the advanced features of SQL Server are not
supported on Azure’s SQL Database, but for our purposes
they are the same.

Figure C.25 Create Your Free Azure

Account

Azure, Microsoft Corporation

Figure C.26 Azure Dashboard to Manage

Your Services

Azure, Microsoft Corporation

Configuring SQL Database

Azure, Microsoft Corporation

in Azure

Azure, Microsoft Corporation

Screen