ITEC617

Data and Microsoft Certified: Azure Data

Fundamentals
Information
Management
Week 1
Introduction to databases
Lecturer in Charge: Dr Mehdi Rajaeian

Copyright Warning: Slides are subject to

Copyright. All rights reserved for the
respective publisher indicated in the
footnotes/ACU. May not be scanned,
copied or duplicated, or posted to a publicly
accessible website, in whole or in part.
Learning Outcomes
LO1 Describe core data concepts, how to work with relational
and non-relational data and analytics workload on Microsoft
Azure
LO2 Apply information requirements specification processes to
capture the organisation information requirements and model
them using conceptual data modelling and normalisation
techniques
LO3 Develop and implement a database solution using
database and SQL
LO4 Select appropriate database tools to ensure data security
and data quality for various types of database applications to
ensure human dignity

2 |
Learning Resources

Recommended textbook:

Coronel C and Morris S 2022, Database systems: design,

implementation, and management, 14th edn, Course Technology,
Boston, MA.

Further resources:

Kroenke D.M., Auer D.A., Vandenberg L. and Yoder R.C. 2022,

Database Processing: Fundamentals, Design, and
Implementation, 16th edition, Pearson.

LinkedIn Learning Video Training resources (Free for ACU

students) – An excellent resource for self-paced learning.

Online: Microsoft Azure Data Fundamentals modules on

Microsoft Learn

3 |
Technology Requirements

Laptop (Windows or Mac)

4 |
 Assessment Overview
Weighting Learning Graduate
(%) Outcomes Attributes
Assessment tasks assessed assessed
Assessment Task 1: Fortnightly Lab Assessment 25% LO2, LO3 GA5, GA10
Submission Type: Individual – Up to 1 week after each lab
Lab attendance is Compulsory
Assessment Method: Lab Practical Task
Artefact: ER Diagram + SQL code
Assessment Task 2: Certification Exam – Week 12 lab 35% LO1 GA5, GA8
session – On campus

Assessment Task 3: Database design and 40% LO2, LO3, LO4 GA1, GA5,
GA10
Implementation Project
Submission Type: Individual – Exam week 1
Assessment Method: Practical task + Report
Artefact: Database file+ SQL code+ Written report +
Presentation

5 |
What are we doing this week?
• After completing this module, you will be able to:
• Identify characteristics of relational and non-
relational data
• Describe what a database is, and various types of
databases
• Identify data job roles
• Identify common tasks and tools for data job roles
• Define tables, index, view, stored procedure, data
dictionary, system catalog, and indexing

6 |
About
Microsoft Azure
Data Fundamentals course
• Course format:
• online training at
https://aka.ms/AzureLearn_DataFundamentals

Course objectives:
• Describe core data concepts in Azure
• Explain concepts of relational data in Azure
• Explain concepts of non-relational data in Azure
• Identify components of a modern data warehouse in Azure

© Copyright Microsoft Corporation. All rights reserved.

Microsoft Azure Data Fundamentals - Course agenda
Module Lessons
Module 1: • Explore core data concepts
Explore core data concepts • Explore roles and responsibilities in the world of data (optional lesson)
• Describe concepts of relational data
• Explore concepts of non-relational data
• Explore concepts of data analytics

Module 2: • Explore relational data offerings in Azure

Explore relational data in Azure • Explore provisioning and deploying relational database offerings in Azure
• Query relational data in Azure

Module 3: • Explore non-relational data offerings in Azure

Explore non-relational data in • Explore provisioning and deploying non-relational data services in Azure
Azure • Manage non-relational data stores in Azure

Module 4: • Examine components of a modern data warehouse

Explore modern data warehouse • Explore data ingestion in Azure
analytics • Explore data storage and processing in Azure
• Get started building with Power BI

This modules are integrated into the content of this unit.

© Copyright Microsoft Corporation. All rights reserved.

Microsoft Azure Data Fundamentals

Lab environment
The materials in this workbook are designed to be used alongside
online modules in Microsoft Learn. Throughout the course, you’ll find
references to specific Learn modules containing labs that you should
use to gain hands-on experience

© Copyright Microsoft Corporation. All rights reserved.

 What is data?

• Collection of facts, numbers, descriptions, objects , stored in a structured, semi-structured,

unstructured way

Structured Semi-structured Unstructured

Table

© Copyright Microsoft Corporation. All rights reserved.

Transactional vs analytical data
stores
Online Transactional Processing (OLTP) Online Analytical Processing (OLAP)

Customer
CustomerID CustomerName CustomerPhone

Orders
OrderID CustomerID OrderDate

Data is periodically loaded,

aggregated and stored in a cube
Data is stored one transaction at a
time

© Copyright Microsoft Corporation. All rights reserved.

 Transactional workloads
Transactional data is information that tracks the interactions related to an
organization's activities.
Atomicity – each transaction is treated as a single
unit, which success completely or fails completely.
Consistency – transactions can only take the data
in the database from one valid state to another. -$
Isolation – concurrent execution of transactions
leave the database in the same state. +$
Durability – once a transaction has been
committed, it will remain committed.

© Copyright Microsoft Corporation. All rights reserved.

 Analytical Workloads
Analytical workloads are used for data analysis and decision making.

• Summaries
2020 Transactions

• Trends

• Business information

© Copyright Microsoft Corporation. All rights reserved.

 Data Processing
Data processing is the conversion of raw data to meaningful information through
a process.

Batch Processing: data elements are Daily

collected into a group. The whole Batch Job
group is then processed at a future
time as a batch All Input All Output

System System
Stream Processing: each new piece of Upload Upload
data is processed when it arrives.
Input 1 Input 2

© Copyright Microsoft Corporation. All rights reserved.

Introducing the Database
Shared, integrated computer structure that stores data
• End-user data: raw facts of interest to end user
• Metadata: data about data, through which the end-
user data is integrated and managed
• Describes data characteristics and relationships
Database management system (DBMS)
• Collection of programs
• Manages the database structure
• Controls access to data stored in the database

15 |
Role and Advantages of the DBMS
Database management system (DBMS): intermediary
between the user and the database
• Enables data to be shared
• Presents the end user with an integrated view of data
• Provides more efficient and effective data management
• Improves sharing, security, integration, access, decision-
making, productivity, etc.

16 |
Problems with File System Data
Processing
Problems with file systems challenge the types of information that can be
created from data as well as information accuracy
• Lengthy development times
• Difficulty of getting quick answers
• Complex system administration
• Lack of security and limited data sharing
• Extensive programming

17 |
Data Redundancy
Unnecessarily storing the same data at different places
• Islands of information (i.e., scattered data locations)
• Increases the probability of having different versions of
the same data

Possible results of uncontrolled data redundancy

• Poor data security
• Data inconsistency
• Data-entry errors
• Data integrity problems

18 |
Data Anomalies
Develop when not all of the required changes in the redundant data are
made successfully
• Update anomalies
• Insertion anomalies
• Deletion anomalies

19 |
Database Systems
Logically related data stored in a single logical data repository
• Physically distributed among multiple storage facilities
• DBMS eliminates most of file system’s data inconsistency, data anomaly,
data dependence, and structural dependence problems
Current generation DBMS software
• Stores data structures, relationships between structures, and access paths
• Defines, stores, and manages all access paths and components

20 |
Managing the Database System

Challenges of database systems

• Cost
• Management complexity
• Maintaining currency
• Vendor dependence
• Frequent upgrade/replacement cycles

21 |
Preparing for Your Database Professional Career
TABLE 1.3 DATABASE CAREER OPPORTUNITIES

JOB TITLE DESCRIPTION SAMPLE SKILLS REQUIRED

Database Developer Create and maintain database-based Programming, database fundamentals, SQL
applications
Database Designer Design and maintain databases Systems design, database design, SQL

Database Administrator Manage and maintain DBMS and databases Database fundamentals, SQL, vendor courses

Database Analyst Develop databases for decision support QL, query optimization, data warehouses
reporting
Database Architect Design and implementation of database DBMS fundamentals, data modeling, SQL, hardware
environments (conceptual, logical, and knowledge, etc.
physical)

Database Consultant Help companies leverage database Database fundamentals, data modeling, database
technologies to improve business processes design, SQL, DBMS, hardware, vendor-specific
and achieve specific goals technologies, etc.
Database Security Implement security policies for data DBMS fundamentals, database administration, SQL,
Officer administration data security technologies, etc.

Cloud Computing Data Design and implement the infrastructure for Internet technologies, cloud storage technologies, data
Architect next-generation cloud database systems security, performance tuning, large databases, etc.

Data Scientist Analyze large amounts of varied data to Data analysis, statistics, advanced mathematics, SQL,
generate insights, relationships, and predictable programming, data mining, machine learning, data
behaviors visualization

Source: Coronel et al (2019) © Cengage.

22 |
 Roles in data

• Database
• Data Engineer • Data Analyst
Administrator
• Provides insights into the data
Database Management • Data Pipelines and processes • Visual Reporting
Implements Data Security • Data Ingestion storage • Modeling Data for Analysis
Backups • Prepare data for Analytics • Combines data for visualization
User Access • Prepare data for analytical and analysis
Monitors performance processing

© Copyright Microsoft Corporation. All rights reserved.

 Common tools – Database
administrator
• SQL Server
• Azure Data Studio • Azure Portal/CLI
Management Studio

Graphical interface for managing • Graphical interface for • Tools for management and
on-premises and cloud-based data managing on-premises and provisioning of Azure
services cloud-based data services Data Services
Runs on Windows, macOS, Linux • Runs on Windows • Manual and automation of
• Comprehensive Database scripts using Azure Resource
Administration tool Manager or Command Line
Interface scripting

© Copyright Microsoft Corporation. All rights reserved.

 Common tools – Data engineering

• Azure Synapse • SQL Server

• Azure Portal/CLI
Studio Management Studio

Azure Portal integrated to manage • Graphical interface for • Tools for management and
Azure Synapse managing on-premises and provisioning of Azure resources
Data Ingestion (Azure Data cloud-based data services • Manual and automation of
Factory) • Runs on Windows scripts using Azure Resource
Management of Azure Synapse • Comprehensive Database Manager or Command Line
assets (SQL Pools/Spark Pool) Administration tool Interface scripting

© Copyright Microsoft Corporation. All rights reserved.

Common tools – Data analyst

• Power BI Portal/ • Power BI Report

• Power BI Desktop
Power BI Service Builder

Data Visualization tool • Authoring and management of • Data Visualization tool for
Model and Visualize Data Power BI reports paginated reports
• Model and Visualize paginated
Management of Azure Synapse • Authoring of Power BI
assets (SQL Pools/Spark Pool) dashboards reports

• Share Reports/Datasets

© Copyright Microsoft Corporation. All rights reserved.

 Tables
Customers
CustomerID CustomerName CustomerPhone
Data is stored in a table
100 Muisto Linna XXX-XXX-XXXX
101 Noam Maoz XXX-XXX-XXXX Table consists of rows and columns
102 Vanja Matkovic XXX-XXX-XXXX
103 Qamar Mounir XXX-XXX-XXXX
104 Zhenis Omar XXX-XXX-XXXX All rows have same # of columns
105 Claude Paulet XXX-XXX-XXXX
106 Alex Pettersen XXX-XXX-XXXX
Each column is defined by a datatype
107 Francis Ribeiro XXX-XXX-XXXX

© Copyright Microsoft Corporation. All rights reserved.

 Entities
Customers
CustomerID CustomerName CustomerPhone
100 Muisto Linna XXX-XXX-XXXX
101 Noam Maoz XXX-XXX-XXXX
102 Vanja Matkovic XXX-XXX-XXXX
103 Qamar Mounir XXX-XXX-XXXX
104 Zhenis Omar XXX-XXX-XXXX
105 Claude Paulet XXX-XXX-XXXX
106 Alex Pettersen XXX-XXX-XXXX

An entity is a representation of an item which can be physical (such as a customer or a product), or virtual
(such as an order).
Entities are connected by relations enabling interaction. For example, a customer can place an order for a
product

© Copyright Microsoft Corporation. All rights reserved.

 Other common database objects
Views Stored Procedures Indexes
Pre-defined SQL queries that Pre-defined SQL statements that Tree-based structures that
behave as virtual tables can include parameters improve query performance
CREATE VIEW Deliveries CREATE PROCEDURE RenameProduct CREATE INDEX idx_ProductName
AS @ProductID INT, ON Product(Name);
SELECT o.OrderNo, o.OrderDate, @NewName VARCHAR(20)
c.Address, c.City AS
FROM Order AS o JOIN Customer AS c UPDATE Product
ON o.Customer = c.ID; SET Name = @NewName ●
Customer Order WHERE ID = @ProductID; Product
… … … … … … ... A-L M-Z
ID Name Price
… … … … … …
EXEC RenameProduct 201, 'Spanner'; 123 Hammer 2.99

Deliveries Product 162 Screwdrive 3.49

r
OrderNo OrderDate Address City ID Name Price
201 Spanner 4.25
1000 1/1/2022 1 Main St. Seattle 201 Wrench Spanner 4.25
1001 1/1/2022 123 Elm Pl. New York

© Copyright Microsoft Corporation. All rights reserved.

 Normalization
Customers Orders
CustomerID CustomerName CustomerPhone OrderID CustomerName CustomerPhone
100 Muisto Linna XXX-XXX-XXXX AD100 Noam Maoz XXX-XXX-XXXX
101 Noam Maoz XXX-XXX-XXXX AD101 Noam Maoz XXX-XXX-XXXX
102 Vanja Matkovic XXX-XXX-XXXX AD102 Noam Maoz XXX-XXX-XXXX
103 Qamar Mounir XXX-XXX-XXXX AX103 Qamar Mounir XXX-XXX-XXXX
104 Zhenis Omar XXX-XXX-XXXX AS104 Qamar Mounir XXX-XXX-XXXX
105 Claude Paulet XXX-XXX-XXXX AR105 Claude Paulet XXX-XXX-XXXX
106 Alex Pettersen XXX-XXX-XXXX MK106 Muisto Linna XXX-XXX-XXXX

Data is normalized to:

Reduce storage Avoid data duplication Improve data quality

Normalization will be covered in more details in week 3.

© Copyright Microsoft Corporation. All rights reserved.

 Relations
Customers Orders
CustomerID CustomerName CustomerPhone OrderID CustomerID SalesPersonID
100 Muisto Linna XXX-XXX-XXXX AD100 101 200
101 Noam Maoz XXX-XXX-XXXX AD101 101 200
102 Vanja Matkovic XXX-XXX-XXXX AD102 101 200
103 Qamar Mounir XXX-XXX-XXXX AX103 103 201
104 Zhenis Omar XXX-XXX-XXXX AS104 103 201
105 Claude Paulet XXX-XXX-XXXX AR105 105 200
106 Alex Pettersen XXX-XXX-XXXX MK106 105 201

In a normalized database schema:

Primary Keys and Foreign keys are used to No data duplication exists (other than key values Data is retrieved by joining tables together
define relationships in 3rd Normal Form (3NF) in a query

© Copyright Microsoft Corporation. All rights reserved.

Data Dictionary and the System Catalog
Data dictionary
• Description of all tables in the database created by the
user and designer
System catalog
• System data dictionary that describes all objects within
the database

32 | Faculty of Business and Law | Peter Faber Business School

Types of Databases
Microsoft Azure SQL Database

Amazon Relational Database Service

Google Cloud SQL

Oracle Database

IBM Db2 on Cloud

Microsoft SQL Server

MySQL

Oracle

…
Hybrid solutions : Cloud + On-premises

33 |
Types of Databases
Single-user database: supports one user at a time
• Desktop database: single-user database on a personal
computer (example: MS Access)
Multiuser database: supports multiple users at the same time
• Workgroup databases: supports a small number of
users or a specific department
• Enterprise database: supports many users across
many departments

34 |