Cloud Digital Leader Class Notes - Apr 2023

Venue: Virtual

Important references

https://cdl.guide/

Webassor profile :
https://webassessor.com/wa.do?page=createAccount&branding=GOOGLECLOUD

Instructor details:
Dr. Nabil Hadj-Ahmed
[email protected]
https://www.linkedin.com/in/nabil-hadj-ahmed/

General:

Keeping informed:
● Google Cloud Console: https://console.cloud.google.com/
● Sketchnotes : https://thecloudgirl.dev/sketchnote.html
● Google Cloud Platform (GCP) podcast: https://www.gcppodcast.com/
● GCP blog: https://cloud.google.com/blog/
● GCP newsletter: https://cloud.google.com/newsletter
● GCP YouTube channel: https://www.youtube.com/user/googlecloudplatform
● GCP Medium page: https://medium.com/google-cloud
● Recent GCP product changes (release notes): ​https://cloud.google.com/release-notes
● Product specific release notes: https://cloud.google.com/release-notes/all
● **What's new in Google Cloud:
https://cloud.google.com/blog/topics/inside-google-cloud/whats-new-google-cloud
Starter information:
● About Google Cloud Services:
https://cloud.google.com/docs/overview/cloud-platform-services
● **Google cloud developer’s cheat sheet (GCP product list):
○ https://googlecloudcheatsheet.withgoogle.com/
● **Great graphics related to Google Cloud: https://thecloudgirl.dev/sketchnote.html
● **Google Regions: https://cloud.google.com/about/locations
● Case studies and customers: https://cloud.google.com/customers/

What is Cloud Computing?

The practice of using a network of remote services hosted on the internet to store, manage and
process data rather than a local server or PC (personal computer).

On-premises Public Cloud

- You purchase and house your own - You rent servers

servers - You rent experts/people
- You have to hire people to manage the - You are only responsible for the
servers configuration and code
- You own the risk - Shared responsibility model

Common Cloud Terminology

GCP: Google Cloud Platform

Availability: your ability to ensure a service remains available.

Scalability: your ability to grow rapidly or unimpeded .

Elasticity: your ability to shrink and grow to meet the demand.

Disaster Recovery: your ability to recover from a failure.

Load Balancer: a load balance allows you to evenly distribute traffic to multiple servers in one or

more data centers.

High Availability (HA): ability for your service to remain available by ensuring there is no single

point of failure and or ensure a certain level of performance.

Highly Scalable: your ability to increase your capacity based on the increasing demand of traffic,

memory, and computer power.

Vertical scaling: or “scaling-up”, which is upgrading to a bigger server.

Horizontal Scaling: or “scaling-out”, which is adding more servers of the same size or

“scaling-in”, which is removing servers.

High Elasticity: ability to automatically increase or decrease your capacity based on the current

demand of traffic memory or computing power.

NOTE: the difference between high elasticity and highly scalable is that elasticity involves automation in either

scaling up or down.

Failover: is when you have a plan to shift traffic to a redundant system in case the primary

systems fail.

High Durability: your ability to recover from a disaster and to prevent the loss of data. Solutions

that recover from a disaster are known as disaster recovery (DR).

The Evolution of Computing

Dedicated Server Virtual Private Server Shared Hosting Cloud Hosting

One physical One physical One physical Multiple physical

machine for a single machine dedicated to machine shared by machines that act as
business and a single a single business, but hundreds of one with multiple
app on site. runs multiple businesses, relies on cloud services.
applications on a site most tenants under
on virtualized utilizing their
sub-machines. resources.

Dedicated Server (pros/cons)

- A physical server wholly utilized by a single customer
 - Customers overpay for an underutilized server
- You can’t vertical scale, you need manual migration
- Replacing a server is difficult
- You are limited by your host operating system (OS)
- Multiple apps can result in conflicts in resource sharing
- You have a ‘guarantee’ of security, privacy and full utility of underlying resources
(guarantee is only based on the skills of your IT team)

Virtual Machines (pros/cons)

- Can run multiple VMs (virtual machines) on one machine
- Hypervisor is the software layer that lets you run the VM
- A physical server shared by multiple customers
- Pay for a fraction of the server
- You will overpay for an underutilized VM
- Limited by your guest operating system
- Multiple apps on a single VM can result in conflicts in resource sharing
- Easy to export or import images for migration
- Easy to vertical or horizontal scale

Containers (pros/cons)
- VM runs multiple containers
- You can maximize the utilization of the available capacity which is most cost
effective
- Your containers share the same underlying OS so containers are more efficient
than multiple VMs
- Multiple apps can run side by side without being limited to the same operating
system requirements and will not cause conflicts during resource sharing

Functions (pros/cons)
- Are managed VMs running managed containers (known as serverless compute)
- You upload a piece of code, choose the amount of memory and duration
- Only responsible for the code/data
- Very cost effective, only pay for the time code is running, VMs only run when
there is code to be executed
- Cold start (booting up) is a con

Types of Environments
 1. On-premises - sometimes called ‘private cloud’
2. Public Cloud - sometimes called ‘cloud native’ (Azure, AWS, Google Cloud, etc)
3. Hybrid - mixture of of both on-premises and public cloud
4. Multi Cloud - mixture of different public clouds, sometimes called ‘cross cloud’

Databases and Data Analytics

What is a database?

A database is a data-store that stores semi-structured and structured data. Databases have a
rich set of functionality to either use a specialized language to query (retrieve data)​specialized
modeling strategies to optimize retrieval for different use cases​.

What is structured and unstructured data?

Structured data consists of clearly defined data types with patterns that make them easily
searchable; while unstructured data – “everything else” – is composed of data that is usually
not as easily searchable, including formats like audio, video, and social media postings.

Typical types of databases

Relational Databases​ Non-relational Databases​

- Structured data that strongly - Semi-structured that may or may not

represents tabular data (tables, rows distantly resemble tabular data
and columns)​
- Row-oriented or Columnar-oriented

What is a data warehouse?

A data warehouse is a relational datastore designed for analytic workloads,

What is a Document store?​

A document store is a NoSQL database that stores documents as its primary data structure.​A
document could be an XML but more commonly is JSON or JSON-Like​

What is a Key/Value store?

A key-value database is a type of non-relational database (NoSQL) that uses a simple key-​value
method to store data.​A key/value stores a unique key alongside a value​.

What is a data lake?

A data lake is a centralized repository designed to store, process, and secure large amounts of
structured, semistructured, and unstructured data. It can store data in its native format and
process any variety of it, ignoring size limits.

NoSQL: Cloud Run large analytical and operational workloads using

Databases
Key-value Bigtable this fully managed, scalable NoSQL database service.

NoSQL: A highly scalable NoSQL database for your web and

Databases Datastore
Indexed mobile applications.

Manage relational data for MySQL, PostgreSQL, and

Databases RDBMS Cloud SQL
SQL Server for workloads under 64TB.

Easily develop rich applications using a fully managed,

Document scalable, and serverless document database for both
Databases Firestore
data storage web and mobile apps. Key feature: Able to store and
sync data in realtime.

Database
SQL database Migrate databases to Cloud SQL from on-premises,
Databases Migration N/A Compute Engine, and other clouds
migration
Service

In-memory Reduce latency with scalable, secure, and highly

Databases Memorystore
data store available in-memory service.

Fully managed relational database with massive scale,

RDBMS for Cloud
Databases strong consistency worldwide, and up to 99.999%
global reach Spanner
availability.

SERVERLESS Serverless, highly scalable, and cost-effective

Data Analytics Data BigQuery multi-cloud data warehouse designed for business
Warehouse agility. Built-in ML.

Deploy open-source data and analytics (batch)

Open Source
Data Analytics Dataproc processing services (Apache Hadoop, Apache
Processing
Spark, etc.) with improved efficiency and security.
 Business intelligence software and big data analytics
Business
Data Analytics Looker platform that helps you explore, analyze and share
Intelligence
real-time business analytics easily.

Quickly build and manage data pipelines using

Data Cloud Data
Data Analytics fully managed, code-free data integration with a
Integration Fusion
graphical interface​

Data
Discover and understand your data using a fully
discovery and
Data Analytics Data Catalog managed and scalable data discovery and metadata
metadata
management service.​
management

Serverless
Unify stream and batch data processing that's
Data Analytics data Dataflow
serverless, fast (real-time), and cost-effective.
processing

Create, schedule, monitor, and manage workflows

Workflow Cloud
Data Analytics using a fully managed orchestration service built on
orchestration Composer
Apache Airflow.​

Streaming analytics and data integration pipelines to

ingest and distribute data. It is equally effective as a
Data Analytics Messaging Pub/Sub
messaging- oriented middleware for service integration
or as a queue to parallelize tasks.

Dataprep by Trifacta is an intelligent data service for

Intelligent data visually exploring, cleaning, and preparing structured
Data Analytics Dataprep
preparation and unstructured data for analysis, reporting, and
machine learning.

Important notes for databases and analytics for exams.

- Understand BigQuery, Bigtable and Spanner.

- Remember that Firestore has a realtime feature.
- Know the difference between Dataflow, Dataproc, and Cloud Data Fusion.

Benefits of Cloud Computing

1. Cost Effective: you pay for what you consume, no upfront cost. On-demand pricing or
pay as you go with thousands of customers sharing the cost of resources.
2. Global: launch workloads anywhere in the world, just choose a region.
3. Secure: Cloud providers take care of the physical security. Cloud services are secure by
default or you have the ability to configure access down to a granular level.
4. Reliable: data backup, disaster recovery, data replication and fault tolerance.
 5. Scalable: Increase or decrease resources and services based on demand.
6. Elastic: automate scaling during spikes and drops in demand.
7. Current: the underlying hardware and managed software is patched, upgraded and
replaced by the cloud provider without service interruptions.

Types of Cloud Computing

Software as a Services (SaaS): generally for end user

customers. Examples include Gmail or Twitter.

Platform as a Service (PaaS): mainly for developers who

don’t want to worry about operating systems.
Examples include Google AppEngine

Infrastructure as a Service (IaaS): the cloud providers

handle compute, networking, hardware etc. Example
Compute Engine

Shared Responsibility Model for Google Cloud

The chart below defines what the customer is responsible for VS what Google Cloud is
responsible for.
Shared Responsibility Rules
1. The customer is responsible for the data and the configuration of access controls that
reside in Google Cloud.

2. The customer is responsible for the configuration of cloud services and granting
access to users via permissions.

3. Google is generally responsible for the underlying infrastructure.

 Shared Responsibility Model for Compute

Infrastructure as a Service (IaaS)

Bare Metal - Compute Engine Virtual Machine - Computer Engine Containers - Google Kubernetes
Engine (GKE)

Customer Customer Customer

- The host OS - The guest OS configuration - Configuration of
configuration - Container runtime containers
- hypervisor Google - Deployment of
Google - Hypervisor, physical machine containers
 - Physical machines - Storage of containers
Google
- The OS, the hypervisor,
container runtime

Platform as a Service Software as a Service Function as

(PaaS) (SaaS) a Service (FaaS)
Google App Engine Google Doc/ Google Sheets Google Cloud Functions

Customer Customer Customer

- Uploading code - Contents of documents - Upload your own code
- Some configuration of - Management of files Google
environment - Configuration of sharing - Deployment, container
- Deployment strategy access runtime, networking,
- Configuration of Google storage, security,
associated services - Servers, OS, networking, physical machine
(basically everything)
Google storage, security
- Servers, OS, networking, Note: Function as a Service is
storage, security sometimes called ‘Serverless
Computing’

Total Cost of Ownership (TCO)

CapEx (on-premises): money used by a company to acquire, upgrade, and maintain physical
infrastructure assets such as data centers, people, buildings, electricity, technology, and
equipment. Often hidden costs that we don't immediately think of, shown below in the iceberg
analogy.

OpEx (cloud): are the day-to-day expenses a company incurs to keep its business operational.
Less hidden costs because infrastructure is managed by public cloud providers.

Global Infrastructure Terminology

Regions: are independent geographic areas that consist of zones - smaller areas within a
region.
Zones: physical locations made up of one or more data centers or hardware.

Edge Networking: is the practice of having compute and data storage resources as close as
possible to the end user in order to deliver the lowest latency and to save bandwidth.

Points of Presence (POP): is an intermediate location between a GCP region and the end user.
This location could be a third party datacenter or collection of hardware.

Edge POP: a location where a user can quickly enter (ingress) the GCP network for accelerated
access to cloud resources.

CDN POP: a location to serve (egress) cached website files and assets so they load very fast for
the end user.

Cloud Media Edge: a location specialized for the delivery of media such as video content.

Resource Scoping Terms

Zonal Resource: a resource that resides in multiple zones in a single region.

Multiple-regional Resource: a resource that resides across multiple specified zones.

Global Service: a resource that resides globally and regions/zones are abstracted away.

Internal Services: a foundation service used by many other services. You don't interact with
these services directly, they are managed by Google.

Data Residency: the physical or geographical location of where an organization's data or cloud
resources reside.

Compliance Boundaries: a regulatory compliance by a government or organization that

describes where data and cloud resources are allowed to reside. For example, hospitals and
private health data.

Assured Workloads: a feature that allows you to apply various ‘security controls’ based on
attributes, personal support case ownership controls and encryption.
Cloud Interconnect
Cloud Interconnect provides direct physical connections between your on-premises network and
Google’s network. Cloud Interconnect enables you to transfer large amounts of data between
networks, which can be more cost-effective than purchasing additional bandwidth over the
public internet.

Two Offerings | Dedicated or Partner

Dedicated: a direct physical connection between the on-premises network and Google’s network
through a co-location. (Ideal for 10-200GB).

Partner: a direct physical connection between the on-premises network and Google's network
through a trusted third-party. (Ideal for 50MB-10GB)

Latency and Lag

Latency: the time delay between two physical systems.

Lag: the noticeable delay between the actions of input and the reactions of the server sent back
to the client.

Important for exam:

Inter-regional Latency: triple digit milliseconds (500ms)
Inter-zonal Latency: double digit milliseconds (10ms)

Computing Power for GCP

CPU or Central Processing Unit: CPU processors are found in Google's offering Compute
Engine. These are basic CPUS and are normally Xeon.

Tensor Computing: TPUs or Tensor Processing Units are 50x faster than traditional CPUs,
created by Google and are cooled with water. TPUs are great for AI/ML and the Google product
is called Cloud TPU.

Quantum Computing: 100 million times faster. Will be essential to super artificial intelligence
and Google's products are Google Quantum AI. The latest quantum computing chip is called
Sycamore (2018), followed by Bristlecone (2017) and then Foxtail (2016)
Google Cloud Tools
Google Cloud Console: the GCP portal is a web-based, unified console, that you can build,
manage, and monitor everything from a web browser.

Cloud SDK: SDK means software development kit and it is a collection of software development
tools in one installable package. Cloud SDK is used to programmatically create, modify, delete or
interact with GCP resources. Cloud SDK is offered in various languages: JAVA, Python, Nodejs,
Ruby, Go, .Net and PHP.

Cloud CLI: CLI means command line interface and it processes commands to a computer
program in the form of lines of text.

Cloud Shell: command line access for managing your infrastructure (basically the code editor in
the GCP console for managing your infrastructure).

Compute Services and Products

PRODUCT GCP
CATEGORY ICON DESCRIPTION
TYPE PRODUCT
Serverless Application platform for apps and back
Compute/ Serverless ends. Build highly scalable applications on a fully
Serverless Applications App Engine managed serverless platform. Build your application in
Computing Platform Node.js, Java, Ruby, C#, Go, Python, or PHP—or
bring your own language runtime.

Virtual machines running in Google's data center.

Compute
Compute Secure and customizable compute service that lets
Compute Service (VMs
Engine you create and run virtual machines on Google’s
in the Cloud) infrastructure.

Server and virtual machine migration to Compute

Migrate for Engine. With Migrate for Compute Engine simple “as a
VM Migration
Compute Compute N/A service” interface within Cloud Console and flexible
as a service
Engine migration options. Migrate VM instances to Google
Cloud from AWS, Azure, or VMWare VSphere.
 Dedicated & Sole-tenancy lets you have exclusive access to a
Physical Sole-Tenant sole-tenant node, which is a physical Compute Engine
Compute N/A server that is dedicated to hosting only your project's
Compute Nodes
Engine Server VMs.

Easily lift and shift your VMware-based applications to

VMware app VMware Google Cloud without changes to your apps, tools, or
Compute migration as a Engine N/A processes. The service provides all the hardware and
service (GCVE) VMware licenses you need to run in a dedicated
VMware SDDC in Google Cloud.

Develop and deploy highly scalable containerized

Containers
applications on a fully managed serverless platform.
Compute without Cloud Run
Run stateless containers on a fully managed
infrastructure environment or on Anthos.​

Add GPUs (graphics processing unit) to your

Compute Core compute Cloud GPUs workloads for machine learning, scientific computing,
and 3D visualization.​

Cloud cost Optimize your Google Cloud usage with proactive,

Compute Recommender N/A easily actionable recommendations
optimization

Run your code with zero server management with this

scalable, pay-as-you-go functions-as-a-service (FaaS)
Serverless Cloud
FaaS offering.
Computing Functions
Scalable to Zero

Serverless Workflow Orchestrate and automate Google Cloud and

Workflows
Computing orchestration HTTP-based API services with serverless workflows.

App Engine Environments

App Engine is a Platform as a Service (PaaS) for your application where you can quickly deploy
and scale web-applications without worrying about the underlying infrastructure.​

App Engine has two types of environments: Flexible and Standard​. You can simultaneously use
both environments for your application.​App Engine is well suited to applications that are
designed using a microservice architecture​.

Standard (serverless compute)​ Flexible (fully managed containers)​

- starts in seconds​ - starts in minutes​

- Runs in a sandbox​ - Runs within Docker Containers on
- designed for rapid scaling (sudden Compute Engine (VMs)​
traffic spikes)​
 - supports specific language versions, - designed for predictable and
not custom runtime​ consistent traffic​
- can scale to zero instances (scale to - supports generally any language
zero)​ version or run custom time​
- pricing based on hours​ - must have at least once instance
- cannot SSH to debug​ running​
- no background processes​ - Supports autoscaling
- Flexible fully managed container - pricing based on vCPUs, Memory and
Disks​
- can SSH to debug​
- can have background processes​

Kubernetes and Containers

Quick Facts about Kubernetes:
- The name Kubernetes originates from a Greek word meaning helmsmen or steersmen,
hence the wheel of a ship as the logo
- It was initially called Project Seven of Nine, based on the female Borg character in Star
Trek, which is why there are seven spokes in the wheel
- Kubernetes can also be referred to as K8s (K + the eight letters of ubernete)
- It was designed by Google in 2014, but was donated to the Cloud Native Computing
Foundation

What are Kubernetes and containers?

Containers: is a standard unit of software that packages up code and all its dependencies, so
the application runs quickly and reliably from one computing environment to another.”

Container Management: is the process of organizing, adding, or replacing large numbers of

software containers. This leads to the need for container-orchestration, which is basically
Kubernetes, a tool that automates scaling, management, and cloning of your containers plus
much more.

Kubernetes: is an open source container-orchestration platform for your applications that can
be deployed on your on-premises or cloud environment. It can regulate your cloud costs, but
most importantly it offers high-availability to your applications by being able to automatically
spin up identical containers and offload them when no longer needed.

A unique component of kubernetes is pods. A pod is a group of containers with shared storage,
network resources and other shared settings.
Kubernetes is ideal for microservice-architectures where a company has tens to hundreds of
services they need to manage. Best choice for when you have A LOT of services.

Container Store, manage, and secure your container images.

Container
Containers registry & Deploy only trusted containers on GKE.
Registry
security Only multi-regional repositories.

Considered as the next generation of container

Container
Artifact registry. A fully-managed service with support for
Containers registry &
Registry both container images and non-container artifacts.
security

Continuously build, test, and deploy containers

Containers CI/CD Cloud Build
using the Google Cloud infrastructure.​

Containers Develop and deploy highly scalable containerized

Containers without Cloud Run applications on a fully managed serverless
Infrastructure platform.

Container Container Secure your container environment on Google Cloud,

Containers N/A GKE, or Anthos.
security Security

Google
Secured and managed Kubernetes service with
Containers CaaS Kubernetes
four-way auto scaling and multi-cluster support.
Engine (GKE)

Deep Take advantage of containers preconfigured with data

AI Container
Containers Learning N/A science frameworks, libraries, and tools. Efficiently run
Management
Containers​ batch jobs using Kubernetes.​

Kubernetes
applications
Pre-built
Containers on Google N/A Deploy pre built containerized apps.​
Containers
Cloud
Marketplace