Unit - I
Unit - I
UNIT I - DEVOPS
DevOps -Definition - Practices -DevOps life cycle process - need for DevOps -
Barriers - Introduction To AWS, GCP, Azure - Version control systems: Git and
Github.
What is DevOps?
Why DevOps?
Definition:
DevOps Architecture
Development and operations both play essential roles in order to deliver applications.
The deployment comprises analyzing the requirements, designing, developing,
and testing of the software components or frameworks.
The operation consists of the administrative processes, services, and support for the
software. When both the development and operations are combined with
collaborating, then the DevOps architecture is the solution to fix the gap between
deployment and operation terms; therefore, delivery can be faster.
DevOps architecture is used for the applications hosted on the cloud platform and
large distributed applications.
Agile Development is used in the DevOps architecture so that integration and delivery
can be contiguous.
When the development and operations team works separately from each other, then it
is time-consuming to design, test, and deploy.
And if the terms are not in sync with each other, then it may cause a delay in the
delivery. So DevOps enables the teams to change their shortcomings and increases
productivity.
1) Build
Without DevOps, the cost of the consumption of the resources was evaluated based on
the pre-defined individual usage with fixed hardware allocation. And with DevOps,
the usage of cloud, sharing of resources comes into the picture, and the build is
dependent upon the user's need, which is a mechanism to control the usage of
resources or capacity.
2) Code
Many good practices such as Git enables the code to be used, which ensures writing
the code for business, helps to track changes, getting notified about the reason behind
the difference in the actual and the expected output, and if necessary reverting to the
original code developed. The code can be appropriately arranged in files, folders, etc.
And they can be reused.
3) Test
The application will be ready for production after testing. In the case of manual
testing, it consumes more time in testing and moving the code to the output. The
testing can be automated, which decreases the time for testing so that the time to
deploy the code to production can be reduced as automating the running of the scripts
will remove many manual steps.
4) Plan
DevOps use Agile methodology to plan the development. With the operations and
development team in sync, it helps in organizing the work to plan accordingly to
increase productivity.
5) Monitor
Continuous monitoring is used to identify any risk of failure. Also, it helps in tracking
the system accurately so that the health of the application can be checked. The
monitoring becomes more comfortable with services where the log data may get
monitored through many third-party tools such as Splunk.
6) Deploy
Many systems can support the scheduler for automated deployment. The cloud
management platform enables users to capture accurate insights and view the
optimization scenario, analytics on trends by the deployment of dashboards.
7) Operate
DevOps changes the way traditional approach of developing and testing separately.
The teams operate in a collaborative way where both the teams actively participate
throughout the service lifecycle. The operation team interacts with developers, and
they come up with a monitoring plan which serves the IT and business requirements.
8) Release
DevOps Lifecycle
DevOps defines an agile relationship between operations and Development. It is a
process that is practiced by the development team and operational engineers together
from beginning to the final stage of the product.
Learning DevOps is not complete without understanding the DevOps lifecycle phases.
The DevOps lifecycle includes seven phases as given below:
1) Continuous Development
This phase involves the planning and coding of the software. The vision of the project
is decided during the planning phase. And the developers begin developing the code
for the application. There are no DevOps tools that are required for planning, but there
are several tools for maintaining the code.
2) Continuous Integration
This stage is the heart of the entire DevOps lifecycle. It is a software development
practice in which the developers require to commit changes to the source code more
frequently. This may be on a daily or weekly basis. Then every commit is built, and
this allows early detection of problems if they are present. Building code is not only
involved compilation, but it also includes unit testing, integration testing, code
review, and packaging.
3) Continuous Testing
This phase, where the developed software is continuously testing for bugs. For
constant testing, automation testing tools such as TestNG, JUnit, Selenium, etc are
used. These tools allow QAs to test multiple code-bases thoroughly in parallel to
ensure that there is no flaw in the functionality. In this phase, Docker Containers can
be used for simulating the test environment.
4) Continuous Monitoring
Monitoring is a phase that involves all the operational factors of the entire DevOps
process, where important information about the use of the software is recorded and
carefully processed to find out trends and identify problem areas. Usually, the
monitoring is integrated within the operational capabilities of the software
application.
5) Continuous Feedback
The application development is consistently improved by analyzing the results from
the operations of the software. This is carried out by placing the critical phase of
constant feedback between the operations and the development of the next version of
the current software application.
6) Continuous Deployment
In this phase, the code is deployed to the production servers. Also, it is essential to
ensure that the code is correctly used on all the servers.The new code is deployed
continuously, and configuration management tools play an essential role in executing
tasks frequently and quickly.
7) Continuous Operations
All DevOps operations are based on the continuity with complete automation of the
release process and allow the organization to accelerate the overall time to market
continuingly.
Barriers in Devops
1. Cultural Barriers
Resistance to change
Lack of collaboration between Development and Operations
Fear of job displacement
Difficulty in shifting to a shared responsibility model
2. Organizational Barriers
Siloed teams and departments
Hierarchical and rigid organizational structure
Misaligned objectives and incentives between teams (Dev vs Ops)
Lack of leadership support for DevOps transformation
3. Technical Barriers
Legacy systems and infrastructure
Lack of tool integration or incompatible tools
Complexity in automating existing systems
Difficulty in scaling infrastructure (especially in hybrid or cloud environments)
4. Process Barriers
Manual processes and workflows
Slow or lengthy release cycles
Lack of proper feedback loops (metrics and monitoring)
Poor incident management and resolution processes
5. Security Barriers
Traditional security processes conflicting with speed of DevOps
Insufficient security awareness or expertise in DevOps teams
Difficulty in automating security testing (DevSecOps)
Fear of compromising security for speed
6. Resource Barriers
Lack of skilled personnel (e.g., DevOps engineers, automation specialists)
Budget limitations for tools, infrastructure, and training
Insufficient time for implementing and scaling DevOps
7. Tooling Barriers
Overwhelming number of DevOps tools and technologies
Tool compatibility and integration issues
Tool sprawl (using too many tools, leading to complexity)
Resistance to adopting new tools or technologies
Introduction to AWS
Amazon Web Services (AWS):
* AWS stands for Amazon Web Service. It is the most comprehensive cloud service
provider by Amazon.
* AWS manages and maintains hardware and infrastructure, saving organizations and
individuals the cost and complexity of purchasing and running resources on-site.
These resources may be accessed for free or on a pay-per-use basis.
* It provides different services of cloud such as Infrastructure as a Service (IaaS),
Platform as a Service (PaaS) and packaged Software as a Service (SaaS),
* PaaS: The platform as a service means it is a service where a third party provider
provides both hardware and software tools to the clients. It provides elastic scaling of
your application which allows developers to build applications and services over the
internet and the deployment models include public, private and hybrid. PaaS is used
by developers. Examples of cloud services - Facebook and Google Search Engine.
SaaS: Software as a Service model that hosts software to make it available to clients.
It is used by the end users. Examples of cloud services Google Apps.
Features of AWS
1) Flexibility:
* The AWS always allows the user to use the operating system, programming
languages and web application platforms that the user is comfortable with.
* Flexibility means that migrating legacy applications to the cloud should be easy.
2) Cost effective:
Instead of purchasing and creating our own expensive servers, we can use AWS
where we need to pay only for the tools and service that we use.
AWS offers a pay-as-you-go pricing method, which means that we only pay for the
services that are needed and have been used for a period of time.
AWS is scalable because the AWS auto scaling service automatically increases the
capacity of constrained resources as per requirements so that the application is always
available.
* Elasticity is one of the AWS advantages. If we use fewer resources and do not need
he rest of them, the AWS itself shrinks the resources to fit our requirements. In short,
upsizing. and downsizing of resources is possible with AWS
4) Secure:
5 High performances
High performance computing is the ability to process massive amounts of data at high
speed.
AWS often a high-performance computing service so that the companies need not
worry about the spend.
Applications of AWS
The most five important services provided by Amazon Web Services are -
1) Amazon Elastic Cloud Compute(EC2):
EC2 stands for Elastic Cloud Compute.
Amazon EC2 is one of the most used and most basic services on Amazon.
EC2 is a machine with an operating system and hardware components of
developer's choice, But the difference is that it is totally virtualized. The
developer can run multiple virtual computers, in a single physical hardware.
EC2 enables on-demand, scalable computing capacity in the AWS cloud.
No upfront investments on hardware is required, we only pay for what we use.
The most important thing is that EC2 is secure.
Features
Following are the reasons why Google Cloud Platform-
1) Tools and services:
Google has many innovative tools for handling cloud data.
Google Cloud Platform offers the same core data storage and virtual machine
functionality of AWS and Azure or any other cloud provider. Google's strength.
lies in big data processing tools, Artificial Intelligence (AI) and machine learning
initiatives and container support.
Google's BigQuery and Dataflow bring strong analytics and processing
capabilities for companies that work heavily with data, while Google's
Kubernetes container technology allows for container cluster management,
GCP offers number of physical, logistical and human-resource-related
components, such as wiring, routers, switches, firewalls, load balancers services.
2) Speed:
Google Cloud provides its Google Cloud and Google App customers network
speeds of up to 10 Tbs
The network has connections throughout the world in the United States, Europe,
main cities in Japan, major hubs in Asia and much more.
3) Cost effective:
Google Cloud offers a monthly pricing plan. The price model of GCP is
awesome, it allows you to pay only for what we use.
Google Cloud pricing provides discounts.
4) Security:
Advantages
1) GCP offers a first rate security level. A big advantage is that the company
constantly releases updates, adds improvements to security.
2) In comparison with competitors, like Amazon Web Services, Digital Ocean, or
Microsoft Azure, GCP's advantage is the opportunity to not pay any extra money for
unused services. Google enables users to get Google Cloud hosting at the cheapest
rates.
3) Once the account is configured on GCP, it can be accessed from anywhere. That
means that the user can use GCP across different devices from different places.
4) Google always keeps backup of user's data with built-in redundant backup
integration.
5) Migrating to Google's Cloud Platform eliminates the need to buy on-premises
infrastructure.
6) There is Global Presence of Data Centers, Networks and Cloud Services by GCP.
Services using GCP
2) Networking
GCP Provides Networking services tools to manage and scale your network easily.
It offers following suit of services -
Virtual Private Cloud (VPC): VPC provides the flexibility to scale and control
how workloads connect regionally and globally. By using VPC you can bring
your own IP addresses to google network across all the regions.
Cloud DNS: Google Cloud DNS (Domain name system) is scalable, reliable and
managed authoritative DNS service. Cloud DNS is also programmable.
Cloud SQL: It is a web-service that enables users to create, manage and use
relational databases stored on Google Cloud servers.
4) Big Data
Big data is the massive amount of data available to organizations that because of
its volume and complexity is not easily managed or analyzed by marry business
intelligence tools
BigQuery is a data warehouse that processes and analyzes large data sets using
SQL queries. These services can capture and examine streaming data for real-
time analytics
Google Cloud Dataproc is a managed Apache Hadoop and Spark service for
batch processing, querying, streaming and machine learning.
Cloud Dataflow is a serverless stream and batch processing service. Users can
build a pipeline to manage and analyze data in the cloud, while Cloud Dataflow
automatically manages the resources.
5)Cloud Al
Google Cloud Machine Learning Engine is useful to rum complex ML. models. It
is scalable across different machines. It supports the increased performance of
linear algebra computations.
AutoML enables developers with limited machine learning expertise to build
custom ML. models in minutes. It is a cloud-based ML. platform and uses a No-
Code approach with a set of prebuilt models via APIs.
Microsoft Azure :
Features
4) Security: Data security and compliance are top priorities for any business and
Microsoft Azure provides a range of features and services to help organizations
protect their data in the cloud.
5) Cost effective: With Azure, there are no upfront costs for hardware or
infrastructure and organizations only pay for the services they use. This means that
businesses can reduce their capital expenditures and operational expenses, while still
having access to the latest technology and features.
6) Strong support in analytics: Microsoft Azure is equipped with built-in support that
analyzes data and key insights. The service provides features such as Cortana
Analytics, Stream Analytics, Machine Learning, and SQL services. These features
will help to enhance customer service and make informed decisions.
The Azure compute domain allows its user to host the model for the computing
resources on which our application can run. Following are some main options
available under compute domain of Azure-
App Service:
This service allows hosting web apps, mobile apps with back ends, automated
processes and RESTful APIs.
It is a managed PaaS offering.
Virtual Machine:
It is an IaaS service.
Virtual Machines, or VMs, are software emulations of physical computers.
They include a virtual processor, memory, storage and networking resources.
They host an Operating System (OS), of our choice and run software just like a
physical computer.
And by using a remote desktop client, we can use and control the virtual machine.
Functions:
It is a server-less computing service,
It is a service that allows developers to quickly create and deploy small pieces of
back- end code, or "functions", without having to worry about the underlying
infrastructure,
Users can upload applications with multiple functions.
Azure function can be written in a variety of languages, including c#,
F#,Node.js,Python,powershell.
2) Storage:
Blob Storage:
BLOB is an acronym and means Binary Large OBject.
Azure Blob storage is a service that stores unstructured data in the cloud as
objects/blobs.
Blob storage can store any type of text or binary data, such as a document, media
file, or application installer.
Blob storage is also referred to as object storage.
Queue Storage:
Queue storage in Azure is a fully managed, cloud-based service for storing and
retrieving large numbers of messages.
It is used to build asynchronous, loosely-coupled, scalable, and reliable
applications by allowing communication between microservices,
A single queue message can be up to 64 KB in size, and a queue can contain
millions of messages, up to the total capacity limit of a storage account.
File Storage:
File Storage in Azure is a fully managed, cloud-based file storage service that
allows applications to access and store files using the standard Server Message
Block (SMB) protocol.
It provides a fully managed network file share that can be accessed from
anywhere and by multiple concurrent clients.
An account can contain an unlimited number of shares and a share can store an
unlimited number of files, up to the 5 TH total capacity of the file share
3) Database:
Using various database services from the Azure users can easily deploy one or more
databases to virtual machines or applications.
The core database services available in Microsoft Azure revolve around the following
five offerings.
Cosmos DB, MySQL, PostgreSQL, MS SQL and SQL Managed Instance.
Cosmos DB:
zure Cosmos DB is a globally distributed, multi-model database service that
supports various data models and query languages.
It is designed for building highly scalable, globally distributed applications that
require low latency and high availability.
It is a NoSQL data store that is available in Azure.
MySQL:
Azure Database for MySQL is a relational database service powered by the
MySQL community edition.
Azure Database for MySQL is cost effective and easy to set up, operate and
scale.
MSSQL:
4) Networking:
Auto-scaling and load balancer are two important features in network services of
Azure:
Auto scaling service is used in a cloud to scale out or scale in the virtual machine
instances according to some condition, for instance - If CPU usage is heavy then we
can increase the virtual machine instances and if it drops to some limit then we can
reduce the virtual machine instance.
4) Networking:
Auto-scaling and load balancer are two important features in network services of
Azure:
Auto scaling service is used in a cloud to scale out or scale in the virtual machine
instances according to some condition, for instance - If CPU usage is heavy then we
can increase the virtual machine instances and if it drops to some limit then we can
reduce the virtual machine instance.
Load balancing is a feature that balances the traffic between different servers. It is
used to divide the traffic between the different servers or if one of the machines stops
responding it directs its traffic to the different server. In the cloud, we could use the
load balancer to balance the traffic between our virtual machines.
Version control systems are a category of software tools that helps in recording
changes made to files by keeping a track of modifications done in the code.
Features of Git
1) It is a free and open source tool.
2) It keeps track of any addition, modification and deletion of any file in the project
repository.
3) It creates backups.
4) Creating branches and making changes on particular branches is easy.
5) This tool is used for project management in distributed environments.
6) It is scalable.
7) Git allows users from all over the world to perform operations on a project
remotely. Thus it helps in non-linear development.
8). It is secure. Git keeps a record of all the commits done by each of the collaborators
on the local copy of the developer.
Concept of GitHub
GitHub is a web-based hosting service for Git repositories. It makes Git more
user-friendly and also provides a platform for developers to share code with
others.
Additionally, GitHub repositories are open to the public. Developers worldwide
can interact and contribute to one another's code, modify or improve it, making
GitHub a networking site for web professionals.
The process of interaction and contribution is also called social coding.
What is a repository?
Repository means a storage space or directory that holds all the project's files.
Each file has revision history. It is also called a repo.
The repository helps to communicate and manage project's work systematically.
We can keep files and images inside our repository.
Features
1) It makes the project management smooth by keeping track of all the updates made
in the project.
2) We can push the package privately and secure the projects from being public.
Similarly we can distribute the work publicly if we want to do so.
3) GitHub helps the team members to review, develop and propose new code. It
allows users to share the changes in the code among themselves.
4) Code hosting is easy using GitHub
5) It helps in streamlining the project workflow
6) It allows effective team management.
7) It is intuitive and easy to learn.
Git is focused on version control and code GitHub is focused on centralized source code
5.
sharing. hosting.
Git has no user management feature. GitHub has a built-in user management
8.
feature.
10. Git has minimal external tool GitHub has an active marketplace for tool
S.No. Git GitHub
configuration. integration.
Git provides a Desktop interface named Git GitHub provides a Desktop interface named
11.
Gui. GitHub Desktop.