DevOps

Unit 3
Introduction to project management
The need for source code control:
Source code control (also known as version control) is an essential part of DevOps practices. Here
are a few reasons why:
Collaboration: Source code control allows multiple team members to work on the same codebase
simultaneously and track each other's changes.

Traceability: Source code control systems provide a complete history of changes to the code,
enabling teams to trace bugs, understand why specific changes were made, and roll back to
previous versions if necessary.
Branching and merging: Teams can create separate branches for different features or bug fixes,
then merge the changes back into the main codebase. This helps to ensure that different parts of
the code can be developed independently, without interfering with each other.
Continuous integration and delivery: Source code control systems are integral to continuous
integration and delivery (CI/CD) pipelines, where changes to the code are automatically built,
tested, and deployed to production.

In summary, source code control is a critical component of DevOps practices, as it enables teams
to collaborate, manage changes to code, and automate the delivery of software.

History of source code management

The history of source code management (SCM) in DevOps dates back to the early days of software
development. Early SCM systems were simple and focused on tracking changes to source code
over time.

In the late 1990s and early 2000s, the open-source movement and the rise of the internet led to a
proliferation of new SCM tools, including CVS (Concurrent Versions System), Subversion, and
Git. These systems made it easier for developers to collaborate on projects, manage multiple
versions of code, and automate the build, test, and deployment process.
As DevOps emerged as a software development methodology in the mid-2000s, SCM became an
integral part of the DevOps toolchain. DevOps teams adopted Git as their SCM tool of choice,
DevOps

leveraging its distributed nature, branch and merge capabilities, and integration with CI/CD
pipelines.

Today, Git is the most widely used SCM system in the world, and is a critical component of
DevOps practices. With the rise of cloud-based platforms, modern SCM systems also offer features
like collaboration, code reviews, and integrated issue tracking.

Roles and code in Devops

In DevOps, roles and code play a critical role in the development, delivery, and operation of
software.

Roles:
● Development team: responsible for writing and testing code.
● Operations team: responsible for the deployment and maintenance of the code in
production.
● DevOps team: responsible for bridging the gap between development and operations,
ensuring that code is delivered quickly and reliably to production.

Code:
● Code is the backbone of DevOps and represents the software that is being developed,
tested, deployed, and maintained.
● Code is managed using source code control systems like Git, which provide a way to track
changes to the code over time, collaborate on the code with other team members, and
automate the build, test, and deployment process.
● Code is continuously integrated and tested, ensuring that any changes to the code do not
cause unintended consequences in the production environment.
In conclusion, both roles and code play a critical role in DevOps. Teams work together to ensure
that code is developed, tested, and delivered quickly and reliably to production, while operations
teams maintain the code in production and respond to any issues that arise.
Overall, SCM has been an important part of the evolution of DevOps, enabling teams to
collaborate, manage code changes, and automate the software delivery process.

Source code management system and migrations

● A source code management (SCM) system is a software application that provides version
control for source code. It tracks changes made to the code over time, enabling teams to
revert to previous versions if necessary, and helps ensure that code can be collaborated on
by multiple team members.
DevOps

● SCM systems typically provide features such as version tracking, branching and merging,
change history, and rollback capabilities. Some popular SCM systems include Git,
Subversion, Mercurial, and Microsoft Team Foundation Server.
● Source code management (SCM) systems are often used to manage code migrations, which
are the process of moving code from one environment to another. This is typically done as
part of a software development project, where code is moved from a development
environment to a testing environment and finally to a production environment.
SCM systems provide a number of benefits for managing code migrations, including:

1. Version control
2. Branching and merging
3. Rollback
4. Collaboration
5. Automation
1) Version control: SCM systems keep a record of all changes to the code, enabling teams to track
the code as it moves through different environments.

Purpose of Version Control:

● Multiple people can work simultaneously on a single project. Everyone works on and edits
their own copy of the files and it is up to them when they wish to share the changes made by
them with the rest of the team.
● It also enables one person to use multiple computers to work on a project, so it is valuable even
if you are working by yourself.
● It integrates the work that is done simultaneously by different members of the team. In some
rare cases, when conflicting edits are made by two people to the same line of a file, then human
assistance is requested by the version control system in deciding what should be done.
● Version control provides access to the historical versions of a project. This is insurance against
computer crashes or data loss. If any mistake is made, you can easily roll back to a previous
version. It is also possible to undo specific edits that too without losing the work done in the
meanwhile. It can be easily known when, why, and by whom any part of a file was edited.

Benefits of the version control system:

● Enhances the project development speed by providing efficient collaboration,

● Leverages the productivity, expedites product delivery, and skills of the employees through
better communication and assistance,
● Reduce possibilities of errors and conflicts meanwhile project development through
traceability to every small change,
● Employees or contributors of the project can contribute from anywhere irrespective of the
different geographical locations through this VCS,
DevOps

● For each different contributor to the project, a different working copy is maintained and not
merged to the main file unless the working copy is validated. The most popular example is Git,
Helix core, Microsoft TFS,
● Helps in recovery in case of any disaster or contingent situation,
● Informs us about Who, What, When, Why changes have been made.

Types of Version Control Systems:

● Local Version Control Systems

● Centralized Version Control Systems
● Distributed Version Control Systems

Local Version Control Systems: It is one of the simplest forms and has a database that kept all
the changes to files under revision control. RCS is one of the most common VCS tools. It keeps
patch sets (differences between files) in a special format on disk. By adding up all the patches it
can then re-create what any file looked like at any point in time.
Centralized Version Control Systems: Centralized version control systems contain just one
repository globally and every user need to commit for reflecting one’s changes in the repository.
It is possible for others to see your changes by updating.
Two things are required to make your changes visible to others which are:
● You commit
● They update
DevOps

The benefit of CVCS (Centralized Version Control Systems) makes collaboration amongst
developers along with providing an insight to a certain extent on what everyone else is doing on
the project. It allows administrators to fine-grained control over who can do what.
It has some downsides as well which led to the development of DVS. The most obvious is the
single point of failure that the centralized repository represents if it goes down during that period
collaboration and saving versioned changes is not possible. What if the hard disk of the central
database becomes corrupted, and proper backups haven’t been kept? You lose absolutely
everything.
Distributed Version Control Systems:
Distributed version control systems contain multiple repositories. Each user has their own
repository and working copy. Just committing your changes will not give others access to your
changes. This is because commit will reflect those changes in your local repository and you need
to push them in order to make them visible on the central repository. Similarly, When you update,
you do not get others’ changes unless you have first pulled those changes into your repository.
To make your changes visible to others, 4 things are required:

● You commit
● You push
● They pull
● They update
The most popular distributed version control systems are Git, and Mercurial. They help us
overcome the problem of single point of failure.
DevOps

2)Branching and merging: Teams can create separate branches of code for different
environments, making it easier to manage the migration process.
Branching and merging are key concepts in Git-based version control systems, and are widely used
in DevOps to manage the development of software.
Branching in Git allows developers to create a separate line of development for a new feature or
bug fix. This allows developers to make changes to the code without affecting the main branch,
and to collaborate with others on the same feature or bug fix.

Merging in Git is the process of integrating changes made in one branch into another branch. In
DevOps, merging is often used to integrate changes made in a feature branch into the main branch,
incorporating the changes into the codebase.

Branching and merging provide several benefits in DevOps:

Improved collaboration: By allowing multiple developers to work on the same codebase at the
same time, branching and merging facilitate collaboration and coordination among team members.

Improved code quality: By isolating changes made in a feature branch, branching and merging
make it easier to thoroughly review and test changes before they are integrated into the main
codebase, reducing the risk of introducing bugs or other issues.
Increased transparency: By tracking all changes made to the codebase, branching and merging
provide a clear audit trail of how code has evolved over time.

Overall, branching and merging are essential tools in the DevOps toolkit, helping to improve
collaboration, code quality, and transparency in the software development process.
Rollback: In the event of a problem during a migration, teams can quickly revert to a previous
version of the code.
Rollback in DevOps refers to the process of reverting a change or returning to a previous version
of a system, application, or infrastructure component. Rollback is an important capability in
DevOps, as it provides a way to quickly and efficiently revert changes that have unintended
consequences or cause problems in production.

There are several approaches to rollback in DevOps, including:

Version control: By using a version control system, such as Git, DevOps teams can revert to a
previous version of the code by checking out an earlier commit.

Infrastructure as code: By using infrastructure as code tools, such as Terraform or Ansible,

DevOps teams can roll back changes to their infrastructure by re-applying an earlier version of the
code.
DevOps

Continuous delivery pipelines: DevOps teams can use continuous delivery pipelines to automate
the rollback process, by automatically reverting changes to a previous version of the code or
infrastructure if tests fail or other problems are detected.

Snapshots: DevOps teams can use snapshots to quickly restore an earlier version of a system or
infrastructure component.
Overall, rollback is an important capability in DevOps, providing a way to quickly revert changes
that have unintended consequences or cause problems in production. By using a combination of
version control, infrastructure as code, continuous delivery pipelines, and snapshots, DevOps
teams can ensure that their systems and applications can be quickly and easily rolled back to a
previous version if needed.

Collaboration: SCM systems enable teams to collaborate on code migrations, with team
members working on different aspects of the migration process simultaneously.

Collaboration is a key aspect of DevOps, as it helps to bring together development, operations,

and other teams to work together towards a common goal of delivering high-quality software
quickly and efficiently.

In DevOps, collaboration is facilitated by a range of tools and practices, including:

Version control systems: By using a version control system, such as Git, teams can collaborate
on code development, track changes to source code, and merge code changes from multiple
contributors.

Continuous integration and continuous deployment (CI/CD): By automating the build, test,
and deployment of code, CI/CD pipelines help to streamline the development process and reduce
the risk of introducing bugs or other issues into the codebase.
Code review: By using code review tools, such as pull requests, teams can collaborate on code
development, share feedback, and ensure that changes are thoroughly reviewed and tested before
they are integrated into the codebase.
Issue tracking: By using issue tracking tools, such as JIRA or GitHub Issues, teams can
collaborate on resolving bugs, tracking progress, and managing the development of new features.

Communication tools: By using communication tools, such as Slack or Microsoft Teams, teams
can collaborate and coordinate their work, share information, and resolve problems quickly and
efficiently.
Overall, collaboration is a critical component of DevOps, helping teams to work together
effectively and efficiently to deliver high-quality software. By using a range of tools and practices
to facilitate collaboration, DevOps teams can improve the transparency, speed, and quality of their
software development processes.
DevOps

Automation: Many SCM systems integrate with continuous integration and delivery (CI/CD)
pipelines, enabling teams to automate the migration process.
In conclusion, SCM systems play a critical role in managing code migrations. They provide a way
to track code changes, collaborate on migrations, and automate the migration process, enabling
teams to deliver code quickly and reliably to production.

Shared authentication
Shared authentication in DevOps refers to the practice of using a common identity management system to
control access to the various tools, resources, and systems used in software development and operations.
This helps to simplify the process of managing users and permissions and ensures that everyone has the
necessary access to perform their jobs. Examples of shared authentication systems include Active Directory,
LDAP, and SAML-based identity providers.

Hosted Git servers

Hosted Git servers are online platforms that provide Git repository hosting services for software
development teams. They are widely used in DevOps to centralize version control of source code, track
changes, and collaborate on code development. Some popular hosted Git servers include GitHub, GitLab,
and Bitbucket. These platforms offer features such as pull requests, code reviews, issue tracking, and
continuous integration/continuous deployment (CI/CD) pipelines. By using a hosted Git server, DevOps
teams can streamline their development processes and collaborate more efficiently on code projects.

Different Git server implementations

There are several different Git server implementations that organizations can use to host their Git
repositories. Some of the most popular include:
GitHub: One of the largest Git repository hosting services, GitHub is widely used by developers
for version control, collaboration, and code sharing.

GitLab: An open-source Git repository management platform that provides version control, issue
tracking, code review, and more.
Bitbucket: A web-based Git repository hosting service that provides version control, issue
tracking, and project management tools.

Gitea: An open-source Git server that is designed to be lightweight, fast, and easy to use.
Gogs: Another open-source Git server, Gogs is designed for small teams and organizations and
provides a simple, user-friendly interface.

GitBucket: A Git server written in Scala that provides a wide range of features, including issue
tracking, pull requests, and code reviews.
DevOps

Organizations can choose the Git server implementation that best fits their needs, taking into
account factors such as cost, scalability, and security requirements.

Docker intermission
Docker is an open-source project with a friendly-whale logo that facilitates the deployment of
applications in software containers. It is a set of PaaS products that deliver containers (software
packages) using OS-level virtualization. It embodies resource isolation features of the Linux kernel
but offers a friendly API.

In simple words, Docker is a tool or platform design to simplify the process of creating, deploying,
and packaging and shipping out applications along with its parts such as libraries and other
dependencies. Its primary purpose is to automate the application deployment process and
operating-system-level virtualization on Linux. It allows multiple containers to run on the same
hardware and provides high productivity, along with maintaining isolated applications and
facilitating seamless configuration.

Docker benefits include:

● High ROI and cost savings
● Productivity and standardization
● Maintenance and compatibility
● Rapid deployment
● Faster configurations
● Seamless portability
● Continuous testing and deployment
● Isolation, segregation, and security
Docker vs. Virtual Machines

Virtual Machine is an application environment that imitates dedicated hardware by providing an

emulation of the computer system. Docker and Vmboth have their set of benefits and uses, but
when it comes to running applications in multiple environments, both can be utilized. So which
one wins? Let's get into a quick Docker vs. VM comparison.

OS Support: VM requires a lot of memory when installed in an OS, whereas Docker containers
occupy less space.

Performance: Running several VMs can affect the performance, whereas, Docker containers are
stored in a single Docker engine; thus, they provide better performance.
DevOps

Boot-up time: VMs have a longer booting time compared to Docker.

Efficiency: VMs have lower efficiency than Docker.

Scaling: VMs are difficult to scale up, whereas Docker is easy to scale up.

Space allocation: You cannot share data volumes with VMs, but you can share and reuse them
among various Docker containers.

Portability: With VMs, you can face compatibility issues while porting across different platforms;
Docker is easily portable.
Clearly, Docker is a hands-down winner.

Gerrit
Gerrit is a web based code review tool which is integrated with Git and built on top of Git version
control system (helps developers to work together and maintain the history of their work). It allows
to merge changes to Git repository when you are done with the code reviews.
Gerrit was developed by Shawn Pearce at Google which is written in Java, Servlet, GWT(Google
Web Toolkit). The stable release of Gerrit is 2.12.2 and published on March 11, 2016 licensed
under Apache License v2.
Why Use Gerrit?
Following are certain reasons, why you should use Gerrit.
● You can easily find the error in the source code using Gerrit.
● You can work with Gerrit, if you have regular Git client; no need to install any Gerrit client.
● Gerrit can be used as an intermediate between developers and git repositories.
Features of Gerrit
● Gerrit is a free and an open source Git version control system.
● The user interface of Gerrit is formed on Google Web Toolkit.
● It is a lightweight framework for reviewing every commit.
● Gerrit acts as a repository, which allows pushing the code and creates the review for your
commit.
Advantages of Gerrit
● Gerrit provides access control for Git repositories and web frontend for code review.
● You can push the code without using additional command line tools.
● Gerrit can allow or decline the permission on the repository level and down to the branch
level.
● Gerrit is supported by Eclipse.
Disadvantages of Gerrit
● Reviewing, verifying and resubmitting the code commits slows down the time to market.
● Gerrit can work only with Git.
DevOps

● Gerrit is slow and it's not possible to change the sort order in which changes are listed.
● You need administrator rights to add repository on Gerrit.

What is Gerrit?

Gerrit is an exceptionally extensible and configurable apparatus for online code survey and
storehouse the executives for projects utilizing the Git rendition control framework. Gerrit is
similarly helpful where all clients are believed committers, for example, might be the situation
with shut source business advancement.

It is used to store the merged code base and the changes under review that have not being merged
yet. Gerrit has the limitation of a single repository per project.

Gerrit is first and foremost an arranging region where changes can be looked at prior to turning
into a piece of the code base. It is likewise an empowering agent for this survey cycle, catching
notes and remarks about the progressions to empower conversation of the change. This is
especially valuable with conveyed groups where this discussion can’t occur eye to eye.
 DevOps

How Gerrit Works Architecture?

Use case of Gerrit

● Knowledge exchange:
o The code review process allows newcomers to see the code of other more experienced
developers.
o Developers can get feedback on their suggested changes.
o Experienced developers can help to evaluate the impact on the whole code.
o Shared code ownership: by reviewing code of other developers the whole team gets a solid
knowledge of the complete code base.

The pull request model

Pull request is a feature of Git-based version control systems that allows developers to propose
changes to a Git repository and request feedback or approval f rom other team members. It is
widely used in DevOps to facilitate collaboration and code review in the software development
process.
DevOps

In the pull request model, a developer creates a new branch in a Git repository, makes changes to
the code, and then opens a pull request to merge the changes into the main branch. Other team
members can then review the changes, provide feedback, and approve or reject the request.

Pull Requests are a mechanism popularized by github, used to help facilitate merging of work,
particularly in the context of open-source projects. A contributor works on their contribution in a
fork (clone) of the central repository. Once their contribution is finished they create a pull
request to notify the owner of the central repository that their work is ready to be merged into the
mainline. Tooling supports and encourages code review of the contribution before accepting the
request. Pull requests have become widely used in software development, but critics are
concerned by the addition of integration friction which can prevent continuous integration.

Pull requests essentially provide convenient tooling for a development workflow that existed in
many open-source projects, particularly those using a distributed source-control system (such as
git). This workflow begins with a contributor creating a new logical branch, either by starting a
new branch in the central repository, cloning into a personal repository, or both. The contributor
then works on that branch, typically in the style of a Feature Branch, pulling any updates
from Mainline into their branch. When they are done they communicate with the maintainer of
the central repository indicating that they are done, together with a reference to their commits.
This reference could be the URL of a branch that needs to be integrated, or a set of patches in an
email.

Once the maintainer gets the message, she can then examine the commits to decide if they are
ready to go into mainline. If not, she can then suggest changes to the contributor, who then has
opportunity to adjust their submission. Once all is ok, the maintainer can then merge, either with
a regular merge/rebase or applying the patches from the final email.

Github's pull request mechanism makes this flow much easier. It keeps track of the clones
through its fork mechanism, and automatically creates a message thread to discuss the pull
request, together with behavior to handle the various steps in the review workflow. These
conveniences were a major part of what made github successful and led to "pull request"
becoming a fundamental part of the developer's lexicon.

So that's how pull requests work, but should we use them, and if so how? To answer that
question, I like to step back from the mechanism and think about how it works in the context of a
source code management workflow. To help me think about that, I wrote down a series
of patterns for managing source code branching. I find understanding these (specifically the Base
and Integration patterns) clarifies the role of pull requests.
In terms of these patterns, pull requests are a mechanism designed to implement a combination
of Feature Branching and Pre-Integration Reviews. Thus to assess the usefulness of pull requests
we first need to consider how applicable those patterns are to our situation. Like most patterns,
DevOps

they are sometimes valuable, and sometimes a pain in the neck - we have to examine them based
on our specific context. Feature Branching is a good way of packaging together a logical
contribution so that it can be assessed, accepted, or deferred as a single unit. This makes a lot of
sense when contributors are not trusted to commit directly to mainline. But Feature Branching
comes at a cost, which is that it usually limits the frequency of integration, leading to
complicated merges and deterring refactoring. Pre-Integration Reviews provide a clear place to
do code review at the cost of a significant increase in integration friction. [1]

That's a drastic summary of the situation (I need a lot more words to explain this further in the
feature branching article), but it boils down to the fact that the value of these patterns, and thus
the value of pull requests, rest mostly on the social structure of the team. Some teams work better
with pull requests, some teams would find pull requests a severe drag on the effectiveness. I
suspect that since pull requests are so popular, a lot of teams are using them by default when they
would do better without them.

While pull requests are built for Feature Branches, teams can use them within a Continuous
Integration environment. To do this they need to ensure that pull requests are small enough, and
the team responsive enough, to follow the CI rule of thumb that everybody does Mainline
Integration at least daily. (And I should remind everyone that Mainline Integration is more than
just merging the current mainline into the feature branch). Using the ship/show/ask classification
can be an effective way to integrate pull requests into a more CI-friendly workflow.
The wide usage of pull requests has encouraged a wider use of code review, since pull requests
provide a clear point for Pre-Integration Review, together with tooling that encourages it. Code
review is a Good Thing, but we must remember that a pull request isn't the only mechanism we
can use for it. Many teams find great value in the continuous review afforded by Pair
Programming. To avoid reducing integration frquency we can carry out post-integration code
review in several ways. A formal process can record a review for each commit, or a tech lead can
examine risky commits every couple of days. Perhaps the most powerful form of code review is
one that's frequently ignored. A team that takes the attitude that the codebase is a fluid system,
one that can be steadily refined with repeated iteration carries out Refinement Code
Review every time a developer looks at existing code. I often hear people say that pull requests
are necessary because without them you can't do code reviews - that's rubbish. Pre-integration
code review is just one way to do code reviews, and for many teams it isn't the best choice.

The pull request model provides several benefits in DevOps:

Improved code quality: Pull requests encourage collaboration and code review, helping to catch
potential bugs and issues before they make it into the main codebase.
Increased transparency: Pull requests provide a clear audit trail of all changes made to the code,
making it easier to understand how code has evolved over time.
DevOps

Better collaboration: Pull requests allow developers to share their work and get feedback from
others, improving collaboration and communication within the development team.

Overall, the pull request model is an important tool in the DevOps toolkit, helping to improve the
quality, transparency, and collaboration of software development processes.

GitLab
GitLab is an open-source Git repository management platform that provides a wide range of
features for software development teams. It is commonly used in DevOps for version control, issue
tracking, code review, and continuous integration/continuous deployment (CI/CD) pipelines.
GitLab provides a centralized platform for teams to manage their Git repositories, track changes
to source code, and collaborate on code development. It offers a range of tools to support code
review and collaboration, including pull requests, code comments, and merge request approvals.
In addition, GitLab provides a CI/CD pipeline tool that allows teams to automate the process of
building, testing, and deploying code. This helps to streamline the development process and reduce
the risk of introducing bugs or other issues into the codebase.

Overall, GitLab is a comprehensive Git repository management platform that provides a wide
range of tools and features for software development teams. By using GitLab, DevOps teams can
improve the efficiency, transparency, and collaboration of their software development processes.

What is Git?

Git is a distributed version control system, which means that a local clone of the project is a
complete version control repository. These fully functional local repositories make it easy to work
offline or remotely. Developers commit their work locally, and then sync their copy of the
repository with the copy on the server. This paradigm differs from centralized version control
where clients must synchronize code with a server before creating new versions of code.

Git's flexibility and popularity make it a great choice for any team. Many developers and college
graduates already know how to use Git. Git's user community has created resources to train
developers and Git's popularity make it easy to get help when needed. Nearly every development
environment has Git support and Git command line tools implemented on every major operating
system.

Git basics

Every time work is saved, Git creates a commit. A commit is a snapshot of all files at a point in
time. If a file hasn't changed from one commit to the next, Git uses the previously stored file. This
DevOps

design differs from other systems that store an initial version of a file and keep a record of
deltas over time.

Commits create links to other commits, forming a graph of the development history. It's possible
to revert code to a previous commit, inspect how files changed from one commit to the next, and
review information such as where and when changes were made. Commits are identified in Git by
a unique cryptographic hash of the contents of the commit. Because everything is hashed, it's
impossible to make changes, lose information, or corrupt files without Git detecting it.

Branches

Each developer saves changes to their own local code repository. As a result, there can be many
different changes based off the same commit. Git provides tools for isolating changes and later
merging them back together. Branches, which are lightweight pointers to work in progress, manage
this separation. Once work created in a branch is finished, it can be merged back into the team's
main (or trunk) branch.

Files and commits

Files in Git are in one of three states: modified, staged, or committed. When a file is first modified,
the changes exist only in the working directory. They aren't yet part of a commit or the
development history. The developer must stage the changed files to be included in the commit.
The staging area contains all changes to include in the next commit. Once the developer is happy
DevOps

with the staged files, the files are packaged as a commit with a message describing what changed.
This commit becomes part of the development history.

Staging lets developers pick which file changes to save in a commit in order to break down large
changes into a series of smaller commits. By reducing the scope of commits, it's easier to review
the commit history to find specific file changes.

Benefits of Git

The benefits of Git are many.

Simultaneous development

Everyone has their own local copy of code and can work simultaneously on their own branches.
Git works offline since almost every operation is local.

Faster releases

Branches allow for flexible and simultaneous development. The main branch contains stable, high-
quality code from which you release. Feature branches contain work in progress, which are merged
into the main branch upon completion. By separating the release branch from development in
progress, it's easier to manage stable code and ship updates more quickly.

Built-in integration

Due to its popularity, Git integrates into most tools and products. Every major IDE has built-in Git
support, and many tools support continuous integration, continuous deployment, automated
testing, work item tracking, metrics, and reporting feature integration with Git. This integration
simplifies the day-to-day workflow.
DevOps

Strong community support

Git is open-source and has become the de facto standard for version control. There is no shortage
of tools and resources available for teams to leverage. The volume of community support for Git
compared to other version control systems makes it easy to get help when needed.

Git works with any team

Using Git with a source code management tool increases a team's productivity by encouraging
collaboration, enforcing policies, automating processes, and improving visibility and traceability
of work. The team can settle on individual tools for version control, work item tracking, and
continuous integration and deployment. Or, they can choose a solution like GitHub or Azure
DevOps that supports all of these tasks in one place.

Pull requests

Use pull requests to discuss code changes with the team before merging them into the main branch.
The discussions in pull requests are invaluable to ensuring code quality and increase knowledge
across your team. Platforms like GitHub and Azure DevOps offer a rich pull request experience
where developers can browse file changes, leave comments, inspect commits, view builds, and
vote to approve the code.

Branch policies

Teams can configure GitHub and Azure DevOps to enforce consistent workflows and process
across the team. They can set up branch policies to ensure that pull requests meet requirements
before completion. Branch policies protect important branches by preventing direct pushes,
requiring reviewers, and ensuring clean builds.
DevOps

Unit 4
Integrating the system
Build systems
A build system is a key component in DevOps, and it plays an important role in the software
development and delivery process. It automates the process of compiling and packaging source
code into a deployable artifact, allowing for efficient and consistent builds.
Here are some of the key functions performed by a build system:

Compilation: The build system compiles the source code into a machine-executable format, such
as a binary or an executable jar file.
Dependency Management: The build system ensures that all required dependencies are available
and properly integrated into the build artifact. This can include external libraries, components, and
other resources needed to run the application.
Testing: The build system runs automated tests to ensure that the code is functioning as intended,
and to catch any issues early in the development process.

Packaging: The build system packages the compiled code and its dependencies into a single,
deployable artifact, such as a Docker image or a tar archive.
Version Control: The build system integrates with version control systems, such as Git, to track
changes to the code and manage releases.

Continuous Integration: The build system can be configured to run builds automatically
whenever changes are made to the code, allowing for fast feedback and continuous integration of
new code into the main branch.

Deployment: The build system can be integrated with deployment tools and processes to automate
the deployment of the build artifact to production environments.
In DevOps, it's important to have a build system that is fast, reliable, and scalable, and that can
integrate with other tools and processes in the software development and delivery pipeline. There
are many build systems available, each with its own set of features and capabilities, and choosing
the right one will depend on the specific needs of the project and team.
DevOps

Jenkins build server

What is Jenkin?

Jenkins is an open source automation tool written in Java programming language that allows
continuous integration.

Jenkins builds and tests our software projects which continuously making it easier for developers
to integrate changes to the project, and making it easier for users to obtain a fresh build.

It also allows us to continuously deliver our software by integrating with a large number of testing
and deployment technologies.

Jenkins offers a straightforward way to set up a continuous integration or continuous delivery

environment for almost any combination of languages and source code repositories using
pipelines, as well as automating other routine development tasks.

With the help of Jenkins, organizations can speed up the software development process through
automation. Jenkins adds development life-cycle processes of all kinds, including build, document,
test, package, stage, deploy static analysis and much more.

Jenkins achieves CI (Continuous Integration) with the help of plugins. Plugins is used to allow the
integration of various DevOps stages. If you want to integrate a particular tool, you have to install
the plugins for that tool. For example: Maven 2 Project, Git, HTML Publisher, Amazon EC2, etc.

For example: If any organization is developing a project, then Jenkins will continuously test your
project builds and show you the errors in early stages of your development.

Possible steps executed by Jenkins are for example:

o Perform a software build using a build system like Gradle or Maven Apache
o Execute a shell script
o Archive a build result
o Running software tests
DevOps

Jenkin workflow
DevOps

Jenkins Master-Slave Architecture

As you can see in the diagram provided above, on the left is the Remote source code repository.
The Jenkins server accesses the master environment on the left side and the master environment
can push down to multiple other Jenkins Slave environments to distribute the workload.

That lets you run multiple builds, tests, and product environment across the entire architecture.
Jenkins Slaves can be running different build versions of the code for different operating systems
and the server Master controls how each of the builds operates.

Supported on a master-slave architecture, Jenkins comprises many slaves working for a master.
This architecture - the Jenkins Distributed Build - can run identical test cases in different
environments. Results are collected and combined on the master node for monitoring.

Jenkins Applications

Jenkins helps to automate and accelerate the software development process. Here are some of the
most common applications of Jenkins:
DevOps

1. Increased Code Coverage

Code coverage is determined by the number of lines of code a component has and how many of
them get executed. Jenkins increases code coverage which ultimately promotes a transparent
development process among the team members.

2. No Broken Code

Jenkins ensures that the code is good and tested well through continuous integration. The final
code is merged only when all the tests are successful. This makes sure that no broken code is
shipped into production.

What are the Jenkins Features?

Jenkins offers many attractive features for developers:

● Easy Installation

Jenkins is a platform-agnostic, self-contained Java-based program, ready to run with packages for
Windows, Mac OS, and Unix-like operating systems.

● Easy Configuration

Jenkins is easily set up and configured using its web interface, featuring error checks and a built-in help
function.

● Available Plugins

There are hundreds of plugins available in the Update Center, integrating with every tool in the CI and
CD toolchain.

● Extensible

Jenkins can be extended by means of its plugin architecture, providing nearly endless possibilities for
what it can do.
DevOps

● Easy Distribution

Jenkins can easily distribute work across multiple machines for faster builds, tests, and deployments
across multiple platforms.

● Free Open Source

Jenkins is an open-source resource backed by heavy community support.

As a part of our learning about what is Jenkins, let us next learn about the Jenkins architecture.

Jenkins build server

Jenkins is a popular open-source automation server that helps developers automate parts of the software
development process. A Jenkins build server is responsible for building, testing, and deploying software
projects.

A Jenkins build server is typically set up on a dedicated machine or a virtual machine, and is used to manage
the continuous integration and continuous delivery (CI/CD) pipeline for a software project. The build server
is configured with all the necessary tools, dependencies, and plugins to build, test, and deploy the project.

The build process in Jenkins typically starts with code being committed to a version control system (such
as Git), which triggers a build on the Jenkins server. The Jenkins server then checks out the code, builds it,
runs tests on it, and if everything is successful, deploys the code to a staging or production environment.

Jenkins has a large community of developers who have created hundreds of plugins that extend its
functionality, so it's easy to find plugins to support specific tools, technologies, and workflows. For
example, there are plugins for integrating with cloud infrastructure, running security scans, deploying to
various platforms, and more.

Overall, a Jenkins build server can greatly improve the efficiency and reliability of the software
development process by automating repetitive tasks, reducing the risk of manual errors, and enabling
developers to focus on writing code.

Managing build dependencies

Managing build dependencies is an important aspect of continuous integration and continuous
delivery (CI/CD) pipelines. In software development, dependencies refer to external libraries,
tools, or resources that a project relies on to build, test, and deploy. Proper management of
dependencies can ensure that builds are repeatable and that the build environment is consistent and
up-to-date.
Here are some common practices for managing build dependencies in Jenkins:
DevOps

Dependency Management Tools: Utilize tools such as Maven, Gradle, or npm to manage
dependencies and automate the process of downloading and installing required dependencies for
a build.

Version Pinning: Specify exact versions of dependencies to ensure builds are consistent and
repeatable.
Caching: Cache dependencies locally on the build server to improve build performance and reduce
the time it takes to download dependencies.
Continuous Monitoring: Regularly check for updates and security vulnerabilities in dependencies
to ensure the build environment is secure and up-to-date.

Automated Testing: Automated testing can catch issues related to dependencies early in the
development process.
By following these practices, you can effectively manage build dependencies and maintain the
reliability and consistency of your CI/CD pipeline.

Jenkins plugins
Jenkins plugins are packages of software that extend the functionality of the Jenkins automation
server. Plugins allow you to integrate Jenkins with various tools, technologies, and workflows, and
can be easily installed and configured through the Jenkins web interface.

Some popular Jenkins plugins include:

Git Plugin: This plugin integrates Jenkins with Git version control system, allowing you to pull
code changes, build and test them, and deploy the code to production.

Maven Plugin: This plugin integrates Jenkins with Apache Maven, a build automation tool
commonly used in Java projects.
Amazon Web Services (AWS) Plugin: This plugin allows you to integrate Jenkins with Amazon
Web Services (AWS), making it easier to run builds, tests, and deployments on AWS
infrastructure.
Slack Plugin: This plugin integrates Jenkins with Slack, allowing you to receive notifications
about build status, failures, and other important events in your Slack channels.

Blue Ocean Plugin: This plugin provides a new and modern user interface for Jenkins, making it
easier to use and navigate.
Pipeline Plugin: This plugin provides a simple way to define and manage complex CI/CD
pipelines in Jenkins.
DevOps

Jenkins plugins are easy to install and can be managed through the Jenkins web interface. There
are hundreds of plugins available, covering a wide range of tools, technologies, and use cases, so
you can easily find the plugins that best meet your needs.

By using plugins, you can greatly improve the efficiency and automation of your software
development process, and make it easier to integrate Jenkins with the tools and workflows you
use.

Git Plugin
The Git Plugin is a popular plugin for Jenkins that integrates the Jenkins automation server with
the Git version control system. This plugin allows you to pull code changes from a Git repository,
build and test the code, and deploy it to production.

With the Git Plugin, you can configure Jenkins to automatically build and test your code whenever
changes are pushed to the Git repository. You can also configure it to build and test code on a
schedule, such as once a day or once a week.
The Git Plugin provides a number of features for managing code changes, including:

Branch and Tag builds: You can configure Jenkins to build specific branches or tags from your
Git repository.
Pull Requests: You can configure Jenkins to build and test pull requests from your Git repository,
allowing you to validate code changes before merging them into the main branch.
Build Triggers: You can configure Jenkins to build and test code changes whenever changes are
pushed to the Git repository or on a schedule.

Code Quality Metrics: The Git Plugin integrates with tools such as SonarQube to provide code
quality metrics, allowing you to track and improve the quality of your code over time.
Notification and Reporting: The Git Plugin provides notifications and reports on build status,
failures, and other important events. You can configure Jenkins to send notifications via email,
Slack, or other communication channels.

By using the Git Plugin, you can streamline your software development process and make it easier
to manage code changes and collaborate with other developers on your team.

file system layout

In DevOps, the file system layout refers to the organization and structure of files and directories
on the systems and servers used for software development and deployment. A well-designed file
system layout is critical for efficient and reliable operations in a DevOps environment.

Here are some common elements of a file system layout in DevOps:

DevOps

Code Repository: A central code repository, such as Git, is used to store and manage source code,
configuration files, and other artifacts.

Build Artifacts: Build artifacts, such as compiled code, are stored in a designated directory for
easy access and management.
Dependencies: Directories for storing dependencies, such as libraries and tools, are designated for
easy management and version control.

Configuration Files: Configuration files, such as YAML or JSON files, are stored in a designated
directory for easy access and management.
Log Files: Log files generated by applications, builds, and deployments are stored in a designated
directory for easy access and management.
Backup and Recovery: Directories for storing backups and recovery data are designated for easy
management and to ensure business continuity.

Environment-specific Directories: Directories are designated for each environment, such as

development, test, and production, to ensure that the correct configuration files and artifacts are
used for each environment.
By following a well-designed file system layout in a DevOps environment, you can improve the
efficiency, reliability, and security of your software development and deployment processes.

The host server

In Jenkins, a host server refers to the physical or virtual machine that runs the Jenkins automation
server. The host server is responsible for running the Jenkins process and providing resources,
such as memory, storage, and CPU, for executing builds and other tasks.
The host server can be either a standalone machine or part of a network or cloud-based
infrastructure. When running Jenkins on a standalone machine, the host server is responsible for
all aspects of the Jenkins installation, including setup, configuration, and maintenance.

When running Jenkins on a network or cloud-based infrastructure, the host server is responsible
for providing resources for the Jenkins process, but the setup, configuration, and maintenance may
be managed by other components of the infrastructure.
By providing the necessary resources and ensuring the stability and reliability of the host server,
you can ensure the efficient operation of Jenkins and the success of your software development
and deployment processes.
To host a server in Jenkins, you'll need to follow these steps:
DevOps

Install Jenkins: You can install Jenkins on a server by downloading the Jenkins WAR file,
deploying it to a servlet container such as Apache Tomcat, and starting the server.

Configure Jenkins: Once Jenkins is up and running, you can access its web interface to configure
and manage the build environment. You can install plugins, set up security, and configure build
jobs.
Create a Build Job: To build your project, you'll need to create a build job in Jenkins. This will
define the steps involved in building your project, such as checking out the code from version
control, compiling the code, running tests, and packaging the application.
Schedule Builds: You can configure your build job to run automatically at a specific time or when
certain conditions are met. You can also trigger builds manually from the web interface.

Monitor Builds: Jenkins provides a variety of tools for monitoring builds, such as build history,
build console output, and build artifacts. You can use these tools to keep track of the status of your
builds and to diagnose problems when they occur.

Build slaves
Jenkins Master-Slave Architecture

As you can see in the diagram provided above, on the left is the Remote source code repository.
The Jenkins server accesses the master environment on the left side and the master environment
can push down to multiple other Jenkins Slave environments to distribute the workload.
DevOps

That lets you run multiple builds, tests, and product environment across the entire architecture.
Jenkins Slaves can be running different build versions of the code for different operating systems
and the server Master controls how each of the builds operates.

Supported on a master-slave architecture, Jenkins comprises many slaves working for a master.
This architecture - the Jenkins Distributed Build - can run identical test cases in different
environments. Results are collected and combined on the master node for monitoring.

The standard Jenkins installation includes Jenkins master, and in this setup, the master will be
managing all our build system's tasks. If we're working on a number of projects, we can run
numerous jobs on each one. Some projects require the use of specific nodes, which necessitates
the use of slave nodes.
The Jenkins master is in charge of scheduling jobs, assigning slave nodes, and sending builds
to slave nodes for execution. It will also keep track of the slave node state (offline or online),
retrieve build results from slave nodes, and display them on the terminal output. In most
installations, multiple slave nodes will be assigned to the task of building jobs.

Before we get started, let's double-check that we have all of the prerequisites in place for
adding a slave node:

● Jenkins Server is up and running and ready to use

● Another server for a slave node configuration
● The Jenkins server and the slave server are both connected to the same network

To configure the Master server, we'll log in to the Jenkins server and follow the steps below.
First, we'll go to “Manage Jenkins -> Manage Nodes -> New Node” to create a new node:
DevOps

On the next screen, we enter the “Node Name” (slaveNode1), select “Permanent Agent”, then
click “OK”:

After clicking “OK”, we'll be taken to a screen with a new form where we need to
fill out the slave node's information. We're considering the slave node to be running
on Linux operating systems, hence the launch method is set to “Launch agents via ssh”.
In the same way, we'll add relevant details, such as the name, description, and a number
of executors.
We'll save our work by pressing the “Save” button. The “Labels” with the name
“slaveNode1” will help us to set up jobs on this slave node:
DevOps

4. Building the Project on Slave Nodes

Now that our master and slave nodes are ready, we'll discuss the steps for building the project on
the slave node.
For this, we start by clicking “New Item” in the top left corner of the dashboard.
Next, we need to enter the name of our project in the “Enter an item name” field and select the
“Pipeline project”, and then click the “OK” button.
On the next screen, we'll enter a “Description” (optional) and navigate to the “Pipeline” section.
Make sure the “Definition” field has the Pipeline script option selected.
After this, we copy and paste the following declarative Pipeline script into a “script” field:
node('slaveNode1'){
stage('Build') {
sh '''echo build steps'''
}
stage('Test') {
sh '''echo test steps'''
}
}
Copy
Next, we click on the “Save” button. This will redirect to the Pipeline view page.
DevOps

On the left pane, we click the “Build Now” button to execute our Pipeline. After Pipeline execution
is completed, we'll see the Pipeline view:

We can verify the history of the executed build under the Build History by clicking
the build number. As shown above, when we click on the build number and select
“Console Output”, we can see that the pipeline ran on our slaveNode1 machine.
Software on the host
To run software on the host in Jenkins, you need to have the necessary dependencies and tools
installed on the host machine. The exact software you'll need will depend on the specific
requirements of your project and build process. Some common tools and software used in Jenkins
include:
Java: Jenkins is written in Java and requires Java to be installed on the host machine.

Git: If your project uses Git as the version control system, you'll need to have Git installed on the
host machine.

Build Tools: Depending on the programming language and build process of your project, you may
need to install build tools such as Maven, Gradle, or Ant.
Testing Tools: To run tests as part of your build process, you'll need to install any necessary testing
tools, such as JUnit, TestNG, or Selenium.
 DevOps

Database Systems: If your project requires access to a database, you'll need to have the necessary
database software installed on the host machine, such as MySQL, PostgreSQL, or Oracle.

Continuous Integration Plugins: To extend the functionality of Jenkins, you may need to install
plugins that provide additional tools and features for continuous integration, such as the Jenkins
GitHub plugin, Jenkins Pipeline plugin, or Jenkins Slack plugin.
To install these tools and software on the host machine, you can use a package manager such as
apt or yum, or you can download and install the necessary software manually. You can also use a
containerization tool such as Docker to run Jenkins and the necessary software in isolated
containers, which can simplify the installation process and make it easier to manage the
dependencies and tools needed for your build process.

Trigger
These are the most common Jenkins build triggers:

● Trigger builds remotely

● Build after other projects are built
● Build periodically
● GitHub hook trigger for GITScm polling
● Poll SCM

1. Trigger builds remotely :

If you want to trigger your project built from anywhere anytime then you should select Trigger
builds remotely option from the build triggers.

You’ll need to provide an authorization token in the form of a string so that only those who know
it would be able to remotely trigger this project’s builds. This provides the predefined URL to
invoke this trigger remotely.

predefined URL to trigger build remotely:

JENKINS_URL/job/JobName/build?token=TOKEN_NAME

JENKINS_URL: the IP and PORT which the Jenkins server is running

TOKEN_NAME: You have provided while selecting this build trigger.

//Example:

http://e330c73d.ngrok.io/job/test/build?token=12345
DevOps

Whenever you will hit this URL from anywhere you project build will start.

2. Build after other projects are built

If your project depends on another project build then you should select Build after other projects
are built option from the build triggers.

In this, you must specify the project(Job) names in the Projects to watch field section and select
one of the following options:

1. Trigger only if the build is

Note: A build is stable if it was built successfully and no publisher reports it as unstable

2. Trigger even if the build is

Note: A build is unstable if it was built successfully and one or more publishers report it unstable

3. Trigger even if the build fails

After that, It starts watching the specified projects in the Projects to watch section.

Whenever the build of the specified project completes (either is stable, unstable or failed according
to your selected option) then this project build invokes.

3)Build periodically:

If you want to schedule your project build periodically then you should select the Build
periodically option from the build triggers.

You must specify the periodical duration of the project build in the scheduler field section

This field follows the syntax of cron (with minor differences). Specifically, each line consists of 5
fields separated by TAB or whitespace:

MINUTE HOUR DOM MONTH DOW

MINUTE Minutes within the hour (0–59)

 DevOps

HOUR The hour of the day (0–23)

DOM The day of the month (1–31)

MONTH The month (1–12)

DOW The day of the week (0–7) where 0 and 7 are Sunday.

To specify multiple values for one field, the following operators are available. In the order of
precedence,

● * specifies all valid values

● M-N specifies a range of values
● M-N/X or */X steps by intervals of X through the specified range or whole valid range
● A,B,...,Z enumerates multiple values

Examples:

# every fifteen minutes (perhaps at :07, :22, :37, :52)

H/15 * * * *
# every ten minutes in the first half of every hour (three times, perhaps at :04, :14, :24)
H(0-29)/10 * * * *
# once every two hours at 45 minutes past the hour starting at 9:45 AM and finishing at 3:45 PM every weekday.
45 9-16/2 * * 1-5
# once in every two hours slot between 9 AM and 5 PM every weekday (perhaps at 10:38 AM, 12:38 PM, 2:38 PM, 4:38 PM)
H H(9-16)/2 * * 1-5
# once a day on the 1st and 15th of every month except December
H H 1,15 1-11 *

After successfully scheduled the project build then the scheduler will invoke the build periodically
according to your specified duration.

4)GitHub webhook trigger for GITScm polling:

A webhook is an HTTP callback, an HTTP POST that occurs when something happens through a
simple event-notification via HTTP POST.

GitHub webhooks in Jenkins are used to trigger the build whenever a developer commits
something to the branch.

Let’s see how to add build a webhook in GitHub and then add this webhook in Jenkins.
 DevOps

1. Go to your project repository.

2. Go to “settings” in the right corner.
3. Click on “webhooks.”
4. Click “Add webhooks.”
5. Write the Payload URL as

http://e330c73d.ngrok.io/github-webhook

//This URL is a public URL where the Jenkins server is running

Here https://e330c73d.ngrok.io/ is the IP and port where my Jenkins is running.

If you are running Jenkins on localhost then writing https://localhost:8080/github-webhook/ will not
work because Webhooks can only work with the public IP.

So if you want to make your localhost:8080 expose public then we can use some tools.

In this example, we used ngrok tool to expose my local address to the public.

To know more on how to add webhook in Jenkins pipeline, visit: https://blog.knoldus.com/opsinit-

adding-a-github-webhook-in-jenkins-pipeline/

5)Poll SCM:

Poll SCM periodically polls the SCM to check whether changes were made (i.e. new commits) and
builds the project if new commits were pushed since the last build.

You must schedule the polling duration in the scheduler field. Like we explained above in the
Build periodically section. You can see the Build periodically section to know how to schedule.

After successfully scheduled, the scheduler polls the SCM according to your specified duration in
scheduler field and builds the project if new commits were pushed since the last build.LET'S
INITIATE A PARTNERSHIP

Job chaining
Job chaining in Jenkins refers to the process of linking multiple build jobs together in a sequence.
When one job completes, the next job in the sequence is automatically triggered. This allows you
to create a pipeline of builds that are dependent on each other, so you can automate the entire build
process.

There are several ways to chain jobs in Jenkins:

DevOps

Build Trigger: You can use the build trigger in Jenkins to start one job after another. This is done
by configuring the upstream job to trigger the downstream job when it completes.

Jenkinsfile: If you are using Jenkins Pipeline, you can write a Jenkinsfile to define the steps in
your build pipeline. The Jenkinsfile can contain multiple stages, each of which represents a
separate build job in the pipeline.
JobDSL plugin: The JobDSL plugin allows you to programmatically create and manage Jenkins
jobs. You can use this plugin to create a series of jobs that are linked together and run in sequence.
Multi-Job plugin: The Multi-Job plugin allows you to create a single job that runs multiple build
steps, each of which can be a separate build job. This plugin is useful if you have a build pipeline
that requires multiple build jobs to be run in parallel.

By chaining jobs in Jenkins, you can automate the entire build process and ensure that each step
is completed before the next step is started. This can help to improve the efficiency and reliability
of your build process, and allow you to quickly and easily make changes to your build pipeline.

Build pipelines

A build pipeline in DevOps is a set of automated processes that compile, build, and test software,
and prepare it for deployment. A build pipeline represents the end-to-end flow of code changes
from development to production.
DevOps

The steps involved in a typical build pipeline include:

Code Commit: Developers commit code changes to a version control system such as Git.
Build and Compile: The code is built and compiled, and any necessary dependencies are resolved.
Unit Testing: Automated unit tests are run to validate the code changes.

Integration Testing: Automated integration tests are run to validate that the code integrates
correctly with other parts of the system.

Staging: The code is deployed to a staging environment for further testing and validation.
Release: If the code passes all tests, it is deployed to the production environment.
Monitoring: The deployed code is monitored for performance and stability.

A build pipeline can be managed using a continuous integration tool such as Jenkins, TravisCI, or
CircleCI. These tools automate the build process, allowing you to quickly and easily make changes
to the pipeline, and ensuring that the pipeline is consistent and reliable.

In DevOps, the build pipeline is a critical component of the continuous delivery process, and is
used to ensure that code changes are tested, validated, and deployed to production as quickly and
efficiently as possible. By automating the build pipeline, you can reduce the time and effort
required to deploy code changes, and improve the speed and quality of your software delivery
process.

Build servers
When you're developing and deploying software, one of the first things to figure out is how to take
your code and deploy your working application to a production environment where people can
interact with your software.

Most development teams understand the importance of version control to coordinate code
commits, and build servers to compile and package their software, but Continuous Integration (CI)
is a big topic.

Why build servers are important

Build servers have 3 main purposes:

● Compiling committed code from your repository many times a day

● Running automatic tests to validate code
● Creating deployable packages and handing off to a deployment tool, like Octopus Deploy
DevOps

Without a build server you're slowed down by complicated, manual processes and the needless
time constraints they introduce. For example, without a build server:

● Your team will likely need to commit code before a daily deadline or during change
windows
● After that deadline passes, no one can commit again until someone manually creates and
tests a build
● If there are problems with the code, the deadlines and manual processes further delay the
fixes

Without a build server, the team battles unnecessary hurdles that automation removes. A build
server will repeat these tasks for you throughout the day, and without those human-caused delays.

But CI doesn’t just mean less time spent on manual tasks or the death of arbitrary deadlines, either.
By automatically taking these steps many times a day, you fix problems sooner and your results
become more predictable. Build servers ultimately help you deploy through your pipeline with
more confidence.

Building servers in DevOps involves several steps:

Requirements gathering: Determine the requirements for the server, such as hardware
specifications, operating system, and software components needed.
Server provisioning: Choose a method for provisioning the server, such as physical installation,
virtualization, or cloud computing.

Operating System installation: Install the chosen operating system on the server.
Software configuration: Install and configure the necessary software components, such as web
servers, databases, and middleware.

Network configuration: Set up network connectivity, such as IP addresses, hostnames, and

firewall rules.
Security configuration: Configure security measures, such as user authentication, access control,
and encryption.

Monitoring and maintenance: Implement monitoring and maintenance processes, such as

logging, backup, and disaster recovery.
Deployment: Deploy the application to the server and test it to ensure it is functioning as expected.

Throughout the process, it is important to automate as much as possible using tools such as
Ansible, Chef, or Puppet to ensure consistency and efficiency in building servers.
DevOps

Infrastructure as code

Infrastructure as code (IaC) uses DevOps methodology and versioning with a descriptive model
to define and deploy infrastructure, such as networks, virtual machines, load balancers, and
connection topologies. Just as the same source code always generates the same binary, an IaC
model generates the same environment every time it deploys.

IaC is a key DevOps practice and a component of continuous delivery. With IaC, DevOps teams
can work together with a unified set of practices and tools to deliver applications and their
supporting infrastructure rapidly and reliably at scale.

IaC evolved to solve the problem of environment drift in release pipelines. Without IaC, teams
must maintain deployment environment settings individually. Over time, each environment
becomes a "snowflake," a unique configuration that can't be reproduced automatically.
Inconsistency among environments can cause deployment issues. Infrastructure administration and
maintenance involve manual processes that are error prone and hard to track.

IaC avoids manual configuration and enforces consistency by representing desired environment
states via well-documented code in formats such as JSON. Infrastructure deployments with IaC
are repeatable and prevent runtime issues caused by configuration drift or missing dependencies.
Release pipelines execute the environment descriptions and version configuration models to
configure target environments. To make changes, the team edits the source, not the target.

Idempotence, the ability of a given operation to always produce the same result, is an important
IaC principle. A deployment command always sets the target environment into the same
configuration, regardless of the environment's starting state. Idempotency is achieved by either
DevOps

automatically configuring the existing target, or by discarding the existing target and recreating a
fresh environment.

IAC can be achieved by using tools such as Terraform, CloudFormation, or Ansible to define
infrastructure components in a file that can be versioned, tested, and deployed in a consistent and
automated manner.

Benefits of IAC include:

Speed: IAC enables quick and efficient provisioning and deployment of infrastructure.
Consistency: By using code to define and manage infrastructure, it is easier to ensure consistency
across multiple environments.
Repeatability: IAC allows for easy replication of infrastructure components in different
environments, such as development, testing, and production.
Scalability: IAC makes it easier to scale infrastructure as needed by simply modifying the code.

Version control: Infrastructure components can be versioned, allowing for rollback to previous
versions if necessary.
Overall, IAC is a key component of modern DevOps practices, enabling organizations to manage their
infrastructure in a more efficient, reliable, and scalable way.

Building by dependency order

Building by dependency order in DevOps is the process of ensuring that the components of a
system are built and deployed in the correct sequence, based on their dependencies. This is
necessary to ensure that the system functions as intended, and that components are deployed in the
right order so that they can interact correctly with each other.
The steps involved in building by dependency order in DevOps include:

Define dependencies: Identify all the components of the system and the dependencies between
them. This can be represented in a diagram or as a list.
Determine the build order: Based on the dependencies, determine the correct order in which
components should be built and deployed.

Automate the build process: Use tools such as Jenkins, TravisCI, or CircleCI to automate the
build and deployment process. This allows for consistency and repeatability in the build process.
Monitor progress: Monitor the progress of the build and deployment process to ensure that
components are deployed in the correct order and that the system is functioning as expected.
DevOps

Test and validate: Test the system after deployment to ensure that all components are functioning
as intended and that dependencies are resolved correctly.

Rollback: If necessary, have a rollback plan in place to revert to a previous version of the system
if the build or deployment process fails.
In conclusion, building by dependency order in DevOps is a critical step in ensuring the success
of a system deployment, as it ensures that components are deployed in the correct order and that
dependencies are resolved correctly. This results in a more stable, reliable, and consistent system.

Build phases
In DevOps, there are several phases in the build process, including:

Planning: Define the project requirements, identify the dependencies, and create a build plan.
Code development: Write the code and implement features, fixing bugs along the way.

Continuous Integration (CI): Automatically build and test the code as it is committed to a version
control system.
Continuous Delivery (CD): Automatically deploy code changes to a testing environment, where
they can be tested and validated.

Deployment: Deploy the code changes to a production environment, after they have passed testing
in a pre-production environment.

Monitoring: Continuously monitor the system to ensure that it is functioning as expected, and to
detect and resolve any issues that may arise.
Maintenance: Continuously maintain and update the system, fixing bugs, adding new features,
and ensuring its stability.
These phases help to ensure that the build process is efficient, reliable, and consistent, and that
code changes are validated and deployed in a controlled manner. Automation is a key aspect of
DevOps, and it helps to make these phases more efficient and less prone to human error.

In continuous integration (CI), this is where we build the application for the first time. The build
stage is the first stretch of a CI/CD pipeline, and it automates steps like downloading dependencies,
installing tools, and compiling.

Besides building code, build automation includes using tools to check that the code is safe and
follows best practices. The build stage usually ends in the artifact generation step, where we create
a production-ready package. Once this is done, the testing stage can begin.
 DevOps

The build stage starts from code commit and runs from the beginning up to the test stage

We’ll be covering testing in-depth in future articles (subscribe to the newsletter so you don’t miss
them). Today, we’ll focus on build automation.

Build automation verifies that the application, at a given code commit, can qualify for further
testing. We can divide it into three parts:

1. Compilation: the first step builds the application.

2. Linting: checks the code for programmatic and stylistic errors.
3. Code analysis: using automated source-checking tools, we control the code’s quality.
4. Artifact generation: the last step packages the application for release or deployment.

Alternative build servers

There are several alternative build servers in DevOps, including:
Jenkins - an open-source, Java-based automation server that supports various plugins and
integrations.

Travis CI - a cloud-based, open-source CI/CD platform that integrates with Github.

CircleCI - a cloud-based, continuous integration and delivery platform that supports multiple
languages and integrates with several platforms.

GitLab CI/CD - an integrated CI/CD solution within GitLab that allows for complete project and
pipeline management.
Bitbucket Pipelines - a CI/CD solution within Bitbucket that allows for pipeline creation and
management within the code repository.
AWS CodeBuild - a fully managed build service that compiles source code, runs tests, and
produces software packages that are ready to deploy.

Azure Pipelines - a CI/CD solution within Microsoft Azure that supports multiple platforms and
programming languages.
DevOps

Collating quality measures

In DevOps, collating quality measures is an important part of the continuous improvement process.
The following are some common quality measures used in DevOps to evaluate the quality of
software systems:
Continuous Integration (CI) metrics - metrics that track the success rate of automated builds and
tests, such as build duration and test pass rate.

Continuous Deployment (CD) metrics - metrics that track the success rate of deployments, such
as deployment frequency and time to deployment.
Code review metrics - metrics that track the effectiveness of code reviews, such as review
completion time and code review feedback.
Performance metrics - measures of system performance in production, such as response time and
resource utilization.

User experience metrics - measures of how users interact with the system, such as click-through
rate and error rate.
Security metrics - measures of the security of the system, such as the number of security
vulnerabilities and the frequency of security updates.

Incident response metrics - metrics that track the effectiveness of incident response, such as mean
time to resolution (MTTR) and incident frequency.
By regularly collating these quality measures, DevOps teams can identify areas for improvement,
track progress over time, and make informed decisions about the quality of their systems.
DevOps

Unit 5
Testing Tools and automation
As we know, software testing is a process of analyzing an application's functionality as per the
customer prerequisite.

If we want to ensure that our software is bug-free or stable, we must perform the various types of
software testing because testing is the only method that makes our application bug-free.

Various types of testing

The categorization of software testing is a part of diverse testing activities, such as test strategy,
test deliverables, a defined test objective, etc. And software testing is the execution of the
software to find defects.

The purpose of having a testing type is to confirm the AUT (Application Under Test).

To start testing, we should have a requirement, application-ready, necessary resources

available. To maintain accountability, we should assign a respective module to different test
engineers.

The software testing mainly divided into two parts, which are as follows:

o Manual Testing
o Automation Testing
DevOps

What is Manual Testing?

Testing any software or an application according to the client's needs without using any automation
tool is known as manual testing.

In other words, we can say that it is a procedure of verification and validation. Manual testing is
used to verify the behavior of an application or software in contradiction of requirements
specification.

We do not require any precise knowledge of any testing tool to execute the manual test cases. We
can easily prepare the test document while performing manual testing on any application.

To get in-detail information about manual testing, click on the following link:
https://www.javatpoint.com/manual-testing.

Classification of Manual Testing

In software testing, manual testing can be further classified into three different types of testing,
which are as follows:

o White Box Testing

o Black Box Testing
o Grey Box Testing
DevOps

For our better understanding let's see them one by one:

White Box Testing

In white-box testing, the developer will inspect every line of code before handing it over to the
testing team or the concerned test engineers.

Subsequently, the code is noticeable for developers throughout testing; that's why this process is
known as WBT (White Box Testing).

In other words, we can say that the developer will execute the complete white-box testing for the
particular software and send the specific application to the testing team.

The purpose of implementing the white box testing is to emphasize the flow of inputs and outputs
over the software and enhance the security of an application.

White box testing is also known as open box testing, glass box testing, structural testing, clear
box testing, and transparent box testing.

Black Box Testing

Another type of manual testing is black-box testing. In this testing, the test engineer will analyze
the software against requirements, identify the defects or bug, and sends it back to the development
team.

Then, the developers will fix those defects, do one round of White box testing, and send it to the
testing team.
DevOps

Here, fixing the bugs means the defect is resolved, and the particular feature is working according
to the given requirement.

The main objective of implementing the black box testing is to specify the business needs or the
customer's requirements.

In other words, we can say that black box testing is a process of checking the functionality of an
application as per the customer requirement. The source code is not visible in this testing; that's
why it is known as black-box testing.

Types of Black Box Testing

Black box testing further categorizes into two parts, which are as discussed below:

o Functional Testing
o Non-function Testing
DevOps

Functional Testing

The test engineer will check all the components systematically against requirement specifications
is known as functional testing. Functional testing is also known as Component testing.

In functional testing, all the components are tested by giving the value, defining the output, and
validating the actual output with the expected value.

Functional testing is a part of black-box testing as its emphases on application requirement rather
than actual code. The test engineer has to test only the program instead of the system.

Types of Functional Testing

Just like another type of testing is divided into several parts, functional testing is also classified
into various categories.

The diverse types of Functional Testing contain the following:

o Unit Testing
o Integration Testing
o System Testing

Now, Let's understand them one by one:

DevOps

1. Unit Testing

Unit testing is the first level of functional testing in order to test any software. In this, the test
engineer will test the module of an application independently or test all the module functionality
is called unit testing.

The primary objective of executing the unit testing is to confirm the unit components with their
performance. Here, a unit is defined as a single testable function of a software or an application.
And it is verified throughout the specified application development phase.

2. Integration Testing

Once we are successfully implementing the unit testing, we will go integration testing. It is the
second level of functional testing, where we test the data flow between dependent modules or
interface between two features is called integration testing.

The purpose of executing the integration testing is to test the statement's accuracy between each
module.

Types of Integration Testing

Integration testing is also further divided into the following parts:

o Incremental Testing
o Non-Incremental Testing

Incremental Integration Testing

Whenever there is a clear relationship between modules, we go for incremental integration testing.
Suppose, we take two modules and analysis the data flow between them if they are working fine
or not.

If these modules are working fine, then we can add one more module and test again. And we can
continue with the same process to get better results.

In other words, we can say that incrementally adding up the modules and test the data flow between
the modules is known as Incremental integration testing.

Types of Incremental Integration Testing

DevOps

Incremental integration testing can further classify into two parts, which are as follows:

1. Top-down Incremental Integration Testing

2. Bottom-up Incremental Integration Testing

Let's see a brief introduction of these types of integration testing:

1. Top-down Incremental Integration Testing

In this approach, we will add the modules step by step or incrementally and test the data flow
between them. We have to ensure that the modules we are adding are the child of the earlier ones.

2. Bottom-up Incremental Integration Testing

In the bottom-up approach, we will add the modules incrementally and check the data flow
between modules. And also, ensure that the module we are adding is the parent of the earlier
ones.

Non-Incremental Integration Testing/ Big Bang Method

Whenever the data flow is complex and very difficult to classify a parent and a child, we will go
for the non-incremental integration approach. The non-incremental method is also known as the
Big Bang method.

3. System Testing
Whenever we are done with the unit and integration testing, we can proceed with the system
testing.

In system testing, the test environment is parallel to the production environment. It is also known
as end-to-end testing.

In this type of testing, we will undergo each attribute of the software and test if the end feature
works according to the business requirement. And analysis the software product as a complete
system.

Non-function Testing

The next part of black-box testing is non-functional testing. It provides detailed information on
software product performance and used technologies.
DevOps

Non-functional testing will help us minimize the risk of production and related costs of the
software.

Non-functional testing is a combination of performance, load, stress, usability and,

compatibility testing.

Types of Non-functional Testing

Non-functional testing categorized into different parts of testing, which we are going to discuss
further:

o Performance Testing
o Usability Testing
o Compatibility Testing

1. Performance Testing

In performance testing, the test engineer will test the working of an application by applying some
load.

In this type of non-functional testing, the test engineer will only focus on several aspects, such
as Response time, Load, scalability, and Stability of the software or an application.

Classification of Performance Testing

Performance testing includes the various types of testing, which are as follows:

o Load Testing
o Stress Testing
o Scalability Testing
o Stability Testing
o Load Testing

While executing the performance testing, we will apply some load on the particular application to
check the application's performance, known as load testing. Here, the load could be less than or
equal to the desired load.

It will help us to detect the highest operating volume of the software and bottlenecks.

o Stress Testing
DevOps

It is used to analyze the user-friendliness and robustness of the software beyond the common
functional limits.

Primarily, stress testing is used for critical software, but it can also be used for all types of software
applications.

o Scalability Testing

To analysis, the application's performance by enhancing or reducing the load in particular balances
is known as scalability testing.

In scalability testing, we can also check the system, processes, or database's ability to meet an
upward need. And in this, the Test Cases are designed and implemented efficiently.

o Stability Testing

Stability testing is a procedure where we evaluate the application's performance by applying the
load for a precise time.

It mainly checks the constancy problems of the application and the efficiency of a developed
product. In this type of testing, we can rapidly find the system's defect even in a stressful situation.

2. Usability Testing

Another type of non-functional testing is usability testing. In usability testing, we will analyze
the user-friendliness of an application and detect the bugs in the software's end-user interface.

Here, the term user-friendliness defines the following aspects of an application:

o The application should be easy to understand, which means that all the features must be
visible to end-users.
o The application's look and feel should be good that means the application should be
pleasant looking and make a feel to the end-user to use it.

3. Compatibility Testing

In compatibility testing, we will check the functionality of an application in specific hardware and
software environments. Once the application is functionally stable then only, we go
for compatibility testing.
DevOps

Here, software means we can test the application on the different operating systems and other
browsers, and hardware means we can test the application on different sizes.

Grey Box Testing

Another part of manual testing is Grey box testing. It is a collaboration of black box and white
box testing.

Since, the grey box testing includes access to internal coding for designing test cases. Grey box
testing is performed by a person who knows coding as well as testing.

In other words, we can say that if a single-person team done both white box and black-box
testing, it is considered grey box testing.

Automation Testing

The most significant part of Software testing is Automation testing. It uses specific tools to
automate manual design test cases without any human interference.

Automation testing is the best way to enhance the efficiency, productivity, and coverage of
Software testing.

It is used to re-run the test scenarios, which were executed manually, quickly, and repeatedly.

In other words, we can say that whenever we are testing an application by using some tools is
known as automation testing.

We will go for automation testing when various releases or several regression cycles goes on the
application or software. We cannot write the test script or perform the automation testing without
understanding the programming language.
DevOps

Automation of testing Pros and cons

Some other types of Software Testing

In software testing, we also have some other types of testing that are not part of any above
discussed testing, but those testing are required while testing any software or an application.

o Smoke Testing
o Sanity Testing
o Regression Testing
o User Acceptance Testing
o Exploratory Testing
o Adhoc Testing
o Security Testing
o Globalization Testing

Let's understand those types of testing one by one:

In smoke testing, we will test an application's basic and critical features before doing one round
of deep and rigorous testing.

Or before checking all possible positive and negative values is known as smoke testing.
Analyzing the workflow of the application's core and main functions is the main objective of
performing the smoke testing.

Sanity Testing

It is used to ensure that all the bugs have been fixed and no added issues come into existence due
to these changes. Sanity testing is unscripted, which means we cannot documented it. It checks the
correctness of the newly added features and components.

Regression Testing

Regression testing is the most commonly used type of software testing. Here, the
term regression implies that we have to re-test those parts of an unaffected application.

Regression testing is the most suitable testing for automation tools. As per the project type and
accessibility of resources, regression testing can be similar to Retesting.
DevOps

Whenever a bug is fixed by the developers and then testing the other features of the applications
that might be simulated because of the bug fixing is known as regression testing.

In other words, we can say that whenever there is a new release for some project, then we can
perform Regression Testing, and due to a new feature may affect the old features in the earlier
releases.

User Acceptance Testing

The User acceptance testing (UAT) is done by the individual team known as domain
expert/customer or the client. And knowing the application before accepting the final product is
called as user acceptance testing.

In user acceptance testing, we analyze the business scenarios, and real-time scenarios on the
distinct environment called the UAT environment. In this testing, we will test the application
before UAI for customer approval.

Exploratory Testing

Whenever the requirement is missing, early iteration is required, and the testing team has
experienced testers when we have a critical application. New test engineer entered into the team
then we go for the exploratory testing.

To execute the exploratory testing, we will first go through the application in all possible ways,
make a test document, understand the flow of the application, and then test the application.

Adhoc Testing

Testing the application randomly as soon as the build is in the checked sequence is known
as Adhoc testing.

It is also called Monkey testing and Gorilla testing. In Adhoc testing, we will check the
application in contradiction of the client's requirements; that's why it is also known as negative
testing.

When the end-user using the application casually, and he/she may detect a bug. Still, the
specialized test engineer uses the software thoroughly, so he/she may not identify a similar
detection.

Security Testing
DevOps

It is an essential part of software testing, used to determine the weakness, risks, or threats in the
software application.

The execution of security testing will help us to avoid the nasty attack from outsiders and ensure
our software applications' security.

In other words, we can say that security testing is mainly used to define that the data will be safe
and endure the software's working process.

Globalization Testing

Another type of software testing is Globalization testing. Globalization testing is used to check
the developed software for multiple languages or not. Here, the words globalization means
enlightening the application or software for various languages.

Globalization testing is used to make sure that the application will support multiple languages and
multiple features.

In present scenarios, we can see the enhancement in several technologies as the applications are
prepared to be used globally.

Conclusion

In the tutorial, we have discussed various types of software testing. But there is still a list of more
than 100+ categories of testing. However, each kind of testing is not used in all types of projects.

We have discussed the most commonly used types of Software Testing like black-box testing,
white box testing, functional testing, non-functional testing, regression testing, Adhoc testing,
etc.

Also, there are alternate classifications or processes used in diverse organizations, but the general
concept is similar all over the place.

These testing types, processes, and execution approaches keep changing when the project,
requirements, and scope change.

Automation of testing Pros and cons

Pros of Automated Testing:

Automated Testing has the following advantages:
DevOps

1. Automated testing improves the coverage of testing as automated execution of test cases is
faster than manual execution.
2. Automated testing reduces the dependability of testing on the availability of the test
engineers.
3. Automated testing provides round the clock coverage as automated tests can be run all time
in 24*7 environment.
4. Automated testing takes far less resources in execution as compared to manual testing.
5. It helps to train the test engineers to increase their knowledge by producing a repository of
different tests.
6. It helps in testing which is not possible without automation such as reliability testing, stress
testing, load and performance testing.
7. It includes all other activities like selecting the right product build, generating the right test
data and analyzing the results.
8. It acts as test data generator and produces maximum test data to cover a large number of
input and expected output for result comparison.
9. Automated testing has less chances of error hence more reliable.
10. As with automated testing test engineers have free time and can focus on other creative
tasks.
Cons of Automated Testing :Automated Testing has the following disadvantages:
1. Automated testing is very much expensive than the manual testing.
2. It also becomes inconvenient and burdensome as to decide who would automate and who
would train.
3. It has limited to some organisations as many organisations not prefer test automation.
4. Automated testing would also require additionally trained and skilled people.
5. Automated testing only removes the mechanical execution of testing process, but creation
of test cases still required testing professionals.
Selenium
Introduction
Selenium is one of the most widely used open source Web UI (User Interface) automation testing
suite.It was originally developed by Jason Huggins in 2004 as an internal tool at Thought Works.
Selenium supports automation across different browsers, platforms and programming languages.

Selenium can be easily deployed on platforms such as Windows, Linux, Solaris and Macintosh.
Moreover, it supports OS (Operating System) for mobile applications like iOS, windows mobile
and android.

Selenium supports a variety of programming languages through the use of drivers specific to each
Language.
DevOps

Languages supported by Selenium include C#, Java, Perl, PHP, Python and Ruby.

Currently, Selenium Web driver is most popular with Java and C#. Selenium test scripts can be
coded in any of the supported programming languages and can be run directly in most modern web
browsers. Browsers supported by Selenium include Internet Explorer, Mozilla Firefox, Google
Chrome and Safari.

Selenium can be used to automate functional tests and can be integrated with automation test tools
such as Maven, Jenkins, & Docker to achieve continuous testing. It can also be integrated with
tools such as TestNG, & JUnit for managing test cases and generating reports.

Selenium Features

o Selenium is an open source and portable Web testing Framework.

o Selenium IDE provides a playback and record feature for authoring tests without the need
to learn a test scripting language.
o It can be considered as the leading cloud-based testing platform which helps testers to
record their actions and export them as a reusable script with a simple-to-understand and
easy-to-use interface.
o Selenium supports various operating systems, browsers and programming languages.
Following is the list:
o Programming Languages: C#, Java, Python, PHP, Ruby, Perl, and JavaScript
o Operating Systems: Android, iOS, Windows, Linux, Mac, Solaris.
o Browsers: Google Chrome, Mozilla Firefox, Internet Explorer, Edge, Opera, Safari,
etc.
o It also supports parallel test execution which reduces time and increases the efficiency of
tests.
DevOps

o Selenium can be integrated with frameworks like Ant and Maven for source code
compilation.
o Selenium can also be integrated with testing frameworks like TestNG for application
testing and generating rseports.
o Selenium requires fewer resources as compared to other automation test tools.
o WebDriver API has been indulged in selenium whichis one of the most important
modifications done to selenium.
o Selenium web driver does not require server installation, test scripts interact directly with
the browser.
o Selenium commands are categorized in terms of different classes which make it easier to
understand and implement.

JavaScript testing
JavaScript testing is a crucial part of the software development process that helps ensure the quality
and reliability of code. The following are the key components of JavaScript testing:
Test frameworks: A test framework provides a structure for writing and organizing tests. Some
popular JavaScript test frameworks include Jest, Mocha, and Jasmine.

Assertion libraries: An assertion library provides a set of functions that allow developers to write
assertions about the expected behavior of the code. For example, an assertion might check that a
certain function returns the expected result.

Test suites: A test suite is a collection of related tests that are grouped together. The purpose of a
test suite is to test a specific aspect of the code in isolation.
Test cases: A test case is a single test that verifies a specific aspect of the code. For example, a
test case might check that a function behaves correctly when given a certain input.

Test runners: A test runner is a tool that runs the tests and provides feedback on the results. Test
runners typically provide a report on which tests passed and which tests failed.
Continuous Integration (CI): CI is a software development practice where developers integrate
code into a shared repository frequently. By using CI, developers can catch issues early and avoid
integration problems.
The goal of JavaScript testing is to catch bugs and defects early in the development cycle, before
they become bigger problems and impact the quality of the software. Testing also helps to ensure
that the code behaves as expected, even when changes are made in the future.
DevOps

There are different types of tests that can be performed in JavaScript, including unit tests,
integration tests, and end-to-end tests. The choice of which tests to write depends on the specific
requirements and goals of the project.

Testing backend integration points

The term backend generally refers to server-side deployment. Here the process is entirely
happening in the backend which is not shown to the user only the expected results will be
shown to the user. In every web application, there will be a backend language to accomplish
the task.
For Example, while uploading the details of the students in the database, the database will
store all the details. When there is a need to display the details of the students, it will simply
fetch all the details and display them. Here, it will show only the result, not the process and
how it fetches the details.

What is Backend Testing?

Backend Testing is a testing method that checks the database or server-side of the web
application. The main purpose of backend testing is to check the application layer and the
database layer. It will find an error or bug in the database or server-side.
For implementing backend testing, the backend test engineer should also have some
knowledge about that particular server-side or database language. It is also known
as Database Testing.
Importance of Backend Testing: Backend testing is a must because anything wrong or
error happens at the server-side, it will not further proceed with that task or the output will
get differed or sometimes it will also cause problems such as data loss, deadlock, etc.,

Types of Backend Testing

The following are the different types of backend testing:

1. Structural Testing
2. Functional Testing
3. Non-Functional Testing
Let’s discuss each of these types of backend testing.
DevOps

1. Structural Testing
Structural testing is the process of validating all the elements that are present inside the
data repository and are primarily used for data storage. It involves checking the objects of
front-end developments with the database mapping objects.
Types of Structural Testing: The following are the different types of structural testing:
a) Schema Testing: In this Schema Testing, the tester will check for the correctly mapped
objects. This is also known as mapping testing. It ensures whether the objects of the front-
end and the objects of the back-end are correctly matched or mapped. It will mainly focus
on schema objects such as a table, view, indexes, clusters, etc., In this testing, the tester
will find the issues of mapped objects like table, view, etc.,
b) Table and Column Testing: In this, it ensures that the table and column properties are
correctly mapped.
● It ensures whether the table and the column names are correctly mapped on both the
front-end side and server-side.
● It validates the datatype of the column is correctly mentioned.
● It ensures the correct naming of the column values of the database.
● It detects the unused tables and columns.
● It validates whether the users are able to give the correct input as per the requirement.
For example, if we mention the wrong datatype for the column on the server-side which is
different from the front-end then it will raise an error.
c) Key and Indexes Testing: In this, it validates the key and indexes of the columns.
● It ensures whether the mentioned key constraints are correctly provided. For example,
Primary Key for the column is correctly mentioned as per the given requirement.
● It ensures the correct references of Foreign Key with the parent table.
● It checks the length and size of the indexes.
● It ensures the creation of clustered and non-clustered indexes for the table as per the
requirement.
● It validates the naming conventions of the Keys.
d) Trigger Testing: It ensures that the executed triggers are fulfilling the required conditions
of the DML transactions.
● It validates whether the triggers make the data updates correctly when we have executed
them.
● It checks the coding conventions are followed correctly during the coding phase of the
triggers.
● It ensures that the trigger functionalities of update, delete, and insert.
DevOps

e) Stored Procedures Testing: In this, the tester checks for the correctness of the stored
procedure results.
● It checks whether the stored procedure contains the valid conditions for looping and
conditional statements as per the requirement.
● It validates the exception and error handling in the stored procedure.
● It detects the unused stored procedure.
● It validates the cursor operations.
● It validates whether the TRIM operations are correctly applied or not.
● It ensures that the required triggers are implicitly invoked by executing the stored
procedures.
f) Database Server Validation Testing: It validates the database configuration details as
per the requirements.
● It validates that the transactions of the data are made as per the requirements.
● It validates the user’s authentication and authorization.
For Example, If wrong user authentication is given, it will raise an error.

2. Functional Testing

Functional Testing is the process of validating that the transactions and operations made
by the end-users meet the requirements.
Types of Functional Testing: The following are the different types of functional testing:
a) Black Box Testing:
● Black Box Testing is the process of checking the functionalities of the integration of
the database.
● This testing is carried out at the early stage of development and hence It is very helpful
to reduce errors.
● It consists of various techniques such as boundary analysis, equivalent partitioning, and
cause-effect graphing.
● These techniques are helpful in checking the functionality of the database.
● The best example is the User login page. If the entered username and password are
correct, It will allow the user and redirect to the next page.
b) White Box Testing:
● White Box Testing is the process of validating the internal structure of the database.
● Here, the specified details are hidden from the user.
● The database triggers, functions, views, queries, and cursors will be checked in this
testing.
DevOps

● It validates the database schema, database table, etc.,

● Here the coding errors in the triggers can be easily found.
● Errors in the queries can also be handled in this white box testing and hence internal
errors are easily eliminated.
3. Non-Functional Testing
Non-functional testing is the process of performing load testing, stress testing, and
checking minimum system requirements are required to meet the requirements. It will also
detect risks, and errors and optimize the performance of the database.
a) Load Testing:
● Load testing involves testing the performance and scalability of the database.
● It determines how the software behaves when it is been used by many users
simultaneously.
● It focuses on good load management.
● For example, if the web application is accessed by multiple users at the same time and
it does not create any traffic problems then the load testing is successfully completed.
b) Stress Testing:
● Stress Testing is also known as endurance testing. Stress testing is a testing process that
is performed to identify the breakpoint of the system.
● In this testing, an application is loaded till the stage the system fails.
● This point is known as a breakpoint of the database system.
● It evaluates and analyzes the software after the breakage of system failure. In case of
error detection, It will display the error messages.
● For example, if users enter the wrong login information then it will throw an error
message.
DevOps

Backend Testing Process

1. Set up the Test Environment: When the coding process is done for the application, set
up the test environment by choosing a proper testing tool for back-end testing. It includes
choosing the right team to test the entire back-end environment with a proper schedule.
Record all the testing processes in the documents or update them in software to keep track
of all the processes.
2. Generate the Test Cases: Once the tool and the team are ready for the testing process,
generate the test cases as per the business requirements. The automation tool itself will
analyze the code and generate all possible test cases for developed code. If the process is
manual then the tester will have to write the possible test cases in the testing tool to ensure
the correctness of the code.
3. Execution of Test Cases: Once the test cases are generated, the tester or Quality Analyst
needs to execute those test cases in the developed code. If the tool is automated, it will
generate and execute the test cases by itself. Otherwise, the tester needs to write and
execute those test cases. It will highlight whether the execution of test cases is executed
successfully or not.
4. Analyzing the Test Cases: After the execution of test cases, it highlights the result of
all the test cases whether it has been executed successfully or not. If an error occurs in the
test cases, it will highlight where the particular error is formed or raised, and in some cases,
DevOps

the automation tool will give hints regarding the issues to solve the error. The tester or
Quality Analyst should analyze the code again and fix the issues if an error occurred.
5. Submission of Test Reports: This is the last stage in the testing process. Here, all the
details such as who is responsible for testing, the tool used in the testing process, number
of test cases generated, number of test cases executed successfully or not, time is taken to
execute each test case, number of times test cases failed, number of times errors occurred.
These details are either documented or updated in the software. The report will be
submitted to the respective team.

Backend Testing Validation

The following are some of the factors for backend testing validation:
● Performance Check: It validates the performance of each individual test and the
system behavior.
● Sequence Testing: Backend testing validates that the tests are distributed according to
the priority.
● Database Server Validations: In this, ensures that the data fed through for the tests is
correct or not.
● Functions Testing: In this, the test validates the consistency in transactions of the
database.
● Key and Indexes: In this, the test ensures that the accurate constraint and the rules of
constraints and indexes are followed properly.
● Data Integrity Testing: It is a technique in which data is verified in the database
whether it is accurate and functions as per requirements.
● Database Tables: It ensures that the created table and the queries for the output are
providing the expected result.
● Database Triggers: Backend Testing validates the correctness of the functionality of
triggers.
● Stored Procedures: Backend testing validates the functions, return statements, calling
the other events, etc., are correctly mentioned as per the requirements,
● Schema: Backend testing validates that the data is organized in a correct way as per the
business requirement and confirms the outcome.

Tools For Backend Testing

The following are some of the tools for backend testing:

DevOps

1. LoadRunner:
● It is a stress testing tool.
● It is an automated performance and testing automation tool for analyzing system
behavior and the performance of the system while generating the actual load.
2. Empirix-TEST Suite:
● It is acquired by Oracle from Empirix. It is a load testing tool.
● It validates the scalability along with the functionality of the application under heavy
test.
● Acquisition with the Empirix -Test suite may be proven effective to deliver the
application with improved quality.
3. Stored Procedure Testing Tools – LINQ:
● It is a powerful tool that allows the user to show the projects.
● It tracks all the ORM calls and database queries from the ORM.
● It enables to see the performance of the data access code and easily determine
performance.
4. Unit Testing Tools – SQL Unit, DBFit, NDbUnit:
● SQL UNIT: SQLUnit is a Unit Testing Framework for Regression and Unit Testing of
database stored procedures.
● DBFit: It is a part of FitNesse and manages stored procedures and custom procedures.
Accomplishes database testing either through Java or .NET and runs from the command
line.
● NDbUnit: It performs the database unit test for the system either before or after
execution or compiled the other parts of the system.
5. Data Factory Tools:
● These tools work as data managers and data generators for backend database testing.
● It is used to validate the queries with a huge set of data.
● It allows performing both stress and load testing.
6. SQLMap:
● It is an open-source tool.
● It is used for performing Penetration Testing to automate the process of detection.
● Powerful detection of errors will lead to efficient testing and result in the expected
behavior of the requirements.
7.phpMyadmin:
● This is the software tool and it is written in PHP.
● It is developed to handle the databases and we can execute test queries to ensure the
correctness of the result as a whole and even for a separate table.
DevOps

8. Automatic Efficient Test Generator (AETG):

● It mechanically generates the possible tests from user-defined requirements.
● It is based on algorithms that use ideas from statistical experimental design theory to
reduce the number of tests needed for a specific level of test coverage of the input test
space.
9. Hammer DB:
● It is an open-source tool for load testing.
● It validates the activity replay functionality for the oracle database.
● It is based on industry standards like TPC-C and TPC-H Benchmarks.
10. SQL Test:
● SQL Test uses an open-source tSQLt framework, views, stored procedures, and
functions.
● This tool stores database object in a separate schema and if changes occur there is no
need for clearing the process.
● It allows running the unit test cases for the SQL server database.

Advantages of Backend Testing

The following are some of the benefits of backend testing:

● Errors are easily detectable at the earlier stage.
● It avoids deadlock creation on the server-side.
● Web load management is easily achieved.
● The functionality of the database is maintained properly.
● It reduces data loss.
● Enhances the functioning of the system.
● It ensures the security and protection of the system.
● While doing the backend testing, the errors in the UI parts can also be detected and
replaced.
● Coverage of all possible test cases.

Disadvantages of Backend Testing

The following are some of the disadvantages of backend testing:

● Good domain knowledge is required.
● Providing test cases for testing requires special attention.
DevOps

● Investment in Organizational costs is higher.

● It takes more time to test.
● If more testing becomes fails then It will lead to a crash on the server-side in some
cases.
● Errors or Unexpected results from one test case scenario will affect the other system
results also.

Test-driven development

Test Driven Development (TDD) is software development approach in which test cases
are developed to specify and validate what the code will do. In simple terms, test cases
for each functionality are created and tested first and if the test fails then the new code
is written in order to pass the test and making code simple and bug-free.
Test-Driven Development starts with designing and developing tests for every small
functionality of an application. TDD framework instructs developers to write new code
only if an automated test has failed. This avoids duplication of code. The TDD full form
is Test-driven development.

The simple concept of TDD is to write and correct the failed tests before writing new code (before
development). This helps to avoid duplication of code as we write a small amount of code at a time
in order to pass tests. (Tests are nothing but requirement conditions that we need to test to fulfill
them).

Test-Driven development is a process of developing and running automated test before actual
development of the application. Hence, TDD sometimes also called as Test First Development.
DevOps

How to perform TDD Test

Following steps define how to perform TDD test,

1. Add a test.
2. Run all tests and see if any new test fails.
3. Write some code.
4. Run tests and Refactor code.
5. Repeat

TDD Vs. Traditional Testing

Below is the main difference between Test driven development and traditional testing:

TDD approach is primarily a specification technique. It ensures that your source code
is thoroughly tested at confirmatory level.
DevOps

● With traditional testing, a successful test finds one or more defects. It is same as
TDD. When a test fails, you have made progress because you know that you need
to resolve the problem.
● TDD ensures that your system actually meets requirements defined for it. It helps
to build your confidence about your system.
● In TDD more focus is on production code that verifies whether testing will work
properly. In traditional testing, more focus is on test case design. Whether the
test will show the proper/improper execution of the application in order to fulfill
requirements.
● In TDD, you achieve 100% coverage test. Every single line of code is tested,
unlike traditional testing.
● The combination of both traditional testing and TDD leads to the importance of
testing the system rather than perfection of the system.
● In Agile Modeling (AM), you should “test with a purpose”. You should know
why you are testing something and what level its need to be tested.

What is acceptance TDD and Developer TDD

There are two levels of TDD

1. Acceptance TDD (ATDD): With ATDD you write a single acceptance test. This test
fulfills the requirement of the specification or satisfies the behavior of the system. After
that write just enough production/functionality code to fulfill that acceptance test.
Acceptance test focuses on the overall behavior of the system. ATDD also was known
as Behavioral Driven Development (BDD).
2. Developer TDD: With Developer TDD you write single developer test i.e. unit test and
then just enough production code to fulfill that test. The unit test focuses on every small
functionality of the system. Developer TDD is simply called as TDD.The main goal of
ATDD and TDD is to specify detailed, executable requirements for your solution on a just
in time (JIT) basis. JIT means taking only those requirements in consideration that are
needed in the system. So increase efficiency.
DevOps

REPL-driven development
REPL-driven development (Read-Eval-Print Loop) is an interactive programming approach that
allows developers to execute code snippets and see their results immediately. This enables
developers to test their code quickly and iteratively, and helps them to understand the behavior of
their code as they work.
In a REPL environment, developers can type in code snippets, and the environment will
immediately evaluate the code and return the results. This allows developers to test small bits of
code and quickly see the results, without having to create a full-fledged application.
REPL-driven development is commonly used in dynamic programming languages such as Python,
JavaScript, and Ruby. Some popular REPL environments include the Python REPL, Node.js
REPL, and IRB (Interactive Ruby).

Benefits of REPL-driven development include:

DevOps

Increased efficiency: The immediate feedback provided by a REPL environment allows developers
to test and modify their code quickly, without having to run a full-fledged application.

Improved understanding: By being able to see the results of code snippets immediately, developers
can better understand how the code works and identify any issues early on.
Increased collaboration: REPL-driven development makes it easy for developers to share code
snippets and collaborate on projects, as they can demonstrate the behavior of the code quickly and
easily.
Overall, REPL-driven development is a useful tool for developers looking to improve their
workflow and increase their understanding of their code. By providing an interactive environment
for testing and exploring code, REPL-driven development can help developers to be more
productive and efficient.

Deployment of the system:

In DevOps, deployment systems are responsible for automating the release of software updates
and applications from development to production. Some popular deployment systems include:

Jenkins: an open-source automation server that provides plugins to support building, deploying,
and automating any project.
Ansible: an open-source platform that provides a simple way to automate software provisioning,
configuration management, and application deployment.
Docker: a platform that enables developers to create, deploy, and run applications in containers.
Kubernetes: an open-source system for automating deployment, scaling, and management of
containerized applications.
AWS Code Deploy: a fully managed deployment service that automates software deployments to
a variety of compute services such as Amazon EC2, AWS Fargate, and on-premises servers.

Azure DevOps: a Microsoft product that provides an end-to-end DevOps solution for developing,
delivering, and deploying applications on multiple platforms.

Virtualization stacks
In DevOps, virtualization refers to the creation of virtual machines, containers, or environments
that allow multiple operating systems to run on a single physical machine. The following are some
of the commonly used virtualization stacks in DevOps:
Docker: An open-source platform for automating the deployment, scaling, and management of
containerized applications.
DevOps

Kubernetes: An open-source platform for automating the deployment, scaling, and management
of containerized applications, commonly used in conjunction with Docker.

VirtualBox: An open-source virtualization software that allows multiple operating systems to run
on a single physical machine.
VMware: A commercial virtualization software that provides a comprehensive suite of tools for
virtualization, cloud computing, and network and security management.

Hyper-V: Microsoft's hypervisor technology that enables virtualization on Windows-based

systems.
These virtualization stacks play a crucial role in DevOps by allowing developers to build, test, and
deploy applications in isolated, consistent environments, while reducing the costs and complexities
associated with physical infrastructure.

code execution at the client

In DevOps, code execution at the client refers to the process of executing code or scripts on client
devices or machines. This can be accomplished in several ways, including:
Client-side scripting languages: JavaScript, HTML, and CSS are commonly used client-side
scripting languages that run in a web browser and allow developers to create dynamic, interactive
web pages.

Remote execution tools: Tools such as SSH, Telnet, or Remote Desktop Protocol (RDP) allow
developers to remotely execute commands and scripts on client devices.
Configuration management tools: Tools such as Ansible, Puppet, or Chef use agent-based or
agentless architectures to manage and configure client devices, allowing developers to execute
code and scripts remotely.
Mobile apps: Mobile applications can also run code on client devices, allowing developers to
create dynamic, interactive experiences for users.

These methods are used in DevOps to automate various tasks, such as application deployment,
software updates, or system configuration, on client devices. By executing code on the client side,
DevOps teams can improve the speed, reliability, and security of their software delivery process.

Puppet master and agents:

Puppet Architecture

Puppet uses master-slave or client-server architecture. Puppet client and server interconnected by
SSL, which is a secure socket layer. It is a model-driven system.
DevOps

Here, the client is referred to as a Puppet agent/slave/node, and the server is referred to as a Puppet
master.

Let's see the components of Puppet architecture:

Puppet Master

Puppet master handles all the configuration related process in the form of puppet codes. It is a
Linux based system in which puppet master software is installed. The puppet master must be in
Linux. It uses the puppet agent to apply the configuration to nodes.

This is the place where SSL certificates are checked and marked.

Puppet Slave or Agent

Puppet agents are the real working systems and used by the Client. It is installed on the client
machine and maintained and managed by the puppet master. They have a puppet agent service
running inside them.

The agent machine can be configured on any operating system such as Windows, Linux, Solaris,
or Mac OS.

Config Repository

Config repository is the storage area where all the servers and nodes related configurations are
stored, and we can pull these configurations as per requirements.
DevOps

Facts

Facts are the key-value data pair. It contains information about the node or the master machine. It
represents a puppet client states such as operating system, network interface, IP address, uptime,
and whether the client machine is virtual or not.

These facts are used for determining the present state of any agent. Changes on any target machine
are made based on facts. Puppet's facts are predefined and customized.

Catalog

The entire configuration and manifest files that are written in Puppet are changed into a compiled
format. This compiled format is known as a catalog, and then we can apply this catalog to the
target machine.

The above image performs the following functions:

o First of all, an agent node sends facts to the master or server and requests for a catalog.
o The master or server compiles and returns the catalog of a node with the help of some information
accessed by the master.
o Then the agent applies the catalog to the node by checking every resource mentioned in the catalog.
If it identifies resources that are not in their desired state, then makes the necessary adjustments to
fix them. Or, it determines in no-op mode, the adjustments would be required to reconcile the
catalog.
o And finally, the agent sends a report back to the master.

Puppet Master-Slave Communication

Puppet master-slave communicates via a secure encrypted channel through the SSL (Secure Socket
Layer). Let's see the below diagram to understand the communication between the master and
slave with this channel:
DevOps

The above diagram depicts the following:

o Puppet slave requests for Puppet Master Certificate.

o Puppet master sends the Master Certificate to the puppet slave in response to the client request.
o Puppet master requests to the Puppet slave for the slave certificate.
o Puppet slave sends the requested slave certificate to the puppet master.
o Puppet slave sends a request for data to the puppet master.
o Finally, the master sends the data to the puppet slave as per the request.

Puppet Blocks
Puppet provides the flexibility to integrate Reports with third-party tools using Puppet APIs.

Four types of Puppet building blocks are

1. Resources
2. Classes
3. Manifest
4. Modules

Puppet Resources:
Puppet Resources are the building blocks of Puppet.

Resources are the inbuilt functions that run at the back end to perform the required operations in
puppet.
DevOps

Puppet Classes:
A combination of different resources can be grouped together into a single unit called class.

Puppet Manifest:
Manifest is a directory containing puppet DSL files. Those files have a .pp extension. The .pp
extension stands for puppet program. The puppet code consists of definitions or declarations of
Puppet Classes.

Puppet Modules:
Modules are a collection of files and directories such as Manifests, Class definitions. They are the
re-usable and sharable units in Puppet.

For example, the MySQL module to install and configure MySQL or the Jenkins module to
manage Jenkins, etc..

Ansible:
Ansible is simple open source IT engine which automates application deployment, intra service
orchestration, cloud provisioning and many other IT tools.
Ansible is easy to deploy because it does not use any agents or custom security infrastructure.
DevOps

Ansible uses playbook to describe automation jobs, and playbook uses very simple language
i.e. YAML (It’s a human-readable data serialization language & is commonly used for
configuration files, but could be used in many applications where data is being stored)which is
very easy for humans to understand, read and write. Hence the advantage is that even the IT
infrastructure support guys can read and understand the playbook and debug if needed (YAML –
It is in human readable form).
Ansible is designed for multi-tier deployment. Ansible does not manage one system at time, it
models IT infrastructure by describing all of your systems are interrelated. Ansible is completely
agentless which means Ansible works by connecting your nodes through ssh(by default). But if
you want other method for connection like Kerberos, Ansible gives that option to you.
After connecting to your nodes, Ansible pushes small programs called as “Ansible Modules”.
Ansible runs that modules on your nodes and removes them when finished. Ansible manages your
inventory in simple text files (These are the hosts file). Ansible uses the hosts file where one can
group the hosts and can control the actions on a specific group in the playbooks.
Sample Hosts File
This is the content of hosts file −
#File name: hosts
#Description: Inventory file for your application. Defines machine type abc
node to deploy specific artifacts
# Defines machine type def node to upload
metadata.

[abc-node]
#server1 ansible_host = <target machine for DU deployment> ansible_user = <Ansible
user> ansible_connection = ssh
server1 ansible_host = <your host name> ansible_user = <your unix user>
ansible_connection = ssh

[def-node]
#server2 ansible_host = <target machine for artifact upload>
ansible_user = <Ansible user> ansible_connection = ssh
server2 ansible_host = <host> ansible_user = <user> ansible_connection = ssh
DevOps

What is Configuration Management

Configuration management in terms of Ansible means that it maintains configuration of the

product performance by keeping a record and updating detailed information which describes an
enterprise’s hardware and software.
Such information typically includes the exact versions and updates that have been applied to
installed software packages and the locations and network addresses of hardware devices. For e.g.
If you want to install the new version of WebLogic/WebSphere server on all of the machines
present in your enterprise, it is not feasible for you to manually go and update each and every
machine.
You can install WebLogic/WebSphere in one go on all of your machines with Ansible playbooks
and inventory written in the most simple way. All you have to do is list out the IP addresses of
your nodes in the inventory and write a playbook to install WebLogic/WebSphere. Run the
playbook from your control machine & it will be installed on all your nodes.

Ansible Workflow

Ansible works by connecting to your nodes and pushing out a small program called Ansible
modules to them. Then Ansible executed these modules and removed them after finished. The
library of modules can reside on any machine, and there are no
daemons, servers, or databases required.

In the above image, the Management Node is the controlling node that controls the entire
execution of the playbook. The inventory file provides the list of hosts where the Ansible modules
DevOps

need to be run. The Management Node makes an SSH connection and executes the small
modules on the host's machine and install the software.

Ansible removes the modules once those are installed so expertly. It connects to the host machine
executes the instructions, and if it is successfully installed, then remove that code in which one
was copied on the host machine.

Terms used in Ansible

Terms Explanation

Ansible It is a machine where Ansible is installed and from which all tasks and playbooks
Server will be executed.

Modules The module is a command or set of similar commands which is executed on the
client-side.

Task A task is a section which consists of a single procedure to be completed.

Role It is a way of organizing tasks and related files to be later called in a playbook.

Fact The information fetched from the client system from the global variables with
the gather facts operation.

Inventory A file containing the data regarding the Ansible client-server.

Play It is the execution of the playbook.

Handler The task is called only if a notifier is present.

Notifier The section attributed to a task which calls a handler if the output is changed.

Tag It is a name set to a task that can be used later on to issue just that specific task
or group of jobs.
DevOps

Ansible Architecture

The Ansible orchestration engine interacts with a user who is writing the Ansible playbook to
execute the Ansible orchestration and interact along with the services of private or public cloud
and configuration management database. You can show in the below diagram, such as:

Inventory

Inventory is lists of nodes or hosts having their IP addresses, databases, servers, etc. which are
need to be managed.

API's

The Ansible API's works as the transport for the public or private cloud services.

Modules

Ansible connected the nodes and spread out the Ansible modules programs. Ansible executes the
modules and removed after finished. These modules can reside on any machine; no database or
servers are required here. You can work with the chose text editor or a terminal or version control
system to keep track of the changes in the content.
DevOps

Plugins

Plugins is a piece of code that expends the core functionality of Ansible. There are many useful
plugins, and you also can write your own.

Playbooks

Playbooks consist of your written code, and they are written in YAML format, which describes
the tasks and executes through the Ansible. Also, you can launch the tasks synchronously and
asynchronously with playbooks.

Hosts

In the Ansible architecture, hosts are the node systems, which are automated by Ansible, and any
machine such as RedHat, Linux, Windows, etc.

Networking

Ansible is used to automate different networks, and it uses the simple, secure, and powerful
agentless automation framework for IT operations and development. It uses a type of data model
which separated from the Ansible automation engine that spans the different hardware quite easily.

Cloud

A cloud is a network of remote servers on which you can store, manage, and process the data.
These servers are hosted on the internet and storing the data remotely rather than the local server.
It just launches the resources and instances on the cloud, connect them to the servers, and you have
good knowledge of operating your tasks remotely.

CMDB

CMDB is a type of repository which acts as a data warehouse for the IT installations.

Puppet Components

Following are the key components of Puppet:

o Manifests
o Module
o Resource
o Factor
DevOps

o M-collective
o Catalogs
o Class
o Nodes

Let's understand these components in detail:

Manifests

Puppet Master contains the Puppet Slave's configuration details, and these are written in Puppet's
native language.

Manifest is nothing but the files specifying the configuration details for Puppet slave. The
extension for manifest files is .pp, which means Puppet Policy. These files consist of puppet scripts
describing the configuration for the slave.

Module

The puppet module is a set of manifests and data. Here data is file, facts, or templates. The module
follows a specific directory structure. These modules allow the puppet program to split into
multiple manifests. Modules are simply self-contained bundles of data or code.

Let's understand the module by the following image:

Resource

Resources are a basic unit of system configuration modeling. These are the predefined functions
that run at the backend to perform the necessary operations in the puppet.

Each puppet resource defines certain elements of the system, such as some particular service or
package.
DevOps

Factor

The factor collects facts or important information about the puppet slave. Facts are the key-value
data pair. It contains information about the node or the master machine. It represents a puppet
client states such as operating system, network interface, IP address, uptime, and whether the client
machine is virtual or not.

These facts are used for determining the present state of any agent. Changes on any target machine
are made based on facts. Puppet's facts are predefined and customized.

M-Collective

M-collective is a framework that enables parallel execution of several jobs on multiple Slaves.
This framework performs several functions, such as:

o This is used to interact with clusters of puppet slaves; they can be in small groups or very
large deployments.
o To transmit demands, use a broadcast model. All Slaves receive all requests at the same
time, requests have filters attached, and only Slaves matching the filter can act on requests.
o This is used to call remote slaves with the help of simple command-line tools.
o This is used to write custom reports about your infrastructure.

Catalogs

The entire configuration and manifest files that are written in Puppet are changed into a compiled
format. This compiled format is known as a catalog, and then we can apply this catalog to the
target machine.

All the required states of slave resources are described in the catalog.

Class

Like other programming languages, the puppet also supports a class to organize the code in a better
way. Puppet class is a collection of various resources that are grouped into a single unit.

Nodes

The nodes are the location where the puppet slaves are installed used to manage all the clients and
servers.
DevOps

Deployment tools
Chef
Chef is an open source technology developed by Opscode. Adam Jacob, co-founder of Opscode is
known as the founder of Chef. This technology uses Ruby encoding to develop basic building
blocks like recipe and cookbooks. Chef is used in infrastructure automation and helps in reducing
manual and repetitive tasks for infrastructure management.
Chef have got its own convention for different building blocks, which are required to manage and
automate infrastructure.
Why Chef?
Chef is a configuration management technology used to automate the infrastructure provisioning.
It is developed on the basis of Ruby DSL language. It is used to streamline the task of configuration
and managing the company’s server. It has the capability to get integrated with any of the cloud
technology.
In DevOps, we use Chef to deploy and manage servers and applications in-house and on the cloud.
Features of Chef
Following are the most prominent features of Chef −
● Chef uses popular Ruby language to create a domain-specific language.
● Chef does not make assumptions on the current status of a node. It uses its mechanisms to
get the current status of machine.
● Chef is ideal for deploying and managing the cloud server, storage, and software.
Advantages of Chef
Chef offers the following advantages −
● Lower barrier for entry − As Chef uses native Ruby language for configuration, a
standard configuration language it can be easily picked up by anyone having some
development experience.
● Excellent integration with cloud − Using the knife utility, it can be easily integrated with
any of the cloud technologies. It is the best tool for an organization that wishes to distribute
its infrastructure on multi-cloud environment.
Disadvantages of Chef
Some of the major drawbacks of Chef are as follows −
● One of the huge disadvantages of Chef is the way cookbooks are controlled. It needs
constant babying so that people who are working should not mess up with others
cookbooks.
DevOps

● Only Chef solo is available.

● In the current situation, it is only a good fit for AWS cloud.
● It is not very easy to learn if the person is not familiar with Ruby.
● Documentation is still lacking.
Key Building Blocks of Chef
Recipe
It can be defined as a collection of attributes which are used to manage the infrastructure. These
attributes which are present in the recipe are used to change the existing state or setting a particular
infrastructure node. They are loaded during Chef client run and comparted with the existing
attribute of the node (machine). It then gets to the status which is defined in the node resource of
the recipe. It is the main workhorse of the cookbook.
Cookbook
A cookbook is a collection of recipes. They are the basic building blocks which get uploaded to
Chef server. When Chef run takes place, it ensures that the recipes present inside it gets a given
infrastructure to the desired state as listed in the recipe.
Resource
It is the basic component of a recipe used to manage the infrastructure with different kind of states.
There can be multiple resources in a recipe, which will help in configuring and managing the
infrastructure. For example −
● package − Manages the packages on a node
● service − Manages the services on a node
● user − Manages the users on the node
● group − Manages groups
● template − Manages the files with embedded Ruby template
● cookbook_file − Transfers the files from the files subdirectory in the cookbook to a location
on the node
● file − Manages the contents of a file on the node
● directory − Manages the directories on the node
● execute − Executes a command on the node
● cron − Edits an existing cron file on the node

Chef - Architecture

● Chef works on a three-tier client server model wherein the working units such as cookbooks
are developed on the Chef workstation. From the command line utilities such as knife, they
are uploaded to the Chef server and all the nodes which are present in the architecture are
registered with the Chef server.
DevOps

● In order to get the working Chef infrastructure in place, we need to set up multiple things
in sequence.
● In the above setup, we have the following components.
● Chef Workstation
● This is the location where all the configurations are developed. Chef workstation is installed
on the local machine. Detailed configuration structure is discussed in the later chapters of
this tutorial.
● Chef Server
● This works as a centralized working unit of Chef setup, where all the configuration files are
uploaded post development. There are different kinds of Chef server, some are hosted Chef
server whereas some are built-in premise.
● Chef Nodes
● They are the actual machines which are going to be managed by the Chef server. All the
nodes can have different kinds of setup as per requirement. Chef client is the key component
of all the nodes, which helps in setting up the communication between the Chef server and
Chef node. The other components of Chef node is Ohai, which helps in getting the current
state of any node at a given point of time.

Salt Stack

Salt Stack is an open-source configuration management software and remote execution engine.
Salt is a command-line tool. While written in Python, SaltStack configuration management is
language agnostic and simple. Salt platform uses the push model for executing commands via the
DevOps

SSH protocol. The default configuration system is YAML and Jinja templates. Salt is primarily
competing with Puppet, Chef and Ansible.
Salt provides many features when compared to other competing tools. Some of these important
features are listed below.
● Fault tolerance − Salt minions can connect to multiple masters at one time by configuring
the master configuration parameter as a YAML list of all the available masters. Any master
can direct commands to the Salt infrastructure.
● Flexible − The entire management approach of Salt is very flexible. It can be implemented
to follow the most popular systems management models such as Agent and Server, Agent-
only, Server-only or all of the above in the same environment.
● Scalable Configuration Management − SaltStack is designed to handle ten thousand
minions per master.
● Parallel Execution model − Salt can enable commands to execute remote systems in a
parallel manner.
● Python API − Salt provides a simple programming interface and it was designed to be
modular and easily extensible, to make it easy to mold to diverse applications.
● Easy to Setup − Salt is easy to setup and provides a single remote execution architecture
that can manage the diverse requirements of any number of servers.
● Language Agnostic − Salt state configuration files, templating engine or file type supports
any type of language.

Benefits of SaltStack
Being simple as well as a feature-rich system, Salt provides many benefits and they can be
summarized as below −
● Robust − Salt is powerful and robust configuration management framework and works
around tens of thousands of systems.
● Authentication − Salt manages simple SSH key pairs for authentication.
● Secure − Salt manages secure data using an encrypted protocol.
● Fast − Salt is very fast, lightweight communication bus to provide the foundation for a
remote execution engine.
● Virtual Machine Automation − The Salt Virt Cloud Controller capability is used for
automation.
● Infrastructure as data, not code − Salt provides a simple deployment, model driven
configuration management and command execution framework.
DevOps

Introduction to ZeroMQ
Salt is based on the ZeroMQ library and it is an embeddable networking library. It is lightweight
and a fast messaging library. The basic implementation is in C/C++ and native implementations
for several languages including Java and .Net is available.
ZeroMQ is a broker-less peer-peer message processing. ZeroMQ allows you to design a complex
communication system easily.
ZeroMQ comes with the following five basic patterns −
● Synchronous Request/Response − Used for sending a request and receiving subsequent
replies for each one sent.
● Asynchronous Request/Response − Requestor initiates the conversation by sending a
Request message and waits for a Response message. Provider waits for the incoming
Request messages and replies with the Response messages.
● Publish/Subscribe − Used for distributing data from a single process (e.g. publisher) to
multiple recipients (e.g. subscribers).
● Push/Pull − Used for distributing data to connected nodes.
● Exclusive Pair − Used for connecting two peers together, forming a pair.

ZeroMQ is a highly flexible networking tool for exchanging messages among clusters, cloud and
other multi system environments. ZeroMQ is the default transport library presented in
SaltStack.

SaltStack – Architecture

The architecture of SaltStack is designed to work with any number of servers, from local network
systems to other deployments across different data centers. Architecture is a simple server/client
model with the needed functionality built into a single set of daemons.
Take a look at the following illustration. It shows the different components of SaltStack
architecture.
DevOps

● SaltMaster − SaltMaster is the master daemon. A SaltMaster is used to send commands

and configurations to the Salt slaves. A single master can manage multiple masters.
● SaltMinions − SaltMinion is the slave daemon. A Salt minion receives commands and
configuration from the SaltMaster.
● Execution − Modules and Adhoc commands executed from the command line against one
or more minions. It performs Real-time Monitoring.
● Formulas − Formulas are pre-written Salt States. They are as open-ended as Salt States
themselves and can be used for tasks such as installing a package, configuring and starting
a service, setting up users or permissions and many other common tasks.
● Grains − Grains is an interface that provides information specific to a minion. The
information available through the grains interface is static. Grains get loaded when the Salt
minion starts. This means that the information in grains is unchanging. Therefore, grains
information could be about the running kernel or the operating system. It is case insensitive.
● Pillar − A pillar is an interface that generates and stores highly sensitive data specific to a
particular minion, such as cryptographic keys and passwords. It stores data in a key/value
pair and the data is managed in a similar way as the Salt State Tree.
● Top File − Matches Salt states and pillar data to Salt minions.
● Runners − It is a module located inside the SaltMaster and performs tasks such as job
status, connection status, read data from external APIs, query connected salt minions and
more.
● Returners − Returns data from Salt minions to another system.
● Reactor − It is responsible for triggering reactions when events occur in your SaltStack
environment.
DevOps

● SaltCloud − Salt Cloud provides a powerful interface to interact with cloud hosts.
● SaltSSH − Run Salt commands over SSH on systems without using Salt minion.
Docker
Docker is a container management service. The keywords of Docker are develop,
ship and run anywhere. The whole idea of Docker is for developers to easily develop applications,
ship them into containers which can then be deployed anywhere.
The initial release of Docker was in March 2013 and since then, it has become the buzzword for
modern world development, especially in the face of Agile-based projects.
Features of Docker
● Docker has the ability to reduce the size of development by providing a smaller footprint
of the operating system via containers.
● With containers, it becomes easier for teams across different units, such as development,
QA and Operations to work seamlessly across applications.
● You can deploy Docker containers anywhere, on any physical and virtual machines and
even on the cloud.
● Since Docker containers are pretty lightweight, they are very easily scalable.
Components of Docker
Docker has the following components
● Docker for Mac − It allows one to run Docker containers on the Mac OS.
● Docker for Linux − It allows one to run Docker containers on the Linux OS.
● Docker for Windows − It allows one to run Docker containers on the Windows OS.
● Docker Engine − It is used for building Docker images and creating Docker containers.
● Docker Hub − This is the registry which is used to host various Docker images.
● Docker Compose − This is used to define applications using multiple Docker containers.

Docker architecture
● Docker uses a client-server architecture. The Docker client talks to the Docker daemon,
which does the heavy lifting of building, running, and distributing your Docker containers.
The Docker client and daemon can run on the same system, or you can connect a Docker
client to a remote Docker daemon. The Docker client and daemon communicate using a
REST API, over UNIX sockets or a network interface. Another Docker client is Docker
Compose, that lets you work with applications consisting of a set of containers.
DevOps

The Docker daemon

The Docker daemon (dockerd) listens for Docker API requests and manages Docker objects such
as images, containers, networks, and volumes. A daemon can also communicate with other
daemons to manage Docker services.

The Docker client

The Docker client (docker) is the primary way that many Docker users interact with Docker. When
you use commands such as docker run, the client sends these commands to dockerd, which carries
them out. The docker command uses the Docker API. The Docker client can communicate with
more than one daemon.

Docker Desktop
Docker Desktop is an easy-to-install application for your Mac, Windows or Linux environment
that enables you to build and share containerized applications and microservices. Docker Desktop
includes the Docker daemon (dockerd), the Docker client (docker), Docker Compose, Docker
Content Trust, Kubernetes, and Credential Helper. For more information, see Docker Desktop.

Docker registries
A Docker registry stores Docker images. Docker Hub is a public registry that anyone can use, and
Docker is configured to look for images on Docker Hub by default. You can even run your own
private registry.
DevOps

When you use the docker pull or docker run commands, the required images are pulled from your
configured registry. When you use the docker push command, your image is pushed to your
configured registry.

Docker objects
When you use Docker, you are creating and using images, containers, networks, volumes, plugins,
and other objects. This section is a brief overview of some of those objects.

Images
An image is a read-only template with instructions for creating a Docker container. Often, an
image is based on another image, with some additional customization. For example, you may build
an image which is based on the ubuntu image, but installs the Apache web server and your
application, as well as the configuration details needed to make your application run.
You might create your own images or you might only use those created by others and published
in a registry. To build your own image, you create a Dockerfile with a simple syntax for defining
the steps needed to create the image and run it. Each instruction in a Dockerfile creates a layer in
the image. When you change the Dockerfile and rebuild the image, only those layers which have
changed are rebuilt. This is part of what makes images so lightweight, small, and fast, when
compared to other virtualization technologies.

Containers
A container is a runnable instance of an image. You can create, start, stop, move, or delete a
container using the Docker API or CLI. You can connect a container to one or more networks,
attach storage to it, or even create a new image based on its current state.

By default, a container is relatively well isolated from other containers and its host machine. You
can control how isolated a container’s network, storage, or other underlying subsystems are from
other containers or from the host machine.

A container is defined by its image as well as any configuration options you provide to it when
you create or start it. When a container is removed, any changes to its state that are not stored in
persistent storage disappear.

Example docker run command

The following command runs an ubuntu container, attaches interactively to your local command-
line session, and runs /bin/bash.
$ docker run -i -t ubuntu /bin/bash
When you run this command, the following happens (assuming you are using the default registry
configuration):
DevOps

1. If you do not have the ubuntu image locally, Docker pulls it from your configured registry,
as though you had run docker pull ubuntu manually.
2. Docker creates a new container, as though you had run a docker container create command
manually.
3. Docker allocates a read-write filesystem to the container, as its final layer. This allows a
running container to create or modify files and directories in its local filesystem.

4. Docker creates a network interface to connect the container to the default network, since
you did not specify any networking options. This includes assigning an IP address to the
container. By default, containers can connect to external networks using the host machine’s
network connection.

5. Docker starts the container and executes /bin/bash. Because the container is running
interactively and attached to your terminal (due to the -i and -t flags), you can provide input
using your keyboard while the output is logged to your terminal.
6. When you type exit to terminate the /bin/bash command, the container stops but is not
removed. You can start it again or remove it.

The underlying technology

Docker is written in the Go programming language and takes advantage of several features of the
Linux kernel to deliver its functionality. Docker uses a technology called namespaces to provide
the isolated workspace called the container. When you run a container, Docker creates a set
of namespaces for that container.
These namespaces provide a layer of isolation. Each aspect of a container runs in a separate
namespace and its access is limited to that namespace.