What is DevOps?

Before we learn about CI/CD we need to familiarize with the

fundamentals DevOps.

DevOps is a software development approach which involves

continuous development, continuous testing, continuous integration,

continuous deployment and continuous monitoring of the software

throughout its development life cycle.

DevOps Stages

What is CI CD Pipeline?

CI stands for Continuous Integration and CD stands for Continuous

Delivery and Continuous Deployment. You can think of it as a process
which is similar to a software development lifecycle.
Now let us see how does it work.
The above pipeline is a logical demonstration of how software will
move along the various phases or stages in this lifecycle, before it is
delivered to the customer or before it is live on production.

Let’s take a scenario of CI CD Pipeline. Imagine you’re going to build a

web application which is going to be deployed on live web servers. You
will have a set of developers who are responsible for writing the code
which will further go on and build the web application. Now, when this
code is committed into a version control system(such as git, svn) by the
team of developers. Next, it goes through the build phase which is the
first phase of the pipeline, where developers put in their code and then
again code goes to the version control system having a proper version
tag.

Suppose we have a Java code and it needs to be compiled before

execution. So, through the version control phase, it again goes to build
phase where it gets compiled. You get all the features of that code from
various branches of the repository, which merge them and finally use a
compiler to compile it. This whole process is called the build phase.

Testing Phase:
Once the build phase is over, then you move on to the testing phase.
In this phase, we have various kinds of testing, one of them is the unit
test (where you test the chunk/unit of software or for its sanity test).

Deploy Phase:

When the test is completed, you move on to the deploy phase, where
you deploy it into a staging or a test server. Here, you can view the code
or you can view the app in a simulator.

Auto Test Phase:

Once the code is deployed successfully, you can run another set of a
sanity test. If everything is accepted, then it can be deployed to
production.

Deploy to Production:

Meanwhile in every step, if there is some error, you can shoot a mail
back to the development team so that they can fix them. Then they will
push it into the version control system and goes back into the pipeline.

Once again if there is any error reported during testing, again the
feedback goes to the dev team where they fix it and the process re-
iterates if required.

Measure+Validate:
So, this lifecycle continues until we get a code or a product which can
be deployed in the production server where we measure and validate
the code.

We have understood CI CD Pipeline and its working, now we will move

on to understand what Jenkins is and how we can deploy the
demonstrated code using Jenkins and automate the entire process.

Awesome!, Now we can proceed to learning CI/CD.

CI/CD
The primary goal of any software project is to earn money through the
automation of the business process. The quicker you can release the
new versions to the customers, the better it is for your company. But
how to implement the release process in a fast way? Well, you could do
it manually but today we’re going to be discussing the automation of a
product releases and development process itself.

How then can we define CI/CD?

Is a set of development practices aimed at automating the building

testing and deployment of application

CI

which stands for Continuous Integration describes the processes of

the changes flows to the repository. It describes the practice of merging
all developers’ working copies to a shared mainline several times a day.
It is responsible for handling everything that involves code.Let’s take a
look at a simple schema that gives an example of team development.
Basic Team Development

A group of people can work simultaneously. But all changes are

transferred to the master branch eventually. Anyway, even such a
simple model raises a couple of questions.

1. How can we know that the code that goes to the master branch
compiles?
 2. We want the developers to write tests for the code. How can
we verify that the test coverage is not decreasing?

3. All team members should format the code with the specified
code style. How can we check the possible violations?

Of course, all of the described requirements may be validated

manually. Though this approach is quite disorganized. More than that,
it becomes harder to keep it going when the team grows.

CI was brought to automate the stated proposals.

Basic CI

The majority of CI processes can be described according to this

algorithm.

1. On each Pull Request opening (and pushing new changes as

well) Git server sends a notification to the CI server.

2. CI server clones the repository, checkouts to the source

branch (for instance, bugfix/wrong-sorting), and merges with
the master branch.

3. Then the build script is being launched. For

example, ./gradlew build.

4. If the command returns 0 code, then the build is successful.

Otherwise, it’s treated as the failed one.
 5. CI server sends the request with the build result to the Git
server.

6. If the build is successful, then the Pull Request is allowed to

merge. Otherwise, the merge is blocked.

The process guarantees that any code that goes to the master branch
does not break the further builds.

Test Coverage Checking

Let’s make the task more complicated. Suppose that we want to set the
minimum test coverage bar. So, at any moment, the coverage of
the master branch should not be lower than 50%. The Jacoco plugin can
solve the problem easily. You just need to configure it in the way to fail
the build, if the test coverage value is less than the accepted one.

The approach implementation is piece of cake. But it has a caveat. It

can only work if the plugin was configured since the project started.

Imagine that you’re working on a product that is five years old. Since
its first commit, there has been no test coverage checking. Developers
added tests randomly without any discipline. But one day you decided
to increase the number of tests. You tune the Jacoco plugin so the
minimum bar equals 60%. After a while, a developer opens a new Pull
Request. Then they suddenly realise that the test coverage is only 30%.
So, to close the task successfully it's obligatory to cover at least 30% of
the product code. As you may guess, it's almost an unresolvable issue
for the five years old project.

What if we validated only the upcoming code changes but not the
whole product? If a developer changed 200 lines within the Pull
Request, they would need to cover at least 120 of them (if the test
coverage bar equals 60%). But it wouldn't be necessary to walk through
the tons of modules that aren't part of the task. This can solve the
issue. How can we apply it to the project? Thankfully, there is a
solution.
CI with test coverage checking

Jacoco report is sent to the test coverage server.

SonarCloud is one the most popular solution.

The server keeps statistics of the previous calculations. It’s a beneficial

point to calculate the upcoming changes’ test coverage as well as the
whole code. Then the analysis result is sent to the CI server that sends
it back to the Git server.

This workflow provides an opportunity to apply the culture of

mandatory testing at any product evolution stage. Because only the
new changes are being validated.

Speaking of code style there aren’t many differences. You can

try Checkstyle plugin. It automatically fails a build that violates any of
the stated requirements. For example, the code might have an unused
import. Besides you can look at cloud services that run the code
analysis and shows the result as a bunch of charts (SonarCloud can
also do that).

CD

Continuous Delivery describes the process of the new product version

automatic deployment.it is a software engineering approach in which
the value is delivered frequently through automated deployments.CD is
responsible for handling everything that involves deployment.

Let’s put some changes to the CI schema. That’s how the CI/CD
process may look like in a real project.
CI/CD process

Firstly, the CI server is named as CI/CD server now. The thing is that
frequently both CI and CD jobs are executed with the same task
manager. So, we’re looking at this approach.

Though that’s not the rule. For example, one can delegate CI jobs
to GitLab CI and CD jobs to Jenkins.
The right part of the schema represents CI. We have discussed it
earlier. The left one pictures CD. The CD job builds the project (or
reuses the artefacts generated during the CI stage) and deploys it to the
end server.

It’s worth mentioning that server is an abstraction in our case. For

example, the deployment might proceed to the Kubernetes cluster.
So, there are might be several servers.

After the deployment stage completion e-mails are usually sent. For
instance, the CD server can notify subscribers of the succeeded or
failed deployment.

Anyway, there is an important question. When should we run CD jobs?

Triggers may vary.

1. Deploy after each Pull Request merge.

2. Deploy according to the schedule.

3. Deploy after each Pull Request merges to the particular

branch.

4. Combined option.

The first point sets the process so the CI and CD jobs always run
sequentially. This approach is rather popular within open-source
development. Semantic Release library helps to tune the project to
integrate this process transparently.

It’s important to be aware of the deploy definition. It doesn't

necessarily mean that something is being launched somewhere. If
you develop a library, then there is no launching. Instead, the
deployment process means the new library version releasing.

The second point is independent of the CI process. Because the project

is deployed according to some predefined schedule. For example, every
day at 01:00 am.

The third point is similar to the first one. Though there are differences.
Suppose that we have two primary branches in our repository.
The develop branch and the master one. The develop contains the most
relevant changes. While the second one has only releases. If we need to
deploy the master branch only, there is no need to trigger the CD job on
merge to the develop.

The last point is the aggregate of all the approaches. For instance,
the develop branch might be deployed according to the schedule to the
dev environment. And the master is deployed to the production on each
Pull Request merge.

Tools
The market offers dozens of solutions to automate CI/CD processes.
Let’s take a look at some of them.

1. Jenkins. One of the most demanded CI/CD tools in the world.

It has become so popular because of its open-source policy.
So, you don’t have to pay anything. Jenkins allows
imperatively describing build pipelines with Groovy. On the
one hand, it provides more flexibility. But on the other hand,
it requires a greater competence level.

2. GitHub Actions. The CI/CD tool is included in GitHub and

GitHub Enterprise. Unlike Jenkins, GitHub Actions provides
declarative builds with YAML configuration. Besides, the
solution has lots of integrations with different Quality
Assurances Systems (for example, SonarCube). So, the build
can be described just in a few lines of text.

3. GitLab CI. It is quite similar to GitHub Actions. Nevertheless,

it has special features. For instance, GitLab CI can point to the
particular tests that failed the build.

4. Travis CI. The cloud CI/CD service. It offers many capabilities

that require no complex configuration. For example,
encryption of data that ought to be hidden in the public
repository. Besides, the nice bonus is the Travis CI can be
applied to GitHub, GitLab, and BitBucket open-source public
projects absolutely for free.