CI - CD DevOps Pipeline Project
CI - CD DevOps Pipeline Project
CI/CD
DEVOPS PIPELINE PROJECT
Deployment of Java Application on Kubernetes
Introduction
In the rapidly evolving landscape of software development, the adoption of DevOps practices
has become imperative for organizations striving for agility, efficiency, and quality in their
software delivery processes. The project at hand focuses on the implementation of a robust
DevOps Continuous Integration/Continuous Deployment (CI/CD) pipeline, orchestrated by
Jenkins, to streamline the development, testing, and deployment phases of a software product.
Architecture
SHUBHAM MUKHERJEE
DevOps Pipeline
Tools Used
SHUBHAM MUKHERJEE
DevOps Pipeline
SEGMENT 1 :
1.Setting up Virtual Machines on AWS
To establish the infrastructure required for the DevOps tools setup, virtual
machines were provisioned on the Amazon Web Services (AWS) platform. Each
virtual machine served a specific purpose in the CI/CD pipeline. Here's an
overview of the virtual machines created for different tools:
1. Kubernetes Master Node: This virtual machine served as the master node in
the Kubernetes cluster. It was responsible for managing the cluster's state,
scheduling applications, and coordinating communication between cluster nodes.
5. Jenkins Server: A virtual machine was allocated for the Jenkins server, which
served as the central hub for orchestrating the CI/CD pipeline. Jenkins
coordinated the execution of pipeline stages, triggered builds, and integrated
with other DevOps tools for seamless automation.
SHUBHAM MUKHERJEE
DevOps Pipeline
Each virtual machine was configured with the necessary resources, including CPU,
memory, and storage, to support the respective tool's functionalities and
accommodate the workload demands of the CI/CD pipeline. Additionally, security
measures such as access controls, network configurations, and encryption were
implemented to safeguard the virtualized infrastructure and data integrity.
EC2 Instances :
Security Group:
SHUBHAM MUKHERJEE
DevOps Pipeline
AWS Setup
Make sure your all instance are in same Security group.
Expose port 6443 in the Security group, so that worker nodes can join the
cluster.
SHUBHAM MUKHERJEE
DevOps Pipeline
SHUBHAM MUKHERJEE
DevOps Pipeline
mkdir -p "$HOME"/.kube
sudo cp -i /etc/kubernetes/admin.conf "$HOME"/.kube/config
sudo chown "$(id -u)":"$(id -g)" "$HOME"/.kube/config
SHUBHAM MUKHERJEE
DevOps Pipeline
SHUBHAM MUKHERJEE
DevOps Pipeline
Save this script in a file, for example, install_jenkins.sh, and make it executable
using:
chmod +x install_jenkins.sh
This script will automate the installation process of OpenJDK 17 JRE Headless
and Jenkins.
KUBECTL
SHUBHAM MUKHERJEE
DevOps Pipeline
#!/bin/bash
# Update package manager repositories
sudo apt-get update
# Install necessary dependencies
sudo apt-get install -y ca-certificates curl
# Create directory for Docker GPG key
sudo install -m 0755 -d /etc/apt/keyrings
# Download Docker's GPG key
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o
/etc/apt/keyrings/docker.asc
# Ensure proper permissions for the key
sudo chmod a+r /etc/apt/keyrings/docker.asc
# Add Docker repository to Apt sources
echo "deb [arch=$(dpkg --print-architecture) signed
by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
# Update package manager repositories
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin
docker-compose-plugin
Save this script in a file, for example, install_docker.sh, and make it executable
using:
chmod +x install_docker.sh
SHUBHAM MUKHERJEE
DevOps Pipeline
5. SetUp Nexus
Execute these commands on Nexues VM
#!/bin/bash
# Update package manager repositories
sudo apt-get update
# Install necessary dependencies
sudo apt-get install -y ca-certificates curl
# Create directory for Docker GPG key
sudo install -m 0755 -d /etc/apt/keyrings
# Download Docker's GPG key
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o
/etc/apt/keyrings/docker.asc
# Ensure proper permissions for the key
sudo chmod a+r /etc/apt/keyrings/docker.asc
# Add Docker repository to Apt sources
echo "deb [arch=$(dpkg --print-architecture) signed
by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
# Update package manager repositories
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin
docker-compose-plugin
Save this script in a file, for example, install_docker.sh, and make it executable
using:
chmod +x install_docker.sh
SHUBHAM MUKHERJEE
DevOps Pipeline
SHUBHAM MUKHERJEE
DevOps Pipeline
6. SetUp SonarQube
Save this script in a file, for example, install_docker.sh, and make it executable
using:
chmod +x install_docker.sh
SHUBHAM MUKHERJEE
DevOps Pipeline
SHUBHAM MUKHERJEE
DevOps Pipeline
SHUBHAM MUKHERJEE
DevOps Pipeline
SHUBHAM MUKHERJEE
DevOps Pipeline
SEGMENT-3 | CICD
Install below Plugins in Jenkins
1. Eclipse Temurin Installer:
o This plugin enables Jenkins to automatically install and configure the Eclipse
Temurin JDK (formerly known as AdoptOpenJDK).
o To install, go to Jenkins dashboard -> Manage Jenkins -> Manage Plugins ->
Available tab.
o Search for "Eclipse Temurin Installer" and select it.
o Click on the "Install without restart" button.
4. SonarQube Scanner:
o SonarQube is a code quality and security analysis tool.
o This plugin integrates Jenkins with SonarQube by providing a scanner that
analyzes code during builds.
o You can install it from the Jenkins plugin manager as described above.
5. Kubernetes CLI:
o This plugin allows Jenkins to interact with Kubernetes clusters using the
Kubernetes command-line tool (kubectl).
o It's useful for tasks like deploying applications to Kubernetes from Jenkins
jobs.
SHUBHAM MUKHERJEE
DevOps Pipeline
6. Kubernetes:
o This plugin integrates Jenkins with Kubernetes by allowing Jenkins agents to
run as pods within a Kubernetes cluster.
o It provides dynamic scaling and resource optimization capabilities for Jenkins
builds.
o Install it from the Jenkins plugin manager.
7. Docker:
o This plugin allows Jenkins to interact with Docker, enabling Docker builds and
integration with Docker registries.
o You can use it to build Docker images, run Docker containers, and push/pull
images from Docker registries.
o Install it from the plugin manager.
After installing these plugins, you may need to configure them according to your
specific environment and requirements. This typically involves setting up
credentials, configuring paths, and specifying options in Jenkins global
configuration or individual job configurations. Each plugin usually comes with its
own set of documentation to guide you through the configuration process.
SHUBHAM MUKHERJEE
DevOps Pipeline
Jenkins Pipeline
Create a new Pipeline job .
pipeline {
agent any
environment {
SCANNER_HOME = tool 'sonar-scanner'
}
tools {
jdk 'jdk17'
maven 'maven3'
}
stages {
stage('Git Checkout') {
steps {
git branch: 'main', credentialsId: 'git-cred', url: 'https://github.com/Shubham-
Stunner/BoardGame.git'
}
}
stage('Compile') {
steps {
sh "mvn compile"
}
}
stage('Test') {
steps {
sh "mvn test"
}
}
stage('SonarQube Analysis') {
steps {
withSonarQubeEnv('sonar') {
SHUBHAM MUKHERJEE
DevOps Pipeline
sh '''$SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=BoardGame -
Dsonar.projectKey=BoardGame \
-Dsonar.java.binaries=. '''
}
}
}
stage('Quality Gate') {
steps {
script {
waitForQualityGate abortPipeline: false, credentialsId: 'sonar-token'
}
}
}
stage('Build') {
steps {
sh "mvn package"
}
}
SHUBHAM MUKHERJEE
DevOps Pipeline
}
}
stage('Deploy to Kubernetes') {
steps {
withKubeConfig(caCertificate: '', clusterName: 'kubernetes', contextName: '', credentialsId: 'k8-
cred', namespace: 'webapps', restrictKubeConfigAccess: false, serverUrl: 'https://172.31.8.22:6443') {
sh "kubectl apply -f deployment-service.yaml"
sh "kubectl get pods -n webapps"
}
}
}
}
post {
always {
script {
def jobName = env.JOB_NAME
def buildNumber = env.BUILD_NUMBER
def pipelineStatus = currentBuild.result ?: 'UNKNOWN'
def bannerColor = pipelineStatus.toUpperCase() == 'SUCCESS' ? 'green' : 'red'
emailext(
subject: "${jobName} - Build ${buildNumber} - ${pipelineStatus.toUpperCase()}",
SHUBHAM MUKHERJEE
DevOps Pipeline
body: body,
to: '[email protected]',
from: '[email protected]',
replyTo: '[email protected]',
mimeType: 'text/html',
attachmentsPattern: 'trivy-image-report.html'
)
}
}
}
}
SHUBHAM MUKHERJEE
DevOps Pipeline
SEGMENT-4 | Monitoring
Prometheus
Links to download Prometheus, Node_Exporter & black Box exporter
https://prometheus.io/download/
By default Prometheus runs on Port 9090 and access it using your instance
<IP address>:9090
Grafana
Links to download Grafana https://grafana.com/grafana/download
OR
Run This code on Monitoring VM to Install Grafana
SHUBHAM MUKHERJEE
DevOps Pipeline
Configure Prometheus
scrape_configs:
- job_name: 'blackbox'
metrics_path: /probe
params:
module: [http_2xx] # Look for a HTTP 200 response.
static_configs:
- targets:
- http://prometheus.io # Target to probe with http.
- https://prometheus.io # Target to probe with https.
- http://example.com:8080 # Target to probe with http on port 8080.
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement:<IP address>:9115
Once you run the above command you will get the Id of Prometheus
then use the id and kill it
kill <ID>
SHUBHAM MUKHERJEE
DevOps Pipeline
Results :
JENKINS PIPELINE
PROMETHEUS
SHUBHAM MUKHERJEE
DevOps Pipeline
BLACKBOX
GRAFANA
SHUBHAM MUKHERJEE
DevOps Pipeline
APPLICATION
SHUBHAM MUKHERJEE
DevOps Pipeline
Conclusion
In conclusion, the successful implementation of the DevOps CI/CD pipeline project
marks a significant milestone in enhancing the efficiency, reliability, and quality of
software delivery processes. By automating key aspects of the software
development lifecycle, including compilation, testing, deployment, and
monitoring, the project has enabled rapid and consistent delivery of software
releases, contributing to improved time-to-market and customer satisfaction.
Acknowledgment of Contributions:
I would like to extend my gratitude to DevOps shack for helping me achieving my
goals and objectives.
Final Thoughts
Looking ahead, the project's impact extends beyond its immediate benefits,
paving the way for continuous improvement and innovation in software
development practices. By embracing DevOps principles and leveraging cutting-
edge tools and technologies, we have laid a solid foundation for future projects to
build upon. The scalability, flexibility, and resilience of the CI/CD pipeline ensure
its adaptability to evolving requirements and technological advancements,
positioning our organization for long-term success in a competitive market
landscape.
SHUBHAM MUKHERJEE
DevOps Pipeline
References
1. Jenkins Documentation:
[https://www.jenkins.io/doc/](https://www.jenkins.io/doc/)
2. Maven Documentation:
[https://maven.apache.org/guides/index.html](https://maven.apache.org/guides
/index.html)
3. SonarQube Documentation:
[https://docs.sonarqube.org/latest/](https://docs.sonarqube.org/latest/)
4. Trivy Documentation:
[https://github.com/aquasecurity/trivy](https://github.com/aquasecurity/trivy)
5. Nexus Repository Manager Documentation:
[https://help.sonatype.com/repomanager3](https://help.sonatype.com/repoman
ager3)
6. Docker Documentation: [https://docs.docker.com/](https://docs.docker.com/)
7. Kubernetes Documentation:
[https://kubernetes.io/docs/](https://kubernetes.io/docs/)
8. Prometheus Documentation:
[https://prometheus.io/docs/](https://prometheus.io/docs/)
9. Grafana Documentation:
[https://grafana.com/docs/](https://grafana.com/docs/)
These resources provided valuable insights, guidance, and support throughout the
project lifecycle, enabling us to achieve our goals effectively.
SHUBHAM MUKHERJEE