Jenkins - Self Notes
Jenkins - Self Notes
CI/CD PIPELINE :-
A continuous integration and continuous deployment/delivery (CI/CD) pipeline is a series of steps that
must be performed in order to deliver a new version of software. CI/CD pipelines are a practice
focused on improving software delivery throughout the software development life cycle via
automation.
CI = CONTINUOUS INTEGRATION
WHAT IS JENKINS
JENKINS is a self content open source server which can be used to automate all task related to building,
testing and delivery activity.
What we will do with artifact (jar file)?/ actually artifact is deployed to the server and in DevOps
we deploy it by using JENKINS.
JENKINS can be installed even on stand alone machine with a java runtime environment
STAGES
There are five stages in CI/CD :-
As we know the complete project related source code is in Github repository; the term CONTINUOUS
DOWNLOAD is nothing but downloading the source code from Github repository to machine
(Instance) in which JENKINS is installed.
1. DEV SERVER/INSTANCE
2. QA SERVER/INSTANCE
3. PROD SERVER/INSTANCE
Server is nothing but the EC2 Instance where we install JENKINS in the Dev server and there JENKINS
perform stage one that is CONTINUOUS DOWNLOAD
Now generate artifact (jar file) for which MAVEN needs to be installed in the Dev server.
To download the source code we need to write git command and for this git should be installed.
Now we will deploy the artifact in the QA server, so we need to have tomcat and this process is known
as CONTINUOUS DEPLOYMENT.
--------------------------------------------------------------------------------------------------------------------------------------
Steps for downloading JENKINS JRE, JDK, GIT and MAVEN and log in to the JENKINS and
creating a JOB/FREE STYLE PROJECT :-
Steps for launching EC2 Instance through SSH:-
STEP 1. Go to AWS and launch a EC2 Instance (DEVELOPMENT SERVER) & select UBUNTU O/S
STEP 2. create a .pem file and add security group with all traffic ( http / ssl / https ) and launch
STEP 3. Now Select the Instance and connect via ssh client; a link will be shown : copy it
STEP 4. Now go to the folder where the .pem file is downloaded and open gitbash there
STEP 5. And now in the gitbash, paste the link we copied from the Instance
STEP 1. WHENEVER YOU INSTALL ANY SOFTWARE YOU NEED TO UPDATE APT REPOSITORY
sudo apt update
If we want to install JENKINS for the aws environment go for generic java package that is .war file so
right click on the generic java package and copy the link address and prefix it with wget (means write
wget before this command ) this is the command which helps you to download software to the link
address.
NOTE: - COPY THE LTS (LONG TERM SUPPORT ONLY)
STEP 8. Now we will log in to the JENKINS through our server means through instance.
go to the browser; paste the public ip of your instance:8080 now it will ask for the password
and then click on next
STEP 10. NOW WE HAVE TO GO TO THE GITBASH AGAIN AND THEN COPY THE PASWORD WHICH
WILL BE THERE AND PASTE
STEP 11. NOW IT WILL ASK FOR CREATE A USER SO CREATE A USER GIVE THE NAME AND THE
PASSWORD ETC EMAIL DETAILS AND LOG IN TO THE JENKINS. IN JENKINS WHATEVER ACTIVITY
WE DO WE HAVE TO CREATE A JOB
STEP 12. NOW IN THE DASHBOARD OF JENKINS YOU WILL SEE THE NEW ITEM ON THE LEFT SIDE
SO CLICK ON NEW ITEM
STEP 13. ENTER THE NAME AND CLICK ON FREE STYLE PROJECT AND OK
STEP 14. NOW MY REQUIREMENT IS TO CREATE JOB WHICH DISPLAY THE HELLO WORLD
MESSAGE. NOW GO TO BUILD TAB ADD BUILD STEP
STEP 16. EXECUTE SHELL AND NOW WRITE THE SHELL SCRIPTING LIKE echo "hello" NOW CLICK
ON SAVE AND APPLY
STEP 17. NOW GO TO THE DASHBOARD NOW YOU WILL SEE A NEW JOB CREATED OVER THERE
STEP 18. NOW CLICK ON THE PLAYBUTTON LIKE SYMBOL TO EXECUTE THE JOB WE CREATED
STEP 19. NOW IF YOU WILL SEE IN THE LAST SUCCESS NUMBER OF SUCESSFULL EXECUTION WILL
BE THERE SO CLICK ON IT AND CLICK ON THE CONSOLE RESULT. IT WILL SHOW THE CONSOLE
OUTPUT
Whenever we create the new instance means new server we have to login with a new user so prefer
to stop the server and then start again and work. For that start the server and connect with the gitbash
run the java with war
Now copy the public ip of the instance after starting instance and then:8080 and enter now enter the
password and username and login.
1. Launch an instance name as QASERVER and select UBUNTU O/S then create a keypair .pem
and select the security group of DEVSERVER
2. Now connect it through SSH client to gitbash as same as DEVSERVER
▌↓ ▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓
STEP 3 => sudo apt-get install -y tomcat10-admin (to download admin file)
STEP 4 => To check whether tomcat is running or not paste the server-ec2-public-ip:8080, if it will
show it works means its running
STEP 5 => Create a user in tomcat because minimum one user is required
When we install tomcat a xml file also created with the name "tomcat-users.xml" in this file we need
to provide the user credential
Planning to create in tomcat
cd /etc/tomcat10
STEP 11 => Whenever we do any changes in the server we need to restart the server
so to restart the server command is
STEP 12 => To perform CI/CD stages all the servers need to be in the running mode
STEP 13 => CHECK THE TOMCAT SERVER IS RUNNING OR NOT COPY THE IP OF QA SERVER AND THEN
PASTE ON GOOGLE :8080 = “it works” WILL COME MEANS DONE
STEP 14 => COPY THE IP ADDRESS OF THE DEVSERVER AND OPEN JENKINS
Now we will perform the first stage of ci cd which is continuous download
▌↓ ▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓
NOW
STEP 4=> GO TO SOURCE CODE MANAGEMENT TAB SELECT GIT AND ENTER
NOW ON THE CONSOLE YOU CAN SEE THE OUTPUT AS A SUCCESS MESSAGE
STEP 6=> NOW OPEN ONE MORE TERMINAL OF GITBASH AND COPY THE SSH AND CONNECT TO
DEVSERVER
sudo su
example: root@ip-172-31-9-218:/home/ubuntu#
STEP 9=> AND PASTE THE CONSOLE OUTPUT IN THE NEW TERMINAL AND ENTER YOU WILL SEE THE
pom.xml FILE
example: cd /root/.jenkins/workspace/meitem
STEP 10=> NOW ls TO CHECK THE DATA OF THE REPOSITORY IS CAME OR NOT. THIS IS THE OUR FIRST
STAGE WHICH IS CONTINUOUS DOWNLOAD (completed first stage)
▌↓ ▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓
STEPS:-
STEP 2 = CLICK ON BUILD TAB ADD BUILD STEPS = SELECT INVOKE TOP LEVEL MAVEN TARGET
STEP 3 = NOW YOU NEED TO WRITE THE COMMAND TO GENERATE ARTIFACT = package (WRITE THIS
IN THE GOALS)
STEP 4 = AFTER RUNNING THE GOAL PACKAGE THE WAR FILE WILL GENERATE
STEP 5 = GO TO DASHBOARD AND THEN EXECUTE AND REFRESH AND CHECK THE WAR FILE IN THE
CONSOLE RESULT
▌↓ ▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓
STEP 1 = NOW THE ARTIFACT WHICH WE GET NEED TO BE DEPLOYED IN THE TOMCAT SERVER WHICH
IS RUNNING IN THE QA SERVER
STEP 3 = SO TO INSTALL ADDITIONS PLUGGIN GO TO JENKINS - THEN MANAGE JENKINS AND MANAGE
PLUGGINs
STEP 4 = ONCE THE PLUGGIN IS INSTALLED GO TO YOUR JOB, CLICK ON CONFIGURE, GO TO POST
BUILD ACTIONS, ADD POST BUILD ACTIONS
STEP 5 = NOW SELECT = DEPLOY WAR/EAR TO A CONTAINER AND YOU HAVE TO PROVIDE YOUR FILE
NAME (**/*.war) PROVIDE THIS ONLY
STEP 6 = WRITE THE CONTEXT PATH (YOU HAVE TO REMEMBER THE CONEXT PATH NAME AS IT WILL
USE FOR THE SERVER TO CHECK OF DEPLOYMENT)
STEP 9 = IT WILL ASK FOR THE USERNAME AND PASSWORD WE HAVE TO PROVIDE THE USERNAME
WHICH WE HAVE CREATED IN THE TOMCAT SERVER (QASERVER)
STEP 10 = AND CLICK ON ADD
STEP 11 = NOW IN THE CREDENTIAL YOU NEED TO SELECT YOUR OWN CREDENTIAL AND IN THE
TOMCAT URL TYPE =http://privateip of qaserver:8080
Now my requirement is to access the application which we have deployed in to the tomcat server
▌↓ ▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓
STEP 1 = DOWNLOAD THE TESTING SCRIPT FROM GITHUB AND RUN THE TESTING SCRIPT (.war file
means artifact file). The testing script will be provided by the developer to us they will give us the
github url or will give us the file we will deploy on the GITHUB.
STEP 3 = CLICK ON NEW ITEM PROVIDE NAME, FREESTYLE, SOURCE CODE MANAGEMENT TAB CLICK
ON GIT
STEP 9 = Now build the job execute and then check in the console result and you will see the testing
success message means the testing is done successfully.
▌↓ ▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓
We will do this stage in the production server. For that we have to install the tomcat server in the
production server gitbash terminal.
FIRST = make an instance for the production server ubuntu and then connect with the gitbash
STEP 3 => sudo apt-get install -y tomcat9-admin (to download admin file)
MAKE AN CREDENTIAL IN IT =
esc :wq
STEP 4 => To check with a tomcat is running or not paste the server ec2 public ip:8080 if it will show
it works means its running
STEP 5 => Create a user in tomcat because minimum one user is required
WHEN WE INSTALL TOMCAT A XML FILE ALSO CREATED WITH THE NAME "TOMCAT-USERS.XML" IN
THIS FILE WE NEED TO PROVIDE THE USER CREDENTIAL
STEP 8 => COMMAND TO OPEN THE FILE IS => sudo vim tomcat-users.xml
STEP 9 => ADD THE TAG JUST BEFORE </tomcat-users> (JUST ABOVE THIS TAG)
STEP 11 => Whenever we do any changes in the server we need to restart the server so restart the
server command is = sudo service tomcat9 restart
STEP 12 => TO PERFORM CI/CD STAGES ALL THE SERVERS NEED TO BE IN THE RUNNING MODE
STEP 13 = check the tomcat server is running or not; copy the ip of production server and then paste
on google:8080, it works means done.
*NOW FOR CONTINUOUS DELIVERY ALL THE FOUR STAGES NEED TO BE COMBINED
▌↓ ▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓▌↓
STEP 3 => Click on add post build actions, click on build other projects
STEP 4 => In project to build select the second job that is testing job click on apply and save
STEP 5 => Now execute the first job ( Now you will see when you will execute the job then testing job
will also run, means both the development and testing has done )
STEP 6 => The artifact created need to be copied from devserver to qaserver
STEP 8 => The artifact or war file created by the development job should be passed to the testing job
so that testing job can deploy that into the tomcat in the production server
STEP 9 => To archive the file, go to development first job and click on configure
STEP 10 => Click on post build actions, add post build actions, archive the artifact in the file to archive
provide **/*.war click on apply and close
STEP 11 => Now we will copy the artifact from one job to another job so we need the pluggin called
as = copy artifact
STEP 16 => Provide the project name that is first job which we have created click on apply and save.
Now we will deploy this testing artifact on the production server. For that as we made the tomcat
server on the production gitbash terminal so now we will add the credential in the testing job
STEP 17 => Go the testing job of qaserver in jenkins click on the configure
STEP 18 => Now scroll down and click on add container select tomcat10 and click on +add
STEP 19 => Now click on jenkins and give the username and password of tomcat production server
STEP 21 => Select your made credential and then give the tomcat url http://private ip of production
instance:8080
STEP 22 => Now copy the public ip of the production instance and paste on google:8080/contextpath
NOTE = WHENEVER YOUR WORK IS DONE AND YOU WANT TO SHUT THE LAPTOP THEN STOP THE
INSTANCE AND THEN WHEN YOU WILL START YOUR WORK WILL NOT BE LOST
JENKINS
STEP 1 => To provide restricted access to the user that is creating role and assigning it. For this we
need to download the plugging called as = role based authorization strategy
STEP 2 => Now click in manage JENKINS and scroll down there you will see an option called security
click on it
STEP 3 => Now in security you will see option authorisation click on it and then click on role based
strategy and apply and save
STEP 4 => After this click on users options and make an user. Now when you will log in with the user
id password you will see the access will be denied cause the user don’t have the permissions yet
STEP 5 => Now click on manage and assign roles two types of roles : global and project based
STEP 6 => Now in the global role type the role type in the role to add option and then click on add ex.
employee
STEP 7 => Now give the permissions to the employee in the assign role option
STEP 8 => Now when you will log in ass the user id password you can see the user have the permissions
what we have assigned in it
-----
So to provide the permission we will create a role and role is assigned to the user
There are two types of roles : global level and project level
DEVELOPMENT = DEV.*
TESTING =TEST.*
Development can access the job starting from dev and tester with test
Global roles are those roles which are applied for all the activities which present in jenkins for all
projects.
STEP 1 => First create the project level role, manage and assign roles
STEP 2 => GO TO ITEM ROLES, ROLES TO ADD TO USER & PATTERN (DEV.*)
STEP 4 => User or group to add there you will assign employee to both the user
STEP 5 => In item role add both the user and assign and user one as developer and user two as testing.
Apply and save now you will see that the user one will have the development rights and tester have
the tester rights. And developer will work the projects start with the dev and the tester with test.
CREATING SLAVE MACHINE:-
When there are many number of jobs which are running in parallel, the JENKINS can go down. So to
reduce the load on the Dev-server we will create a new machine called as slave machine. Now the
complete load will not be taken to the dev machine. So the slave machine reduces the load of the
master machine (dev machine) and this master machine is nothing but the machine where JENKINS is
installed.
STEP 2 => Now launch EC2 by using ssh command and connect the gitbash through the instance as we
do
▌↓ ▌ ↓ ▌ ↓ ▌ ↓ ▌
whoami
cd /etc/ssh
STEP 10 => There you will see sshd_config file [sshd means (SECURE SHELL DAEMON
CONFIGURATION)]. We need to make some change in this file
STEP 12 => Now go to insert mode in password authentication provide yes, scroll down and you will
see this : PasswordAuthentication yes
esc :wq
STEP 15 => Now open the master machine that is devserver, open an another gitbash and connect
through ubuntu devserver
Now you can check through the ip address you have been connected to the slave machine. Now in
this scenario we get that it is asking for password, but we have to create the password less connection.
STEP 17 => For this we need to create the key in the devserver and copy the same key to the slave
machine then only the dev machine can communicate with the slave machine without password.
STEP 18 => Now close the connection between the master and slave
STEP 19 => Now to generate the key in the dev machine type
cd /etc/ssh
ssh-keygen and press enter
STEP 28 => Host Key Verification Strategy : non verifying verification strategy
Save
mkdir jenkins
STEP 32 => Now go to the JENKINS and click on myslave and launch the agent and we will see the
agent is online now
STEP 1 => create new job freestyle and add build step > execute shell >
sleep 10
STEP 2 => let’s do number of executer for the both master and slave to 1 to understand in a more
better way
For master to change the number of executer go to > manage jenkins > configure systems > no. of
executer make it 1
If we will build the job it will run on any machine like master or slave but my requirement is that this
should be done on slave only. So now we can achieve that the particular job to only run on the slave
machine and we can achieve that with the help of labels
STEP 3 => go in the job configure and there you will see an option > restrict where this project can be
run
STEP 5 => There give the label we gave in the slave machine and when we will run this job it will always
run on the slave machine.
PIPELINE-AS-A-CODE IN JENKINS
STEP 3 => For controlling the pipeline job you need to install pluggins
build pipeline pluggin and stage view (This pluggins are used for getting better gui for
controlling the job)
This is the set of pluggins which are used for implementing CI/CD from the level of code as pipeline
use the code for creating the code we will use the groovy script and this file as also called as JENKINS
file
As pipeline is implemented as code give the developer the ability to upload into the version controlling
system from where they can edit and review the script. Pipeline script supports complex real time
scenarios where we can implement condition statement, loop etc.
Script pipeline syntax
node
-----------------------
-----------------------
Node
git 'https://github.com/vrushankamare95/hello-world.git'
2. Go to sample step select git provide github link where you want to download the source code and
click generate script
git 'https://github.com/vrushankamare95/SampleWebApp.git'
node
{
git 'https://github.com/vrushankamare95/SampleWebApp.git'
------------------
------------------
sh shell script
mvn package
GENERATE PIpeLINE SCRIPT => COPY THE CODE AND PASTE IN THE CONTINUOUS BUILD
node
git 'https://github.com/vrushankamare95/SampleWebApp.git'
-------------------------
-----------------------
-----------------------
Now we need to established the password less connection between with dev and qa server
save
STEP 6 => Restart the service => sudo systemctl restart sshd
Now go to devserver
STEP 7 => Connect the dev server using gitbash and generate ssh key if not generated
ssh-keygen
STEP 9 => CREATE A CAT FILE IN THE DEVSERVER USING CAT => cat > file1
hello
ctrl d
Now we will transfer the file in the qaserver we are just copying the data from the devserver to the
qaserver
Deployment is nothing but copying the war file from devserver to qaserver
STEP 13 => Get the location of war file from logs of JENKINS (webapp.war file)
STEP 14 => Copy the war file from the logs location
EXAMPLE: /home/ubuntu/.jenkins/workspace/PIPELINE/web/target/time-tracker-web-0.5.0-
SNAPSHOT.war
STEP 15 => Copy the war file to the qaserver using scp
STEP 16 => copy the above line in paste in the jenkins scipt
STEP 18 => NOW inside the shell script paste the above code
WHEN WE BUILD WE WILL GET A FAILURE MASSAGE SAYING PERMISSION DENIED SO WE NEED TO
ASSIGN THE WRITE PERMISSION TO THE TOMCAT9
STEP 20 => GO TO THE QASERVER AND GIVE THE WRITE ACCESS = cd /var/lib
STEP 21 => sudo chmod -R o+w tomcat9/
GO TO JENKINS AND CLICK ON BUILD NOW AND YOU WILL SEEE THE DEPLOYMENT HAS PERFORMED
SUCCESSFULLY
STEP 22 => TO ACCESS THE APPLICATION COPY THE PUBLIC IP AND PASTE ON GOOGLE =
QASERVER:8080/qaenv
-------------------
-------------------
---------------------
----------------------
STEP 1 => Now create a password less connection between devserver and production server
passwordauthentication yes
save = esc:wq
STEP 5 => restart the server => sudo systemctl restart sshd or sudo service ssh restart
STEP 6 => Now go to devserver to make an conection
STEP 7 => generate ssh key => ssh-keygen (Generate the key in the case only if the key is not present)
overwrite n
STEP 8 => copy the key to the production server => ssh-copy-id ubuntu@priveteIP of production
server
STEP 9 => Give the permission to the tomcat10 in the production server
ls
STEP 11 => sudo chmod -R o+w tomcat10/ (To give the permission)
DEPLOYMENT AND DELIVERY ARE ALMOST SAME COPY THE SCRIPT OF CONTINUOUS DEPLOYMENT
AND PASTE IN CONTINUOUS DELIVERY
sh '''scp /home/ubuntu/.jenkins/workspace/PIPELINE/web/target/time-tracker-web-0.5.0-
SNAPSHOT.war ubuntu@privateip of prodcution server:/var/lib/tomcat9/webapps/prodenv.war'''
-----------------------------
▌↓ ▌ ↓ ▌ ↓ ▌ ↓ ▌
node
{
git 'https://github.com/vrushankamare95/time-tracker.git'
sh 'mvn package'
sh '''scp /home/ubuntu/.jenkins/workspace/PIEPLINE/web/target/time-tracker-web-
0.5.0-SNAPSHOT.war [email protected]:/var/lib/tomcat9/webapps/qaenv.war'''
sh '''scp /home/ubuntu/.jenkins/workspace/PIEPLINE/web/target/time-tracker-web-
0.5.0-SNAPSHOT.war
[email protected]:/var/lib/tomcat9/webapps/prodenv.war'''
----------
RESULT
--------