0% found this document useful (0 votes)
161 views26 pages

Study Notes

Study Notes
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
161 views26 pages

Study Notes

Study Notes
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 26

Chapter 1 – Key CI/CD/Jenkins Concepts .....................................................................................

5
Continuous Delivery/Continuous Integration Concepts .................................................... 5
Define continuous integration, continuous delivery, continuous deployment ..... 5
Difference between CI and CD .................................................................................................. 5
Stages of CI and CD ....................................................................................................................... 5
Continuous delivery versus continuous deployment ...................................................... 5
Jobs .......................................................................................................................................................... 6
What are jobs in Jenkins? ........................................................................................................... 6
Types of jobs.................................................................................................................................... 6
Scope of jobs .................................................................................................................................... 6
Builds ...................................................................................................................................................... 6
What are builds in Jenkins? ....................................................................................................... 6
What are build steps, triggers, artifacts, and repositories? .......................................... 6
Build tools configuration ............................................................................................................ 6
Source Code Management .............................................................................................................. 6
What are source code management systems and how are they used? .................... 6
Cloud-based SCMs ......................................................................................................................... 6
Jenkins changelogs ........................................................................................................................ 6
Incremental updates v clean check out................................................................................. 7
Checking in code ............................................................................................................................ 7
Infrastructure-as-Code ................................................................................................................ 7
Branch and Merge Strategies .................................................................................................... 7
Testing .................................................................................................................................................... 7
Benefits of testing with Jenkins ............................................................................................... 7
Define unit test, smoke test, acceptance test, automated verification/functional
tests..................................................................................................................................................... 7
Notifications ......................................................................................................................................... 7
Types of notifications in Jenkins ............................................................................................. 7
Importance of notifications ....................................................................................................... 7
Distributed Builds .............................................................................................................................. 7
What are distributed builds? .................................................................................................... 7
Functions of masters and slaves .............................................................................................. 7
Plugins .................................................................................................................................................... 8
What are plugins? .......................................................................................................................... 8
What is the plugin manager? .................................................................................................... 8
Jenkins Rest API .................................................................................................................................. 8
How to interact with it ................................................................................................................ 8
Why use it? ....................................................................................................................................... 8
Security .................................................................................................................................................. 8
Authentication versus authorization ..................................................................................... 8
Matrix security ............................................................................................................................... 8
Fingerprints.......................................................................................................................................... 8
What are fingerprints? ................................................................................................................ 8
How do fingerprints work? ....................................................................................................... 8
Artifacts .................................................................................................................................................. 9
How to use artifacts in Jenkins ................................................................................................. 9
Configuration Management (Tools such as Chef, Puppet, etc.) ........................................ 9
Elements of software configuration management ........................................................... 9
Change management policies ................................................................................................... 9
Importance of software configuration management....................................................... 9
Using 3rd party tools .......................................................................................................................... 9
How to use 3rd party tools with Jenkins ............................................................................... 9
Chapter 2 – Jenkins Usage ................................................................................................................ 10
Jobs ....................................................................................................................................................... 10
Organizing jobs in Jenkins ...................................................................................................... 10
Parameterized jobs .................................................................................................................... 10
Usage of Freestyle/Pipeline/Matrix/Maven/Literate ................................................. 10
Builds ................................................................................................................................................... 10
Setting up build steps and triggers ...................................................................................... 10
Configuring build tools ............................................................................................................. 10
Running scripts as part of build steps ................................................................................ 10
Source Code Management ........................................................................................................... 11
Polling source code management ........................................................................................ 11
Creating hooks ............................................................................................................................. 11
Including version control tags and version information ............................................ 11
Testing ................................................................................................................................................. 11
Testing for code coverage ....................................................................................................... 11
Test reports in Jenkins ............................................................................................................. 11
Displaying test results .............................................................................................................. 11
Integrating with test automation tools .............................................................................. 12
Breaking builds ........................................................................................................................... 12
Notifications ...................................................................................................................................... 12
Setup and usage .......................................................................................................................... 12
Email notifications, instant messaging, build radiators .............................................. 12
Alarming on notifications ........................................................................................................ 12
Distributed Builds ........................................................................................................................... 12
Setting up and running builds in parallel .......................................................................... 12
Setting up and using SSH slaves, JNLP slaves, cloud slaves ....................................... 12
Monitoring nodes ....................................................................................................................... 12
Plugins ................................................................................................................................................. 12
Setting up and using Plugin Manager ................................................................................. 12
Finding and configuring required plugins ........................................................................ 13
CI/CD .................................................................................................................................................... 13
Using Pipeline (formerly known as Workflow) .............................................................. 13
Integrating automated deployment .................................................................................... 13
Release management process ............................................................................................... 13
Pipeline stage behavior ............................................................................................................ 13
Jenkins Rest API ............................................................................................................................... 13
Using REST API to trigger jobs remotely, access job status, create/delete jobs 13
Security ............................................................................................................................................... 14
Setting up and using security realms.................................................................................. 14
User database, project security, Matrix security ............................................................ 14
Setting up and using auditing ................................................................................................ 14
Setting up and using credentials .......................................................................................... 14
Fingerprints....................................................................................................................................... 15
Fingerprinting jobs shared or copied between jobs ..................................................... 15
Artifacts ............................................................................................................................................... 15
Copying artifacts ......................................................................................................................... 15
Using artifacts in Jenkins ......................................................................................................... 15
Artifact retention policy........................................................................................................... 15
Alerts .................................................................................................................................................... 15
Making basic updates to jobs and build scripts .............................................................. 15
Troubleshooting specific problems from build and test failure alerts .................. 15
Chapter 3 – Building Continuous Delivery Pipelines ............................................................. 16
Pipeline Concepts ............................................................................................................................ 16
Value stream mapping for CD pipelines ............................................................................ 16
Why create a pipeline? ............................................................................................................. 16
Gates within a CD pipeline ...................................................................................................... 16
How to protect centralized pipelines when multiple groups use same tools ..... 16
Definition of binary reuse, automated deployment, multiple environments ..... 16
Elements of your ideal CI/CD pipeline – tools ................................................................ 16
Key concepts in building scripts (including security/password, environment
information, etc.) ........................................................................................................................ 16
Upstreams and downstreams..................................................................................................... 16
Triggering jobs from other jobs ............................................................................................ 16
Setting up the Parameterized Trigger plugin .................................................................. 17
Upstream/downstream jobs .................................................................................................. 17
Triggering........................................................................................................................................... 17
Triggering Jenkins on code changes ................................................................................... 17
Difference between push and pull ....................................................................................... 17
When to use push vs pull ......................................................................................................... 17
Pipeline (formerly known as Workflow) ............................................................................... 17
Benefits of Pipeline vs linked jobs ....................................................................................... 17
Functionalities offered by Pipeline...................................................................................... 17
How to use Pipeline ................................................................................................................... 18
Pipeline stage concurrency..................................................................................................... 18
Visualization...................................................................................................................................... 18
Options to visualize jobs’ relationships ............................................................................. 18
When to use various options for visualizing jobs’ relationships ............................. 18
Information offered by a build pipeline view .................................................................. 18
How to set up build pipeline visualization ....................................................................... 18
Folders ................................................................................................................................................. 18
How to control access to items in Jenkins with folders ............................................... 18
Referencing jobs in folders ..................................................................................................... 18
Parameters ........................................................................................................................................ 18
Setting up test automation in Jenkins against an uploaded executable ................ 18
Passing parameters between jobs ....................................................................................... 19
Identifying parameters and how to use them: file parameter, string parameter
........................................................................................................................................................... 19
Jenkins CLI parameters ............................................................................................................ 19
Promotions ........................................................................................................................................ 19
Promotion of a job...................................................................................................................... 19
Why promote jobs? .................................................................................................................... 19
How to use the Promoted Builds plugin ............................................................................ 19
CD Metrics .......................................................................................................................................... 19
KPIs/metrics for CI/CD ............................................................................................................ 19
Determining how many builds failed, succeeded .......................................................... 19
Determining how long a build takes ................................................................................... 20
Determining how often code is checked-in ...................................................................... 20
How to use metrics/KPIs......................................................................................................... 20
Notifications ...................................................................................................................................... 20
How to radiate information on CD pipelines to teams ................................................ 20
Chapter 4 – CD as Code Best Practices......................................................................................... 21
Distributed builds architecture ................................................................................................. 21
Fungible (replaceable) slaves..................................................................................................... 21
Master-slave connectors and protocol ................................................................................... 21
Tool installations on slaves ......................................................................................................... 21
Cloud slaves ....................................................................................................................................... 21
Containerization .............................................................................................................................. 21
Traceability........................................................................................................................................ 21
High availability ............................................................................................................................... 21
Automatic repository builds ....................................................................................................... 21
Chapter 5 – Cloudbees Jenkins Platform..................................................................................... 22
Reference architecture .................................................................................................................. 22
Role-based Access Control (RBAC) .......................................................................................... 22
Folders Plus ....................................................................................................................................... 22
Templates ........................................................................................................................................... 23
Setting up High Availability (HA) .............................................................................................. 23
CloudBees Jenkins Operations Center (CJOC) ...................................................................... 23
Shared clouds ............................................................................................................................... 23
Cloud configurations ................................................................................................................. 24
Shared slaves................................................................................................................................ 24
Analytics ............................................................................................................................................. 24
Cluster Operations .......................................................................................................................... 24
Pipeline Checkpoints (formerly known as Pipeline Checkpoints)............................... 25
Custom Update Center .................................................................................................................. 25
Multi-branch ..................................................................................................................................... 26
Docker plugins ................................................................................................................................. 26
Chapter 1 – Key CI/CD/Jenkins Concepts

Continuous Delivery/Continuous Integration Concepts


Define continuous integration, continuous delivery, continuous deployment
 Continuous integration – everyone commits to the mainline at least daily and
automated build to verify
 Continuous delivery – can release to prod at any time via a push button
deployment
 Continuous deployment – actually deploying to production continually
 DevOps is cultural and is broader than continuous delivery
 Pipeline has visibility, feedback and continuous delivery
 CI practices
o Single source repository
o Automate the build
o Make your build self testing
o Everyone commits everyday
o Every commit triggers a build
o Fix broken builds immediately
o Keep the commit build fast (and use pipeline for slower builds)
o Test in a clone of the prod environment
o Make it easy to get the latest build
o Visibility
o Automate deployment
 CD principles
o Check in
o Build and unit tests
o Automated acceptance tests
o User acceptance test
o Release

Difference between CI and CD


 CI doesn’t require deploying

Stages of CI and CD
 Start with the commit stage which compiles and runs unit tests.
 Then run longer tests/quality tools/ Ex: acceptance tests (given/when/then)
 Finally, deploy

Continuous delivery versus continuous deployment


 Delivery means the ability to deploy to production. Deployment means actually
doing so
Jobs
What are jobs in Jenkins?
 Job/Project – Runnable tasks

Types of jobs
 Freestyle project
 Maven project
 Pipeline
 Multi configuration
 Multi branch
 Long running

Scope of jobs

 Not sure what this means – Maybe that there is a long running job type?

Builds

What are builds in Jenkins?


 Build – Result of one run of a job/project

What are build steps, triggers, artifacts, and repositories?


 Build step – a single operation withing a build
 Triggers – something that starts a build (time, SCM polling, etc)
 Artifact – output of a build
 Repository – the SCM system where the code to be built lives

Build tools configuration


 In Manage System, set up location of tools like the JDK, Ant and Maven

Source Code Management


What are source code management systems and how are they used?
 Use to track code
 Client/server – one source of truth such as SVN.
 Distributed version control – every developer has copy of repository, peer to
peer, such as Git.

Cloud-based SCMs

 Ex: Git hub

Jenkins changelogs

 List commits since last build


Incremental updates v clean check out
 Incremental updates – faster
 Clean check out – guarantees no extra or changed local files

Checking in code
 Should be at least daily with CI

Infrastructure-as-Code
 Storing everything needed to build your environment

Branch and Merge Strategies


 Branch by release
 Branch by feature – by user story
 Branch by abstraction – one branch, but turn features on/off by release
 Merge regularly

Testing
Benefits of testing with Jenkins
 Fast feedback!

Define unit test, smoke test, acceptance test, automated verification/functional tests
 Unit test – test one class, often involves test doubles
 Integration/functional test – test components together
 Smoke test – sanity check to reject a release. Looking for major errors.
 Acceptance test – user level test for feature

Notifications
Types of notifications in Jenkins
 Failure, second failure, success, etc
 Active/push – radiators/SMS vs passive/pull – rss/dashboard
 RSS - /rssAll, /rssFailed and rssLatest
 Radiator view plugin uses the entire screen
 Extreme feedback – physical/audio devices

Importance of notifications
 Fixing a build is high priority so need to know it is broken
 Communicating the status to all parties

Distributed Builds
What are distributed builds?
 Running builds on a different machine than master

Functions of masters and slaves


 Master – basic Jenkins install
 Slaves – just for running jobs

Plugins
What are plugins?
 Add functionality to core Jenkins

What is the plugin manager?


 UI for uploading/managing plugins

Jenkins Rest API


How to interact with it
 Format: XML or JSON
 Python and Ruby wrapper APIs

Why use it?


 Programmatic access

Security
Authentication versus authorization
 Authentication – identify a user
 Authorization – what user can do

Matrix security
 Maps roles to permissions
 Major categories: overall, slave, job, run, view and SCM
Definition of auditing, credentials, and other key security concepts
 Auditing – logging user operations and changes
 Credentials – username/password or the like for access

Fingerprints
What are fingerprints?
 MD5 checksum of files
 UI says for jar files, but works for any type of file

How do fingerprints work?


 The first time you run a job with a post build step to generate a fingerprint, a
new left navigation option shows up to check a file’s fingerprint.
 You can upload a file you have to see if any file Jenkins knows the fingerprint of
matches.
Artifacts
How to use artifacts in Jenkins
 Download, put in Nexus, deploy, etc
Storing artifacts
 Can archive
 Can control discard policy

Configuration Management (Tools such as Chef, Puppet, etc.)


Elements of software configuration management
 Tracking/controlling changes in the software
 Includes version control

Change management policies


 Not sure what they mean here. This is a big topic

Importance of software configuration management


 Need to know what you deploy!

Using 3rd party tools


How to use 3rd party tools with Jenkins
 Setup in Manage System the location on disk or download from there
 Ex: JDK, Maven, Git
 Can install automatically or from file system
Chapter 2 – Jenkins Usage
Jobs
Organizing jobs in Jenkins
 Jobs are organized in folders

Parameterized jobs
 Check “This build is parameterized” and enter parameters/default values
 Run directly with “Build with Parameters” or call from upstream job with
“trigger parameterized build” post build action and passing parameters

Usage of Freestyle/Pipeline/Matrix/Maven/Literate
 Freestyle – most flexible job
 Pipeline – enter code in DSL. There is a snippet generator which generates
the Groovy for common operations and lists the available environment
variables.
 Matrix (multi-config) – Specify a configuration matrix with one or more
dimensions. Runs all combinations when build.
o Axis: slave, label (for slave) or user defined (string)
o Combination filter: if don’t want cross product of all axis to run
o Can execution “touchstone” builds first to specify which job(s) should
run first and if this should skip the others
 Maven - less options than Freestyle since can assume based on Maven
conventions
 Literate – brand new plugin (Dec 2015) – allows specifying build commands
in README.md file in source control. A literate job is a type of multi-branch
job. (searches for new branches and creates jobs in folder automatically)

Builds
Setting up build steps and triggers
 Common build steps include Maven/Ant, execute shell, start/stop Docker
container
 Common triggers include time/periodic, SCM polling, upon completion of
another job

Configuring build tools


 In Manage Jenkins > Manage System
 Install automatically or via system

Running scripts as part of build steps


 Can run OS script or Groovy script
 Groovy scripts can run as system or user level. System has access to Jenkins
object model
Source Code Management
Polling source code management
 Set schedule using cron format
o minute hour dayOfMonth month dayOfWeek
o For dayofWeek, 0 is Sunday and 7 is Saturday
o Can use H (or H/2 etc) for minute column to use a hash based on the
job name to distribute jobs so don’t all start at the top of the hour.
o Also support, @yearly, @annually, @monthly, @weekly, @daily,
@hourly and @midnight
o @Midnight means between midnight and one am since uses hash to
distribute
 Required URL
 Optional credentials
 Options vary by repo. Ex: SVN lets you specify infinity/immediates/etc as
checkout depth. Git lets you specify a branch specifier

Creating hooks
 Hook script in repository triggers job
 Ex: Github plugin provides hook

Including version control tags and version information


 Git allows you to create a tag for every build
 Version Number plugin lets you include info in build name

Testing
Testing for code coverage
 In build, must create XML file with data
 Post Build Action to publish
 For Java: Cobertura and JaCoCo
 In Cobertura, can set thresholds for weather icons:
o Sunny - % higher than threshold
o Stormy - % lower than threshold
o Unstable - % lower than threshold
 In Jacoco, can set thresholds for sunny and stormy

Test reports in Jenkins


 Publish JUnit or TestNG reports
 In JUnit, can set amplification factor - 1.0 means 10% failure rate scores 90%
health. .1 means 10% failure rate scores 99% health.

Displaying test results


 Configure as Post Build Action
 Point to xml files: ex: reports/*.xml
 Can drill down to see details of tests runs and durations
Integrating with test automation tools
 Can run acceptance tests later in pipeline than unit/component tests

Breaking builds
 JUnit allows choosing whether to fail builds on test failures - default is
“unstable” not failure

Notifications
Setup and usage
 Setup in post build action section

Email notifications, instant messaging, build radiators


 Email
o Same recipient for each one (except can add committers since passed)
 Email ext
o lets you customize the message and tailor the recipients per trigger
o can send on failing, still failing, unstable, still unstable, successful, etc
 Jabber and IRC for instant messaging
 Since build radiators are full screen, the only way to edit is to add /configure
to the URL

Alarming on notifications
 Extreme notifications can have a video or audio cue in the real world

Distributed Builds
Setting up and running builds in parallel
 Builds run on different executors
 Multi-configuration jobs run the pieces in parallel

Setting up and using SSH slaves, JNLP slaves, cloud slaves


 Can launch local slaves with SSH (blocking or non-blocking IO), Java Web
Start, command line on master or Windows service

Monitoring nodes
 Monitoring page uses JMelody
 Memory/CPU/etc stats
 Can see heap dump/GC/etc

Plugins
Setting up and using Plugin Manager
 Can provide a HTTP proxy if needed
 Can specify alternate update center URL for JSON
 Listed installed plugins
 Can install/upgrade/uninstall plugin
 Can unpin plugin so doesn’t use specific version of plugin

Finding and configuring required plugins


 Updates tab – for upgrading plugin already have
 Available tab – for downloading new plugins
 Advanced tab – for uploading plugin hpi/jpi file from disk
 Configure plugins on Manage Jenkins -> Manage System

CI/CD
Using Pipeline (formerly known as Workflow)
 Use DSL to specify jobs to be built
 Example: node { stage ‘x’ echo ‘1’ stage ‘y’ echo ‘2’ }
 Sample commands:
o build 'jeanne-test'
o svn - checkout
o retry – retry body up to X times
o timeout – limit time spent in block
o stash/unstash
o load – include a Groovy script
o parallel – specify two branches to run in parallel and whether to
failFast
 When run build, see table with column and duration for each stage. Row is
build #. Cell color coded for pass/fail. Can see log for each stage.

Integrating automated deployment


 Have the pipeline itself triggered by SCM
 Then the pipeline triggers the commit job first followed by the rest of the
jobs in the pipeline
 The docker variable can be used as a build step in the pipeline or to surround
other lines

Release management process


 Not sure what this refers to. Gates/approvals?

Pipeline stage behavior


 Stages run one at a time unless specify parallel
 A subsequent stage only runs if the prior one was successful

Jenkins Rest API


Using REST API to trigger jobs remotely, access job status, create/delete jobs
 /api shows docs for the REST API at that level of the object model
 /api/xml, /api/json, /api/json?pretty=true, /api/python and
/api/python?pretty-true
 Choose “trigger builds remotely” on build and set token to allow POST call.
o Run build: POST to JENKINS_URL/job/job-
name/build?token=MY_TOKEN
o Run build with reason: POST to JENKINS_URL/job/job-
name/build?token=MY_TOKEN&cause=xyz
o Run Parameterized Build: POST to JENKINS_URL/job/job-
name/buildWithParameters?token=MY_TOKEN&param=xyz
 Error handling:
o If try to call /build for parameterized job, get a 400 error
o If try to call with wrong token, get a 403 error
o If don’t choose “trigger builds remotely”, it worked
 CSRF
o Get token at JENKINS_URL/crumbIssuer/api/xml
o Pass .crumb as header with POST
 All job (at top level) latest status: JENKINS_URL/api/xml
 Build numbers and urls for a job: JENKINS_URL/job/jobName/api/xml
 Build result and details: JENKINS_URL/job/jobName/buildNumber/api/xml
 Create job: POST to JENKINS_URL/createItem?name=jobName and post
config.xml
 Delete job: POST to JENKINS_URL/job/jobName/doDelete
 Enable job: POST to JENKINS_URL/job/jobName/enable
 Disable job: POST to JENKINS_URL/job/jobName/disable

Security
Setting up and using security realms
 Choices include Servlet Container, Google SSO, OpenId, Jenkins user database,
LDAP, UNIX group/user database, JCOC SSO

User database, project security, Matrix security


 People link shows user list + committers
 Matrix based security – control privileges granularly using user ids/groups
 Project based matrix authorization security – Matrix based + set privileges on
job configuration page as well
 Role based matrix authorization security – Manage Roles to control
permissions by group. Adds groups/roles tabs to projects

Setting up and using auditing


 Manage Jenkins > System Log – for logging
 Job Configuration History plugin – for job config
 Audit Trail plugin – for Jenkins config

Setting up and using credentials


 Domain – URL, host etc
 Credentials – username/password, cert, etc
 Use by choosing from pull down in job
Fingerprints
Fingerprinting jobs shared or copied between jobs
 Used to determine if a dependency has changed
 See which projects use a dependency
 See where fingerprinted files came from

Artifacts
Copying artifacts
 Build step to copy artifacts from another build
 Can choose which ones want to include/exclude by pattern

Using artifacts in Jenkins


 Can refer to artifacts after build
 Treated specially not just as part of workspace

Artifact retention policy


 By default, kept same length of time as build log.
 Can keep less time to save disk space

Alerts
Making basic updates to jobs and build scripts
 Not sure what they mean here

Troubleshooting specific problems from build and test failure alerts


 Not sure what they mean here
Chapter 3 – Building Continuous Delivery Pipelines
Pipeline Concepts
Value stream mapping for CD pipelines
 Entire process from concept to cash for a product
 Includes non code aspects such as product discovery
 Shows were time goes in process and where waits/delays are
 CD pipeline is subset of value stream map

Why create a pipeline?


 Automated manifestation of process for getting software from version
control to users
 Allows for phases of increasing fitness

Gates within a CD pipeline


 Provide a point for approval before continuing.

How to protect centralized pipelines when multiple groups use same tools
 Not sure what this means. Approvals? Security?

Definition of binary reuse, automated deployment, multiple environments


 Binary reuse – Use other components as packaged, artifacts that have passed
success criteria
 Automated deployment – using the same script to deploy to every
environment
 Multiple environments – resources/configuration needed to work: ex: test,
QA, Prod

Elements of your ideal CI/CD pipeline – tools


 Source control repository
 Binary repository
 Automated testing
 Capacity testing
 Deployment

Key concepts in building scripts (including security/password, environment


information, etc.)
 Credentials plugin for password
 Keep environment information in source control
 Different script for each stage in the pipeline

Upstreams and downstreams


Triggering jobs from other jobs
 Build other projects
o Comma separated list of jobs
o Can specify to trigger only on good builds, good builds + unstable
builds and always (even on failure)
o All jobs share same trigger
 Trigger parameterized build on other projects
o Comma separated list of jobs
o Can control based on success, unstable, failure only, aborted, etc
o Can set up multiple triggers so each set has different rules on when to
run
o Parameter types include boolean, string, from a property file, current
build parameters, etc
o Can pass through information like slave or Git/SVN trigger info

Setting up the Parameterized Trigger plugin


 Check “This build is parameterized” and setup parameters
 Can use Node to specify slave by name from select list or label to specify
slave’s build label

Upstream/downstream jobs
 If A depends on B, B is the upstream job

Triggering
Triggering Jenkins on code changes
 For a commit build

Difference between push and pull


 Pull - Set up a SCM polling trigger
 Push – Set up a hook from the repository to trigger job

When to use push vs pull


 Pull for when you don’t control the repository or polling is ok
 Push for when you need an immediate build or don’t want to waste resources
on polling

Pipeline (formerly known as Workflow)


Benefits of Pipeline vs linked jobs
 Scripted – can code loops/conditionals
 Resilient – can survive Jenkins restarts
 Pausable – can get manual approval
 Efficient – can restart from checkpoints
 Visualized – can see in dashboard

Functionalities offered by Pipeline


 Build steps, pauses, parallelization, deploy, stash/unstash, etc
 Can run on certain node with node(‘master’) {}
 Can prompt user with input ‘query’
 Can do anything Groovy can do
 Can create stages

How to use Pipeline


 Put commands want to run inside node{}
 Use snippet generated or write groovy script
 Can store global libraries in git at git clone <Jenkins>/workflowLibs.git

Pipeline stage concurrency


 Parallel lets you run stages at same time

Visualization
Options to visualize jobs’ relationships
 Build Pipeline view – shows upstream/downstream dependencies for one
job
 A pipeline automatically creates a stage view – can click to see “Full Stage
View”
 Delivery pipeline view – not on exam? – shows more details about stages

When to use various options for visualizing jobs’ relationships


 Can restrict to only include successful builds

Information offered by a build pipeline view


 Dependencies
 Status
 When run

How to set up build pipeline visualization


 Create a new view
 Choose job to start from
 Can also include in a dashboard view so have more than one per page

Folders
How to control access to items in Jenkins with folders
 Role Based Access Control can control folder
 Can control level as current/child/grandchild

Referencing jobs in folders


 <jenkinsHome>/job/folder/job/name

Parameters
Setting up test automation in Jenkins against an uploaded executable
 File parameter in parameterized job
 Prompted to upload it when running manually

Passing parameters between jobs


 Can type parameters, use property file, etc

Identifying parameters and how to use them: file parameter, string parameter
 String parameter referred to by variable name ${TEST}
 File parameter placed in the workspace in the parameter name

Jenkins CLI parameters


 Download jar from <Jenkins>/jnlpJars/jenkins-cli.jar
 Run as java –jar Jenkins-cli.jar –s <jenkinsUrl> help
 Add –noKeyAuth if don’t want to use SSH key

Promotions
Promotion of a job
 Can run steps after a gate
 Ex: archive artifacts, deploy, etc

Why promote jobs?


 Way of communicating a build is good

How to use the Promoted Builds plugin


 Promote Builds plugins lets you specify actions that require approval
 Adds promotion status link when check “Promote builds when…”
 Approvals include manually, automatically, based on downstream/upstream
builds
 Can run multiple build steps (or post build actions) to run after approval –
retry-able independently. Like a separate build.
 See icon once approved or if steps after approval fail
 Can have multiple promotion processes

CD Metrics
KPIs/metrics for CI/CD
 Cycle time
 Test coverage, cyclomatic complexity, duplication, etc
 Number of defects
 Velocity
 # Commits per day
 # Builds per day – success, failures and total
 Duration of build

Determining how many builds failed, succeeded


 Dashboard view – build stats, job stats
Determining how long a build takes
 Trend on individual job

Determining how often code is checked-in


 Number of commit stage builds

How to use metrics/KPIs


 Tracking improvement
 Identifying limiting constraint

Notifications
How to radiate information on CD pipelines to teams
 Email , radiator, etc
Chapter 4 – CD as Code Best Practices

Distributed builds architecture


 Run jobs on slave
 More secure because jobs run on slave
 More scalable because can add slaves
 Vertical growth – master is responsible for more jobs
 Horizontal growth – creation of more masters
 Recommend to virtualize slaves, but not master for performance

Fungible (replaceable) slaves


 Can configure third party tools to automatically install on slaves
 Best practice is to make slaves interchangeable, but can tie jobs to slaves

Master-slave connectors and protocol


 SSH connector – preferred option. Slaves need SSHD server and
public/private key
 JNLP/TCP connector – Java Network Launch Protocol start web agent on
slave through JWS (Java Web Start). Can start via browser or OS service
 JNLP/HTTP connector – like JNLP/TCP except headless and over HTTP
 Custom script – launch via command line

Tool installations on slaves


 Can install manually or have Jenkins do it

Cloud slaves
 EC2 for Amazon Cloud
 JCloud – for other clouds

Containerization
 Docker image to deploy/run application
 “Build inside a Docker Container” option

Traceability
 Docker Traceability plugin uses fingerprints for images

High availability
 Master must be on network attached storage device
 Don’t do builds on master or at least not with workspace under
JENKINS_HOME
 HAProxy serves as the reverse proxy

Automatic repository builds


 Not sure what this means. It does not exist in any documentation online except
the PDF study guide.
Chapter 5 – Cloudbees Jenkins Platform
Reference architecture
 Products
o Jenkins Enterprise – open source Jenkins plus plugins for High
Availability, RBAC, Update Center, folders, etc.
o Cloudbees Jenkins Operations Center – dashboard, manage multiple
masters
 No builds on CJOC or downstream masters
 Recommend hundreds, not thousands of jobs on each downstream master
 Faster recovery and less frequent failures
 Proxy fronts primary master and checks availability
 CJOC master is a master with CJOC installed
 CJOC master knows about all slaves. Like a cloud for slaves
 Can set up different update centers for different downstream masters

Role-based Access Control (RBAC)


 Setup in manage security. Choose role based matrix authorization strategy
(vs matrix based on project matrix based)
 Defaults to logged in users can do anything and anonymous users can do
nothing
 Default groups – Administrators, Developers, Browsers
 Default roles – anonymous, authenticated, administer, develop, browse
 Roles > Manage – global matrix of role/permission mappings
 Two types of roles – system defined and user defined
 Can’t get rid of anonymous and authenticated roles
 Extended read permission – can view, but not edit config
 Support group definitions out of the box – Jenkins, jobs, Maven modules,
slaves, views and folders
 To prevent folder role from propogating to children - Group icons– blue
means pinned
 To prevent folder role from inheriting from parent – Roles > filter

Folders Plus
Features over folders plugin:
 tie slaves to folders
 move jobs between folders
 health reports other than child with worst health (ex: average health, job
status, enabled projects)
 set icons on folder other than default (ex: aggregate of status, built in icons or
by URL)
 pass environment variables to all jobs in folder
 display jobs from subfolders on higher level view
 restrict what goes in folder
Templates
 Types
o Auxiliary template – nested attributes within another template
o Builder/publisher template – locked down builder/publisher
o Folder/job template – configure folder/job
 If define in folder, limited to that folder
 Transformation types
o Jelly – has ${} and some control tags – like JSTL but different tags.
o Groovy template transformation – like a JSP in Groovy. Remember to
backslash $
o Groovy template for Pipleine
 Variables instance, model, parent (Folder or Jenkins instance itself) and
parentInstance (the folder template where the job template sits)
 When admin updates template, automatically approved. When non-admin
updates template, checked against whitelist of approved code or added to “in
process script approval” list for admin.
 Groovy sandbox – can whitelist method signatures first time used. Format
method class.Name methodName argTypes (or static method). Admins use
whitelist too when sandbox on.
 Creating with REST
o POST to /instantiate
o Or /createItem and specify JobPropertyImpl for template

Setting up High Availability (HA)


 HA for Jenkins is multiple JVMs forming a cluster.
 It is a singleon – only one is master at a time
 Config – NFS /shared disk, at least two servers, floating IP
 Jenkins-ha-monitor provides monitoring on when to switch IP between
servers
 Need three pieces:
o Jenkins enterprise war
o Jenkins enterprise proxy HA war – start this and it proxies/passes
through to regular Jenkins.war
o Jenkins enterprise HA monitoring tool – triggers transfer logic from
outside Jenkins
 Data survives failover except builds in progress and user sessions
 Typically takes a few minutes because has to start up secondary

CloudBees Jenkins Operations Center (CJOC)


Shared clouds
 Same access logic as shared slaves
 Clouds provision slaves to master
 Local Types: java web start or virtual machine

Cloud configurations
 Supports Docker, Amazon EC2 and Microsoft Azure clouds
 Instance caps are managed on each master
 Credentials shared across masters

Shared slaves
 Client masters in the same CJOC can share slave executors
 Client masters must be siblings or in same subfolder
 Slaves are leased to client masters for one job if CJOC is available. If it goes
down, client master keeps slave until comes back.
 Client masters prefer slaves in current “folder” then go to parent
 Client masters are not allowed to use slaves at sibling folder level
 Create shared slaves with CJOC

Analytics
 Jenkins masters report data to CJOC
 Display dashboards
 Can create custom dashboards
 To reindex and get historical data in CJOC
o new Cluster Operations job
o operation = masters
o target masters == from operations root
o step == reindex
 Can run Elastic Search embedded or remote
 Uses Kibana open source analytics and visualization platform
 Includes System/JVM metrics, Web UI metrics, Jenkins metrics, health checks
 Retention of data (reindexing resets clock)
o Every 10 seconds metrics – saved 3 days
o Hourly metrics – saved 3 years
o Build reporting – saved 3 years
o Other info saved forever

Cluster Operations
 Used to performance maintenance operations from CJOC
 Ways to run
o Checkbox on list view to prepare for shutdown or safe restart with left
navigation “cluster operations”
o Left navigation “cluster operations” on single master
o Cluster operations job
 Each operation in job has:
o type = master or update center
o source = root, parent, parameter, etc
o optional filter on path, online status, etc
o steps
 for master – Backup master, install/enable/disable plugin,
execute groovy script, prepare for shutdown, refresh update
center metadata, restart now, safe restart, upgrade jenkins,
upgrade all plugins
 for update center – Delete/promote/update core,
delete/promote/update plugin, pull everything, pull new
versions, refresh upstream sources, track latest core, track
latest plugins
o advanced options
 # parallel items
 timeout per step
 failure mode – immediately, tidy (at end of current step), at
end
 build result to use on failure - unstable, failure, aborted
 If you have multiple items to operate on, they will occur in parallel

Pipeline Checkpoints (formerly known as Pipeline Checkpoints)


 All pipelines can be resumed
 For a more granular resume, put checkpoint ‘name’ in your script.
 Local variables saved at checkpoint. Call stash if want to store files.
 Restart using Checkpoints link or retry icon
 Call unstash to retrieve files into workspace
 Get new build #, but skips all steps prior to checkpoint
 Place checkpoint outside of node{} so not reliant on state of workspace

Custom Update Center


 Benefits: restricting plugins, sharing in house developed plugins
 Options:
o Versions of plugins - Require explicit configuration or Implicitly push
latest
o Signature provider – ex: self signed
o Upstream sources – like proxied updated centers – Jenkins Enterprise,
Open Source or Local. Can also choose types: ex LTS
o Maintenance tasks – pull new versions (of what already in update
center) or pull everything
 Tabs
o Core – Jenkins itself
o Plugins – Jenkins plugins
o Tool installers – ex: Groovy, Chrome Driver
o Upload core – upload Jenkins war from local machine
o Upload plugin – upload plugins from local machine
 Click Store button to save a version locally

Multi-branch
 Benefits of Workflow Multi-Branch: automatic creation/deletion of job for
each new/deleted branch in repo
and configuring properties by branch
 Uses marker file Jenkinsfile to define pipeline logic and recognize a job
should be created
 Job gets deleted when branch or Jenkinsfile removed
 Create new Multibranch Workflow job
 Can give named branches different properties by specifying exceptions
 Creates a folder for these jobs to exist in

Docker plugins
 Docker is containers for deployment
 Dockerhub (hub.docker.com) is like github – hosting for Docker
 Plugins
o Docker – provision slave, run single build and then tear down that
slave
o Dockerhub notification - provides a hook so Docker can trigger
Jenkins jobs when the image is updated
o Docker build and registry - allows publishing to the Docker registry
o Docker traceability – history of deployments/images
o Docker pipeline provides docker variable to pipeline plugin
 Examples:
o Build container: docker.build ‘path/app:${env.BUILD_TAG}’
o Run inside container: docker.image(‘name’).inside { /* commands */ }
o Reference container from outside in docker.withRun(‘name’).inside {
/* commands */ }

You might also like