0% found this document useful (0 votes)
45 views41 pages

Devops Unit-Iv

The document discusses software builds in DevOps, outlining the definition, phases, and types of build systems, including tools like Jenkins. It highlights the advantages and disadvantages of Jenkins, the importance of managing build dependencies, and the significance of a well-structured file system layout. Additionally, it covers alternative build servers, Infrastructure as Code (IaC) practices, and the collection of software quality metrics using Jenkins.

Uploaded by

hrishiareddy004
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
45 views41 pages

Devops Unit-Iv

The document discusses software builds in DevOps, outlining the definition, phases, and types of build systems, including tools like Jenkins. It highlights the advantages and disadvantages of Jenkins, the importance of managing build dependencies, and the significance of a well-structured file system layout. Additionally, it covers alternative build servers, Infrastructure as Code (IaC) practices, and the collection of software quality metrics using Jenkins.

Uploaded by

hrishiareddy004
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 41

UNIT-IV

Software Builds in DevOps


• Definition: The process of transforming code from one form
to another, including compiling, linting, and testing.

• Phases:
- Compilation: Converting source code to native or
bytecode.
- Linting: Static analysis for code errors, originated from
Unix's Lint tool.
- Unit Testing: Running code in controlled environments to
catch errors early.
- Artifacts: Generating deployable code.
Build Systems and Their Types
• Software build tools are used to automate the
creation of executable applications from source code.
• They script or automate a variety of tasks, such as
downloading dependencies, compiling and packaging
code, running tests, and deployment.
• They can be triggered through the command line,
inside an IDE, or by continuous integration tools after
checking the code out of a repository and onto a build
machine.
• Different build systems exist for different languages.
Usually, build systems use either a domain-specific
language or XML to specify a build.
Build Systems and Their Types
• Different build systems for different languages:
- Java: Maven, Gradle, Ant.
- C/C++: Make, Autotools.
- JavaScript: Grunt.
- Clojure: Leiningen.
- Ruby: Rake.
Challenges of standardizing build tools across an organization:
• - Organizations may use multiple systems due to varying
requirements.
Example: Grunt is more efficient for JavaScript, while Maven suits
Java projects.
• Rule of thumb: Developers should be able to build code locally
with minimal surprises.
The Role of Jenkins in Build Management
• Jenkins: A popular build server forked from Hudson,
created by Kohsuke Kawaguchi.
• Jenkins is versatile, supporting builds in Java and
other languages.
• Setup: Easy to install (e.g., on Fedora via dnf),
accessible at http://localhost:8080.
• Job Types:
- Freestyle project: Basic job type with build steps
like running shell commands (e.g., fortune or date).
- Other job types allow more complex setups (e.g.,
chaining jobs, pipelines).
• Jenkins builds and tests our software projects which continuously
making it easier for developers to integrate changes to the project,
and making it easier for users to obtain a fresh build.
• It also allows us to continuously deliver our software by
integrating with a large number of testing and deployment
technologies.
• Possible steps executed by Jenkins are for example:
• Perform a software build using a build system like Gradle or Maven
Apache
• Execute a shell script
• Archive a build result
• Running software tests
Work Flow:
Advantages of Jenkins

• It is an open source tool.


• It is free of cost.
• It does not require additional installations or components. Means it is easy to
install.
• Easily configurable.
• It supports 1000 or more plugins to ease your work. If a plugin does not exist,
you can write the script for it and share with community.
• It is built in java and hence it is portable.
• It is platform independent. It is available for all platforms and different
operating systems. Like OS X, Windows or Linux.
• Easy support, since it open source and widely used.
• Jenkins also supports cloud based architecture so that we can deploy Jenkins in
cloud based platforms.
Disadvantages of Jenkins

• Its interface is out dated and not user friendly


compared to current user interface trends.
• Not easy to maintain it because it runs on a
server and requires some skills as server
administrator to monitor its activity.
• CI regularly breaks due to some small setting
changes. CI will be paused and therefore requires
some developer's team attention.
Managing Build Dependencies
• Automatic dependency fetching: Maven, Grunt, and Go
can automatically fetch dependencies if not present.
• Challenges with C/C++ builds:
- Use of Autotools: Adapts to available dependencies, but
this flexibility can cause issues in enterprise environments.
- RPM (Red Hat Package Manager)spec files: Lists
dependencies and steps to build C/C++ software
consistently.
• Importance of clear dependency management to avoid
missing functionality on production servers.
Jenkins plugins
• Jenkins has a plugin system to add
functionality to the build server. There are
many different plugins available, and they can
be installed from within the Jenkins web
interface.
• Many of them can be installed without even
restarting Jenkins. This screenshot shows a list
of some of the available plugins:
File system layout
• In DevOps, the file system layout refers to how files and directories
are organized and structured within a server or system.
• A well-organized file system is crucial for efficient operations,
version control, CI/CD pipelines, and maintenance.
• Here's an overview of a typical file system layout in DevOps
environments:
• 1. Project Repositories
• /srv/repos/
Stores code repositories, typically managed with Git, Subversion
(SVN), or other version control systems.
– project-1/
– project-2/
– project-n/
2. Build Directories
/var/builds/
Contains all build-related files, including
compiled binaries and logs generated by CI
tools like Jenkins or GitLab CI.
– project-1/
– project-2/
– project-n/
– build_logs/ (stores logs related to build status)
3. Deployment Directories
/var/deploy/
Includes files and configurations for the current
and previous releases of applications. Helps with
rollbacks and maintaining the deployment history.
– project-1/
– project-2/
– releases/ (contains versions of the application
deployed on the server)
– artifacts/ (stores packaged versions ready for
deployment)
4. Configuration Files
/etc/
Holds configuration files for the system, build tools, and
deployed applications.
– jenkins/ (configuration for Jenkins)
– nginx/ (configuration for Nginx server)
– application-config/ (project-specific configurations)

5. Containerization Layout
/var/lib/docker/
Used for container storage when working with Docker. It stores
images, containers, volumes, and other Docker-related data.
– containers/
– images/
– volumes/
6. Logs and Monitoring
/var/log/
Stores log files, crucial for debugging, performance
monitoring, and audits.
– syslog (system logs)
– jenkins/ (CI tool logs)
– nginx/ (web server logs)
– application/ (custom logs from applications)
7. Package/Artifact Storage
/var/artifacts/
Houses compiled binaries, Docker images, and other
deployable artifacts produced by the build process.
– docker/ (Docker images)
– maven/ (Maven-built JAR files)
8. Scripts
/usr/local/bin/
Contains executable scripts or binaries related to the DevOps
environment, including deployment scripts, monitoring scripts, or helper
tools.
deployment-scripts/ (automated scripts for deployments)
cleanup-scripts/ (scripts for managing resources, cleaning old builds)
9. Temporary Files
/tmp/
Temporary files used during builds or deployments. These files are usually
removed after a certain period or at the end of the build process.
ci_tmp/ (temporary files from CI/CD tools)
build_tmp/ (files created during build processes)
10. Virtual Environments and Dependency Management
/var/virtualenvs/ OR
/home/devops/.venvs/
For projects using Python or other interpreted languages, virtual
environments or environment management files are stored here.
project-1-env/
12. Shared Libraries
/usr/local/lib/
Stores shared libraries, including dependencies
and shared components used across multiple
applications or services.
custom-libraries/
13. CI/CD Tool Configurations
/etc/jenkins/ (or the respective directory for the CI
tool you are using)
Stores the configuration files for your CI/CD tools.
jobs/ (defines the pipeline configurations)
plugins/ (manages the plugins used in CI/CD tools)
Considerations:
• Security:
Sensitive files (e.g., secrets, keys, and passwords)
should be stored securely, often in a separate directory
with restricted permissions or using a secrets manager.
• Access Control:
Ensure proper file and directory permissions to restrict
access to critical parts of the system, especially in a
shared environment.
• Consistency:
Keep the layout consistent across environments (dev,
test, prod) to ensure smooth transitions and
deployment processes.
The host server
• The build server is usually a pretty important
machine for the organization.
• Builds shouldn't take too long, so you will need a
server with good specifications for the build server
—with lots of disk space, processor cores, and RAM.
• The build server also has a kind of social aspect: it is
here that the code of many different people and
roles integrates properly for the first time.
• Machines are cheaper than people, so don't let this
particular machine be the area you save money on.
Build Slaves and Build Pipelines
• Build slaves reduce queues by distributing
workloads to different machines.
• Example: Linux master, Windows/Mac slaves for
cross-platform builds.
• Upstream and downstream jobs:
- Upstream: Initial job that triggers further builds.
-Downstream: Jobs triggered after upstream
completion.
• Pipelines: Chaining jobs for complex workflows,
using plugins like Multijob and Workflow
Software on the host
• Depending on the complexity of your builds,
you might need to install many different types
of build tool on your build server.
• Remember that Jenkins is mostly used to
trigger builds, not perform the builds
themselves.
• Most of the build systems are available in the
distribution repositories, so it's very
convenient to install them from there.
Triggers
• You can either use a timer to trigger builds, or you can poll the code
repository for changes and build if there were changes.

• It can be useful to use both methods at the same time

• Git repository polling can be used most of the time so that every
check in triggers a build.

• Nightly builds can be triggered, which are more stringent than


continuous builds, and thus take a longer time. Since these builds
happen at night when nobody is supposed to work, it doesn't matter
if they are slow.

• An upstream build can trigger a downstream build. You can also let
the successful build of one job trigger another job.
Job chaining and build pipelines

• It's often useful to be able to chain jobs


together. In its simplest form, this works by
triggering a second job in the event that the
first job finishes successfully.

• Sometimes, a nicer visualization of the build


steps as well as greater control over the
details of the chain is desired.
• In Jenkins terminology, the first build in a
chain is called the upstream build, and the
second one is called the downstream build.

• Such a build chain is often called a pipeline or


workflow.

• There are many plugins that create improved


pipelines for Jenkins, and the fact that there
are several shows that there is indeed a great
desire for improvements in this area.
Alternative Build Servers
• Jenkins alternatives:
- Travis CI: Popular for open source projects.
- Buildbot: Python-based, customizable.
- Go server (by ThoughtWorks): CI/CD focus.
- Bamboo (by Atlassian): Integrates with Jira for
agile teams.
- GitLab: Built-in CI/CD tools, more integrated
infrastructure-as-code approach.
• Each tool has its strengths; Jenkins excels in
flexibility and plugin availability.
Build servers and infrastructure as code

• Infrastructure as Code is the management of infrastructure


(networks, virtual machines, load balancers, and connection
topology) in a descriptive model, using the same versioning as
DevOps team uses for source code”.

• Put simply, IAC is the practice of managing the infrastructure


using code. It uses machine-readable files to configure and
manage the required IT infrastructure.
The 4 best practices of IaC include:
• Manage infrastructure through source control
to offer detailed audit tracks for modifications.
• Testing the infrastructure for any flaws or
errors using unit testing, integration testing,
and functional testing.
• Allow collaboration around infrastructure
configuration and provisioning.
• Avoid written documentation as the code
itself will document the state of the machine.
Build phases

• One of the principal benefits of the Maven build tool is that


it standardizes builds.
• This is very useful for a large organization, since it won't
need to invent its own build standards.
• Other build tools are usually much more lax regarding how
to implement various build phases.
• The rigidity of Maven has its pros and cons. Sometimes,
people who get started with Maven reminisce about the
freedom that could be had with tools such as Ant.
• You can implement these build phases with any tool, but it's
harder to keep the habit going when the tool itself doesn't
enforce the standard order: building, testing, and
deploying.
Alternative build servers
• While Jenkins appears to be pretty dominant in the build
server scene in my experience, it is by no means alone.
• Travis CI is a hosted solution that is popular among open
source projects.
• Buildbot is a buildserver that is written in, and
configurable with, Python.
• The Go server is another one, from ThoughtWorks.
• Bamboo is an offering from Atlassian. GitLab also supports
build server functionality now.
• Do shop around before deciding on which build server
works best for you. When evaluating different solutions,
be aware of attempts at vendor lock-in.
Collating Software Quality Metrics with Jenkins

• Jenkins collects and visualizes quality metrics


(e.g., Java unit test results).
• Advanced option: SonarQube integration for
deeper code analysis and visualizations.
• SonarQube highlights code issues like bugs,
vulnerabilities, and code smells.
• Downside: Running Sonar builds during nightly
builds to avoid slowing regular build cycles.

You might also like