Devops Unit-Iv
Devops Unit-Iv
• Phases:
- Compilation: Converting source code to native or
bytecode.
- Linting: Static analysis for code errors, originated from
Unix's Lint tool.
- Unit Testing: Running code in controlled environments to
catch errors early.
- Artifacts: Generating deployable code.
Build Systems and Their Types
• Software build tools are used to automate the
creation of executable applications from source code.
• They script or automate a variety of tasks, such as
downloading dependencies, compiling and packaging
code, running tests, and deployment.
• They can be triggered through the command line,
inside an IDE, or by continuous integration tools after
checking the code out of a repository and onto a build
machine.
• Different build systems exist for different languages.
Usually, build systems use either a domain-specific
language or XML to specify a build.
Build Systems and Their Types
• Different build systems for different languages:
- Java: Maven, Gradle, Ant.
- C/C++: Make, Autotools.
- JavaScript: Grunt.
- Clojure: Leiningen.
- Ruby: Rake.
Challenges of standardizing build tools across an organization:
• - Organizations may use multiple systems due to varying
requirements.
Example: Grunt is more efficient for JavaScript, while Maven suits
Java projects.
• Rule of thumb: Developers should be able to build code locally
with minimal surprises.
The Role of Jenkins in Build Management
• Jenkins: A popular build server forked from Hudson,
created by Kohsuke Kawaguchi.
• Jenkins is versatile, supporting builds in Java and
other languages.
• Setup: Easy to install (e.g., on Fedora via dnf),
accessible at http://localhost:8080.
• Job Types:
- Freestyle project: Basic job type with build steps
like running shell commands (e.g., fortune or date).
- Other job types allow more complex setups (e.g.,
chaining jobs, pipelines).
• Jenkins builds and tests our software projects which continuously
making it easier for developers to integrate changes to the project,
and making it easier for users to obtain a fresh build.
• It also allows us to continuously deliver our software by
integrating with a large number of testing and deployment
technologies.
• Possible steps executed by Jenkins are for example:
• Perform a software build using a build system like Gradle or Maven
Apache
• Execute a shell script
• Archive a build result
• Running software tests
Work Flow:
Advantages of Jenkins
5. Containerization Layout
/var/lib/docker/
Used for container storage when working with Docker. It stores
images, containers, volumes, and other Docker-related data.
– containers/
– images/
– volumes/
6. Logs and Monitoring
/var/log/
Stores log files, crucial for debugging, performance
monitoring, and audits.
– syslog (system logs)
– jenkins/ (CI tool logs)
– nginx/ (web server logs)
– application/ (custom logs from applications)
7. Package/Artifact Storage
/var/artifacts/
Houses compiled binaries, Docker images, and other
deployable artifacts produced by the build process.
– docker/ (Docker images)
– maven/ (Maven-built JAR files)
8. Scripts
/usr/local/bin/
Contains executable scripts or binaries related to the DevOps
environment, including deployment scripts, monitoring scripts, or helper
tools.
deployment-scripts/ (automated scripts for deployments)
cleanup-scripts/ (scripts for managing resources, cleaning old builds)
9. Temporary Files
/tmp/
Temporary files used during builds or deployments. These files are usually
removed after a certain period or at the end of the build process.
ci_tmp/ (temporary files from CI/CD tools)
build_tmp/ (files created during build processes)
10. Virtual Environments and Dependency Management
/var/virtualenvs/ OR
/home/devops/.venvs/
For projects using Python or other interpreted languages, virtual
environments or environment management files are stored here.
project-1-env/
12. Shared Libraries
/usr/local/lib/
Stores shared libraries, including dependencies
and shared components used across multiple
applications or services.
custom-libraries/
13. CI/CD Tool Configurations
/etc/jenkins/ (or the respective directory for the CI
tool you are using)
Stores the configuration files for your CI/CD tools.
jobs/ (defines the pipeline configurations)
plugins/ (manages the plugins used in CI/CD tools)
Considerations:
• Security:
Sensitive files (e.g., secrets, keys, and passwords)
should be stored securely, often in a separate directory
with restricted permissions or using a secrets manager.
• Access Control:
Ensure proper file and directory permissions to restrict
access to critical parts of the system, especially in a
shared environment.
• Consistency:
Keep the layout consistent across environments (dev,
test, prod) to ensure smooth transitions and
deployment processes.
The host server
• The build server is usually a pretty important
machine for the organization.
• Builds shouldn't take too long, so you will need a
server with good specifications for the build server
—with lots of disk space, processor cores, and RAM.
• The build server also has a kind of social aspect: it is
here that the code of many different people and
roles integrates properly for the first time.
• Machines are cheaper than people, so don't let this
particular machine be the area you save money on.
Build Slaves and Build Pipelines
• Build slaves reduce queues by distributing
workloads to different machines.
• Example: Linux master, Windows/Mac slaves for
cross-platform builds.
• Upstream and downstream jobs:
- Upstream: Initial job that triggers further builds.
-Downstream: Jobs triggered after upstream
completion.
• Pipelines: Chaining jobs for complex workflows,
using plugins like Multijob and Workflow
Software on the host
• Depending on the complexity of your builds,
you might need to install many different types
of build tool on your build server.
• Remember that Jenkins is mostly used to
trigger builds, not perform the builds
themselves.
• Most of the build systems are available in the
distribution repositories, so it's very
convenient to install them from there.
Triggers
• You can either use a timer to trigger builds, or you can poll the code
repository for changes and build if there were changes.
• Git repository polling can be used most of the time so that every
check in triggers a build.
• An upstream build can trigger a downstream build. You can also let
the successful build of one job trigger another job.
Job chaining and build pipelines