Scribd
Upload
73 views120 pages

Build Your Very Own Private Cloud Foundry: Matt Stine Community Engineer, Cloud Foundry Twitter: @mstine

This document provides an overview of BOSH (Bosh Outer SHell) and how it can be used to deploy and manage Cloud Foundry on different infrastructure platforms like vSphere, AWS, and OpenStack. It explains the key components of BOSH like stemcells, releases, jobs, and deployment manifests. It also demonstrates how to deploy Cloud Foundry using BOSH on vSphere, AWS, and OpenStack through examples.

Uploaded by

Srinivasa Helavar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
73 views120 pages

Build Your Very Own Private Cloud Foundry: Matt Stine Community Engineer, Cloud Foundry Twitter: @mstine

This document provides an overview of BOSH (Bosh Outer SHell) and how it can be used to deploy and manage Cloud Foundry on different infrastructure platforms like vSphere, AWS, and OpenStack. It explains the key components of BOSH like stemcells, releases, jobs, and deployment manifests. It also demonstrates how to deploy Cloud Foundry using BOSH on vSphere, AWS, and OpenStack through examples.

Uploaded by

Srinivasa Helavar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 120

BUILD YOUR VERY OWN

PRIVATE CLOUD FOUNDRY

MATT STINE
COMMUNITY ENGINEER, CLOUD FOUNDRY
[email protected]
HTTP://CLOUDFOUNDRY.ORG
TWITTER: @MSTINE

Agenda
CF Architecture - Quick Overview
What is BOSH? Why BOSH?
BOSH Architecture
BOSH Releases
BOSH Deployments
CF to BOSH: cf-release
[:vsphere, :openstack, :aws].each { |iaas| iaas.deploy_cf }

Cloud Foundry Architecture


(in FIVE minutes or less!)

Cloud Foundry Architecture


Router
User Authen=ca=on and Authoriza=on
Health Manager
Cloud Controller
Apps
Service Broker
Service Nodes

Messaging

Build Packs
DEA Pool

Router
e.
co
m
kt
g.
ac
m
ap
p1
.m

DEV
A

TEST
S

m
co
e.
m
ac
g.

MARKETING ORG

n
.e
p2
ap

Load balancing
Maintain routing table
Access logs

app3.finance.acme.co
m

ROUTER

ENGINEERING ORG

FINANCE ORG

PROD
S

DEV
A

TEST
A

PROD

DEV
A

PLATFORM

TEST

PROD
A

UAA
Token Server
ID Server (User management)
OAuth Scopes (Groups)
Login Server
UAA Database
SAML support (for SSO integration) and Active Directory support
with the VMWare SSO Appliance

Access auditing

Cloud Controller
App expected state
Permissions/Auth
Orgs/Spaces/Users
Services management
App placement
App desired state convergence
Auditing/Journaling
Billing events
Blob storage

Messaging (NATS)
Non-Persistent messaging
Pub/Sub
Queues (app events)
Directed messages (INBOX)

Droplet Execution Agent (DEA)


Manage Linux containers (Warden)
Process
File system
Network
Memory
Manage app lifecycle
App log and file streaming
DEA heartbeats (NATS to CC, HM)

Router

App

App

App

App

DEA/Warden

DEA/Warden

Health Manager
HealthManager
SCHEDULER

Maintains the actual state of


apps

Sends suggestions to make

Compares to expected state

actual match expected

App

App
Schedule
Observation

App

App

HARMONIZER
Observes

Observes

App

Trigger
Harmonization

Desired
State

NUDGER

Bulk API

App

Actual
State
NATS

NATS
Cloud
Controller

NATS

DEA

Service Broker
Advertising service catalog
Makes create/delete/bind/unbind calls to service
nodes

Requests inventory of existing instances and bindings

from cloud controller for caching, orphan management

SaaS marketplace gateway

CF Architecture: Why?
Component Isolation
Scalability
Fault Tolerance
Pre-provisioned Capacity (Containers / Warden)

How do we manage this thing?

We need a toolchain that can manage


a large distributed system through:
Deployment
Configuration Changes
Updates/Upgrades (w/ minimal - zero - downtime!)
Component Failure / Restoration of Service
Scale Out / Scale In
Across multiple IaaS providers: vSphere, OpenStack, AWS,
and beyond...

What about?

BOSH!
(Bosh Outer SHell)

What is BOSH?

Why BOSH?
Provision services, not machines
Eliminate bespoke automation on top of configuration
management

Enable continuous delivery of platform services


Cloud-agnostic view of platform operations
Holistic toolchain to rule them all
How we manage Cloud Foundry in production!

BOSH Architecture

BOSH Architecture

Director
Core orchestrating component
Controls:
creation/deletion of VMs, Disks, Networks
software deployment
lifecycle events for software and services
Via:
Cloud Provider Interface
Director-Agent Interaction

Agent
Carries out instructions from the Director
Fetches packages/jobs from Blobstore
Installs packages/jobs onto VM
Starts/monitors jobs via Monit

Message Bus (NATS)


Non-Persistent messaging
Pub/Sub
Queues (app events)
Directed messages (INBOX)

Health Monitor
Receives from Agent:
VM health status
Agent lifecycle events
Triggers:
Alerts (Email, CloudWatch, DataDog, PagerDuty, ...)
Resurrection!

Stemcells
Pleuripotent VMs
Base VM filesystem image
Never booted
Currently ~Ubuntu 10.04.4 LTS
Embedded BOSH Agent

Blobstore

Stores the content of BOSH Releases


Uploaded via BOSH CLI
Stored via Director
Also stores:

packages compiled by BOSH


intermediate location for large payloads

Options:

EMC Atmos
S3
OpenStack Swift
Simple/Local

Cloud Provider Interface (CPI)


current_vm_id
create_stemcell
delete_stemcell
create_vm
delete_vm
has_vm?
reboot_vm
set_vm_metadata

configure_networks
create_disk
delete_disk
attach_disk
snapshot_disk
delete_snapshot
detach_disk
get_disks

Great! So how do I deploy my stuff


with BOSH?

BOSH Releases

Hello BOSH Release!

Redis

Anatomy of a BOSH Release


Release Manifest
Package
spec
packaging
Job
spec
monit
templates

Release Manifest

redis-boshrelease/releases/redis-1.yml

Package

redis-boshrelease/packages/redis/spec

Packages

redis-boshrelease/packages/redis/packaging

Jobs

redis-boshrelease/jobs/redis/spec

Jobs
http://mmonit.com/monit

redis-boshrelease/jobs/redis/monit

Jobs

redis-boshrelease/jobs/redis/templates/bin/redis_ctl

Jobs

redis-boshrelease/jobs/redis/templates/conf/redis.conf.erb

OK. So what happens when I


deploy?

BOSH Agent

Director creates VM from Stemcell

Director

VM

CPI

Agent Boots
Director

VM

Director Pings Agent


Director

NATS

VM

Director Assigns Job to Agent


Director

NATS

VM

Agent Fetches Blobs from Blobstore

VM

HTTP

Blobstore

Agent Starts Jobs


http://mmonit.com/monit

VM

Now lets go outside-in!

BOSH Deployments

Deployment Manifests
The mapping of a BOSH release to infrastructure
Specifies:
Release
Compilation VMs
Update (canary) settings
Network configuration
Resource pools (VM templates)
Jobs
Job Properties

redis-boshrelease/examples/aws-solo.yml

redis-boshrelease/examples/aws-solo.yml

redis-boshrelease/examples/aws-solo.yml

redis-boshrelease/examples/aws-solo.yml

cf-release

Lets follow one of our


components...
GoRouter
Release Manifest
Package
spec
packaging
Job
spec
monit
templates

Release Manifest

cf-release/releases/cf-release-134.yml

Package

cf-release/packages/gorouter/spec

Package

cf-release/packages/gorouter/packaging

Job

cf-release/jobs/gorouter/spec

Jobs
http://mmonit.com/monit

cf-release/jobs/gorouter/monit

Jobs

cf-release/jobs/gorouter/templates/gorouter_ctl.erb

Jobs

cf-release/jobs/gorouter/templates/gorouter_yml.erb

Cloud Foundry on vSphere

vSphere 101
Minimum Lab HW:
6 Cores (12 threads)
64 GB RAM
4 TB Disk
vSphere 5.1 (2 ESXi hosts, 1 vCenter)
can get 30 day evals!
iSCSI Data Store (http://www.openfiler.com/)
Available IP addresses = 2X number of VMs
https://github.com/cloudfoundry-community/vsphere-home-lab

BOSH Bootstrap
Provide vCenter Credentials (via http://fog.io/)
Provide Network Settings
Provide vCenter Settings (Datacenter, Cluster, ...)
Provide NTP Settings
Provide MicroBOSH VM Sizing
Provision MicroBOSH!

BOSH Bootstrap Deploy

BOSH Prepare CF

vSphere Deployment Manifest

vSphere Deployment Manifest

vSphere Deployment Manifest

vSphere Deployment Manifest

http://xip.io

BOSH Create CF

Cloud Foundry on AWS

AWS 101
Requirements:
AWS Credentials
Capacity to provision 8 servers and 3 elastic IPs
Approximately $0.42/hour you keep it running (useast-1/us-west-1)

Manually configured security group for CF

BOSH Bootstrap
Provide AWS Credentials (via http://fog.io/)
Provide AWS Region
Provision MicroBOSH!

BOSH Bootstrap Deploy

BOSH Prepare CF

AWS Deployment Manifest

AWS Deployment Manifest

AWS Deployment Manifest

AWS Deployment Manifest

BOSH Create CF

Create cf Security Group

Allocate Elastic IP for CF Router

Cloud Foundry on OpenStack

OpenStack 101
Requirements:
OpenStack Credentials
Capacity to provision 8 servers and 3 floating IPs
Manually configured security group for CF

BOSH Bootstrap
Provide OpenStack Credentials (via http://fog.io/)
Provide OpenStack Region (optional)
Provision MicroBOSH!

BOSH Bootstrap Deploy

BOSH Prepare CF

OpenStack Deployment Manifest

OpenStack Deployment Manifest

OpenStack Deployment Manifest

OpenStack Deployment Manifest

BOSH Create CF

Create cf Security Group

Allocate Floating IP for CF Router

Web Console UI

Edit src/main/resources/
styx.properties

mvn clean package

cf push!

THANK YOU!

MATT STINE
COMMUNITY ENGINEER, CLOUD FOUNDRY
[email protected]
HTTP://WWW.CLOUDFOUNDRY.COM
TWITTER: @MSTINE

You might also like