Composable Systems in

Kubernetes
Michele Gazzetti
Software Engineer, IBM Research Europe
Agenda

1. Composable Disaggregated Infrastructure (CDI)

2. The challenge of managing Composable Disaggregated infrastructure (CDI) in Kubernetes

3. Sunfish

4. Composable Resource Operator: A Proof of Concept composing GPUs

5. Demo

6. Summary and next steps

Composable Disaggregated Infrastructure (CDI)

• A set of disaggregated compute, API

memory, storage and network hardware
elements that can be assembled to CDI SW Management
Composable

produce compute servers on-demand. Resource

Pools
• Software to provision and manage
the underlying physical resources, to
assemble and dynamically reconfigure Fabric
servers.
Composable Disaggregated Infrastructure (CDI)

Traditional Infrastructure

Kubernetes

Worker Node Worker Node Worker Node

Composable Disaggregated Infrastructure (CDI)

1 1 1 ✅
Traditional Infrastructure

Kubernetes

Worker Node Worker Node Worker Node

Composable Disaggregated Infrastructure (CDI)

1 1 3 ⏳
Traditional Infrastructure

Kubernetes

Worker Node Worker Node Worker Node

Composable Disaggregated Infrastructure (CDI)

1 1 3 ⏳ 1 1 3 ✅
Traditional Infrastructure Composable Disaggregated Infrastructure (CDI)

Kubernetes Kubernetes

Worker Node Worker Node Worker Node Worker Node Worker Node Worker Node

Resource Pool
Infrastructure
Benefits and Challenges of CDI

Benefits of CDI:
Kubernetes
• Compose high value resources to fit
workload requirements Worker Node Worker Node Worker Node

• Add resources without the need of

adding CPUs and Memory

• Avoid intrusive reconfigurations

Composable
hardware
Resource Pool
Infrastructure
Benefits and Challenges of CDI

Benefits of CDI:
Kubernetes
• Compose high value resources to fit
workload requirements Worker Node Worker Node Worker Node

• Add resources without the need of

adding CPUs and Memory

• Avoid intrusive reconfigurations

Challenge of CDI: increased

Composable
hardware
management complexity

Resource Pool
Infrastructure
CDI In Kubernetes

Benefits of integrating CDI in K8s: Kubernetes

• Request resources via CRDs Worker Node Worker Node Worker Node

• Attach/detach resources safely

• Hide infrastructure complexity

Composable
hardware
Resource Pool
Infrastructure
The Complexity Of Managing CDI Resources

Kubernetes

API hardware API hardware API

CDI Management
Composable
CDI Management
Composable CDI Management
Sunfish

Sunfish is the OpenFabricsAlliance’s response to managing complex composable

systems.

A framework empowering infrastructure clients with:

• Unique API based on DMTF’s Redfish® and SNIA Swordfish®

• Access to the entire CDI infrastructure

• Dynamic (re-)configuration of infrastructure resources

• Mechanism to support multiple CDI Providers

Sunfish

Kubernetes
Resource
Sunfish Composability Layer abstraction and
selection

Sunfish

API API API

hardware
CDI Management Layer
Composable CDI Management Layer CDI Management Layer
Sunfish

Kubernetes
Resource
Sunfish Composability Layer abstraction and
selection

Sunfish

API API API

hardware
CDI Management Layer
Composable CDI Management Layer CDI Management Layer
Composable Resource Operator

A PoC to integrate composable resources in

Kubernetes

• Request resources via the Composability

Request CRD

• Handles resource attachment/detachment

• Devices tested: Nvidia GPUs (V100, A100)

https://github.com/IBM/composable-resource-operator
Composable Resource Operator

ComposabilityRequests

Composable Resource Operator

Kubernetes

Sunfish Composability Layer

Sunfish

API API API

hardware
CDI Management Layer
Composable CDI Management Layer CDI Management Layer
Composability Request CRD
Demo

Pending
kubectl apply nvidia-smi-pod.yaml
Nvidia-smi-pod

composable04

allocatable resources:
"cpu": …,
"memory": …,
"nvidia.com/gpu": ”0",
Demo

Completed
kubectl apply composabililtyrequest.yaml
Nvidia-smi-pod

composable04

allocatable resources:
"cpu": …,
"memory": …,
"nvidia.com/gpu": ”4",
Demo

Watch for allocatable resources

on the Node

Execute kubectl commands

Watch for GPU devices via lspci

Composability Request Creation

Composable Resource K8s Node X

Operator
Create CR: Sunfish Client
Node: X
gpu:
Size: 4 NFD worker
Model: Y

Sunfish

CDI Management
Composability Request Creation

Composable Resource pci-<device>.present: true K8s Node X

Operator
Composable Resource
Operator
Create CR: Sunfish Client
Node: X
gpu:
Composition NFD worker
Size: 4 Request Inspect/select
Model: Y resources to Compose

Sunfish

Translate request in
vendor specific APIs

CDI Management
Compose resources
Composability Request Creation

Wait for update in

allocatable resources

Composable Resource pci-<device>.present: true K8s Node X

Operator
Composable Resource
Operator allocatable: { gpu: 4 }
Create CR: Sunfish Client
Node: X Device driver
gpu:
Composition NFD worker
Size: 4 Request Inspect/select
Model: Y resources to Compose

Sunfish

Translate request in
vendor specific APIs

CDI Management
Compose resources
Composability Request Creation

Wait for update in

allocatable resources

Composable Resource pci-<device>.present: true K8s Node X

Operator
Composable Resource Pod Pod
Operator allocatable: { gpu: 4 }
Create CR: Sunfish Client
Node: X Device driver
gpu:
Composition NFD worker
Size: 4 Request Inspect/select
Model: Y resources to Compose

Sunfish

Translate request in
vendor specific APIs

CDI Management
Compose resources
Hard Lesson Learned On Resource Detachment

Composable Resource Pci-<device>.present:

pci-<device>.present: true K8s Node X
Operator
Composable Resource Pod Pod
Operator allocatable: { gpu: 4 }
Delete CR Sunfish Client
Device driver
Undo
Composition NFD worker
Request

Sunfish

Translate the request in

vendor specific APIs

CDI Management
Detach resources
Hard Lesson Learned On Resource Detachment

Composable Resource K8s Node X

Operator
Composable Resource Pod Pod
Operator
Delete CR Sunfish Client
Device driver
Undo
Composition NFD worker
Request

Sunfish
❌
Error: GPU has fallen off
Translate the request in
vendor specific APIs the bus 🚌 ⤵
CDI Management
Detach resources
Composability Request Deletion

Composable Resource pci-<device>.present: true K8s Node X

Operator
Composable Resource Pod Pod
Operator allocatable: { gpu: 4 }
Delete CR Sunfish Client
Device driver

NFD worker

Sunfish

CDI Management
Composability Request Deletion

1. Set the node

Unschedulable

Composable Resource pci-<device>.present: true K8s Node X

Operator
Composable Resource Pod Pod
Operator allocatable: { gpu: 4 }
Delete CR Sunfish Client
unschedulable Device driver

NFD worker

Sunfish

CDI Management
Composability Request Deletion

1. Set the node

Unschedulable

2. Evict
Composable Resource GPU pci-<device>.present: true K8s Node X
Operator
Composable Resource workloads
Operator allocatable: { gpu: 4 }
Delete CR Sunfish Client
unschedulable Device driver

NFD worker

Sunfish

CDI Management
Composability Request Deletion

1. Set the node

Unschedulable

2. Evict
Composable Resource GPU pci-<device>.present: true K8s Node X
Operator workloads
unschedulable
Delete CR Sunfish Client 3. Force driver
removal
NFD worker

Sunfish

CDI Management
Composability Request Deletion

1. Set the node

Unschedulable

2. Evict
Composable Resource GPU K8s Node X
Operator workloads
unschedulable
Delete CR Sunfish Client 3. Force driver
removal
4. Undo
Composition NFD worker
Request

Sunfish

Translate the request in

vendor specific APIs

CDI Management
Detach resources
Composability Request Deletion

1. Set the node

Unschedulable

2. Evict
Composable Resource GPU K8s Node X
Operator workloads

Delete CR Sunfish Client 3. Force driver

removal
4. Undo
Composition NFD worker
Request 5. Set node schedulable

Sunfish

Translate the request in

vendor specific APIs

CDI Management
Detach resources
Summary And What’s Next

Important aspects working on this PoC:

• No more assumptions, resources might come and go.
• All levels of the stack should have coherent name and metadata on devices.
• Adding/removing resources is the first step, we need to react to workload requirements!

Potential next steps:

• Embrace the approaches outlined by the community (Dynamic Resource Allocation).
• Keep client compatibility as Sunfish works towards its first official release.
• Explore more clients, more vendors, more types of resources (Disk and memory).
• Inspect the impact on sustainability.
• More…
If You Want To Know More

Links:

• Sunfish: https://www.openfabrics.org/openfabrics-management-framework/

• Composable Resource Operator: https://github.com/IBM/composable-resource-operator

Thank you!

Please scan the QR Code to

leave a feedback on this session.