© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.

KUB301

How to build scalable platforms

with Amazon EKS

Nirmal Mehta Isaac Mosquera John Weber

(he/him) (he/him) (he/him)
Principal Solutions Architect Head of Containers & Serverless GTM Senior Director Developer Platforms
Amazon Web Services Amazon Web Services Adobe

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Who we are

Nirmal Mehta Isaac Mosquera John Weber

Principal Solutions Head of Containers & Senior Director
Architect Serverless GTM Developer Platforms
Amazon Web Services Amazon Web Services Adobe

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
 Why are platforms abandoned?

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
 Logic

Authenticity Empathy

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
 Transparency

Collaborative Reliable

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
 How do we meet the
scale demands without
losing trust?

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Automation

Puppet Chef Ansible Terraform

2004 2006 2012 2014

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
 Platform team Development team

Network Infrastructure Observability Compliance App devs

Networking, Clusters Monitoring Tools Applications AWS resources

IAM

Auth Auth Auth Auth Auth Auth

Abstraction Abstraction Abstraction Abstraction Abstraction Abstraction

Policy Policy Policy Policy Policy Policy

Automation Automation Automation Automation Automation Automation

State State State State State State

AWS

Networking Amazon EKS Monitoring Security tools Application AWS resources

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Continuous compliance
 “Git is not the source
of truth; it’s the
source of hope”
Engineer's Incident Diary

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
18
 Platform team Development team

Network Infrastructure Observability Compliance App devs

Networking, Clusters Monitoring Tools Applications AWS resources

IAM

Auth Auth Auth Auth Auth Auth

Abstraction Abstraction Abstraction Abstraction Abstraction Abstraction

Policy Policy Policy Policy Policy Policy

Automation Automation Automation Automation Automation Automation

State State State State State State

AWS

Networking Amazon EKS Monitoring Security tools Application AWS resources

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
 Platform team Development team

Network Infrastructure Observability Compliance App devs

Networking, Clusters Monitoring Tools Applications Dependencies

IAM

Automation Automation Automation Automation Automation Automation

AWS

Networking Amazon EKS Monitoring Security tools Application AWS resources

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
 “Any improvements made
anywhere besides the
bottleneck are an illusion.”
― Gene Kim, The Phoenix Project
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Towards platform engineering

Capability 1 Capability 2 Capability N

API Auth Abstraction Automation Policy Event bus State

Platform framework

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
 … …
Team 1 Team N Business unit 1 Business unit N

Consume
Feature requests

Capability 1 Capability 2 Capability N

API Auth Abstraction Automation Policy Event bus State

Platform
Prioritization

Network Infrastructure CI/CD

Observability Compliance

Platform engineers

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
 … … Builds

Team 1 Team N Business unit 1 Business unit N

Consume
Feature requests

Capability 1 Capability 2 Capability N E F

API Auth Abstraction Automation Policy Event bus State

Platform Platform
Prioritization

Network Infrastructure CI/CD

Observability Compliance

Platform engineers

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
 … … Extends

Team 1 Team N Business unit 1 Business unit N

Consume
Feature requests

Capability 1 Capability 2 Capability N E F

API Auth Abstraction Automation Policy Event bus State

Platform
Prioritization

Network Infrastructure CI/CD

Observability Compliance

Platform engineers

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
 Facilitation X as a service Collaboration
helping or teach another team Consuming or providing Building and working closely
to clear impediments something with minimal together with another team
collaboration

ORGANIZATIONAL SCALABILIT Y

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What if we used an OSS framework?

Capability 1 Capability 2 Capability N

API Auth Abstraction Automation Policy Event bus State

Platform framework

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What if we used Kubernetes
as the platform framework?
Controller 1 Controller 2 Controller N

Events & reconciliation

API Mutating Schema Validation Persist

RBAC ETCD
handler admission validation admission

Mutation

Decision
Kubernetes control plane

Webhook Webhook
controller controller

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Composable abstractions
Blue/Green
Deployments

ReplicaSets
… ReplicaSets

Pods … Pods

Containers … Containers

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Composable abstractions

App

Blue/Green Ingress Bucket Ticket

ReplicaSets
… ReplicaSets IAM

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Composable abstractions

Environment

Account VPC EKS Ticket

Subnet … Subnet Node Group … Node Group

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Composable abstractions

App Environment Ticket

Blue/Green Ingress Bucket Ticket

ReplicaSets
… ReplicaSets IAM

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Developers care about apps, not infrastructure

App Environment Ticket

Blue/Green Ingress Bucket Ticket

ReplicaSets
… ReplicaSets IAM

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Large controller ecosystem

Custom
controller
Operator SDK
FUNCTIONALITY

Crossplane

ACK

OPA

Flux Kyverno
External Istio
External
secrets
DNS
ArgoCD

Volumes
ELB ALB

EFFORT

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Evolving how technical teams collaborate
Capabilities

App team
architects
Platform
engineer
Guardrails
review build consume
API
Database
architects Git repo Developers
Auth

Permissions Abstraction

Security Policy

Policies Automation

Compliance State

Controllers

Community

Blueprints
AWS
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Building a cloud platform with Kubernetes
Central AWS account provision
6
status 7
API
EKS management cluster

System namespace Tenant namespace

monitors
2 Controller 5
OPA Resource CRDs
git push

3
1

K8s control plane

8 4 ETCD
Devs
API server Admission controller

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
36
Key takeaways
Community & collaborate

Product mindset

Abstract complexity with APIs

Leverage OSS for transparency and

Reduce your undifferentiated lifting

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Adobe Ethos

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Long wait & cycle times
It takes new developers more than a month* to get an existing service into a production-ready state. Below is
a breakdown of the various checkpoints in that journey
~70% of the total time

Zero to Training & Third-party Internal tools Client API setup Third-party Production
hello world tutorials system tickets (CI/CD, & connections system wait readiness
& setup Kubernetes, etc.) times

Internal developers traverse across at least 20 internal/external tools/portals leading to

high cognitive load and wait times
“I would like us to make integration to other services simple and easy for secret
management, logging, db, alerts, and gateway onboarding to name a few.”
Adobe internal developer
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. *Most of this time was captured manually by going through the production-readiness journey
Adobe’s services landscape
Our Motto: Help Developers Write Better Software Faster

Document cloud Creative cloud Experience cloud

Service
Adobe products & services Service

Sensei Data More

Content
ML platforms platforms
platform Service Service Service Service
platform

Service
Ethos Service

AWS Cloud provider Data center

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Adobe’s journey
Adobe Flex
• Flexible & paved path CI/CD
Platform champions • GitOps-based, Argo-powered
• Adobe’s IDP Workshop
Platform Champion Model
during ArgoCon 2022
enabled 1000+ services on
• Flex launch
Apache Mesos

2015–2017 2019–2021 2023–24

2018 2022 2024–26

Ethos Migration to K8s Developer portal & flex scalability

Early bets on Docker + Apache • CaaS: Paved-path CICD
Mesos with abstraction and • PaaS: Do it yourself (DIY) • Developer portal powered by Backstage
opinionated CICD • Flex: Vertical and horizontal scalability
(Moonbeam) • Flexible + opinionated secure CICD with GHA
• Generative AI–powered support bot

Blog: https://blog.developer.adobe.com/how-ethos-powers-a-cloud-native-transformation-at-adobe-16c1a2e2f67a
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Ethos overview

Ethos is a developer platform that gives Adobe developers key

capabilities to build reliable, performant, and scalable services

Cloud native Simplified developer Security & compliance Efficiency w/built-in

infrastructure experience w/Adobe requirements measurement,
reporting, & optimization

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
We are a product team

Internal customers (yes, they are customers!)

• Clear, concise documentation
• Intelligent UX
• Low barriers to entry

External customers
• “Dial tone” for the company

AWS
• Aligned roadmap and shared incentives

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Building a community

Platform champions ”Inner Source” model Customer advisory boards

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
OpenDev in numbers

More contributions
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
OpenDev in numbers

Outside contributors

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Accountability and transparency

Product ownership You build, you run Cost attribution and

chargeback

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Automatic resource configurator (ARC)

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reliability and resiliency

SLIs/SLOs Amazon EKS – RCAs

Fully supported by AWS

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Error budgets

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Error budgets made simple(r)

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A multifaceted investment
Benefits

Control 10:1 30:1

plane
Cluster-to-operator ratio

CI/CD

Build and deploy options

Developer 6 1
experience
User interfaces

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How to win!
Developer
01 velocity
Deploy a new service in 3 days (from 30 days)

02 Reliability 75% improvement in MTTD

Cost
03 optimizations
Targets achieved!

Developer
04 sentiment
71% positive

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cloud-native operational excellence: CNOE.io

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Attractive platform takeaways!

Community & collaborate Transparency

Abstract complexity

Foster transparency

Product mindset Collaborative Reliable

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Check out these other sessions
KUB404: Building production-grade resilient architectures with Amazon EKS
Monday (Dec 2) @ 10:00 AM Mandalay Bay | Lower Level North | Islander F*

KUB402: Amazon EKS: Infrastructure as code, GitOps, or CI/CD

Wednesday (Dec 4) @ 1:00pm – MGM, 305*

KUB312: Automated cluster infrastructure with Amazon EKS and Karpenter

Wednesday (Dec 3) @ 2:30pm – MGM, Chairmans 355*

KUB201: The future of Kubernetes on AWS

Thursday (Dec 5) @ 11:30am – MGM, Grand 122*

KUB308: IDP fast track: Racing to deploy with CNOE for enterprise DevOps
Thursday (Dec 5) @ 3:00 PM - 5:00 PM - Mandalay Bay | Level 2 South | Oceanside A*

*Times and locations are subject to change; check session catalog for up-to-date information
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Continue your Amazon EKS learning
Learn at your Increase your Earn Amazon
own pace knowledge EKS badge

Take the Amazon EKS Use our Best Practices Guide Demonstrate your
Workshop to expand to build your Kubernetes knowledge by achieving
your EKS skills knowledge digital badges

https://github.com/aws-samples/reinvent24

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Session resources

https://github.com/aws-samples
/reinvent24/tree/main/sessions/KUB301

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you! Please complete the session
survey in the mobile app

Nirmal Mehta Isaac Mosquera John Weber

Nirmalkmehta [email protected] jweber93
[email protected]

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.