Databricks Cluster
Databricks Cluster
Cluster Types
Cluster Configuration
Creating a cluster
Pricing
Cost Control
Cluster Pools
Cluster Policy
DATABRICKS CLUSTER
VM
Driver
VM VM VM
Created
Created by Jobs
manually
Terminated at the end of the job
Persistent
Suitable for automated workloads
Single Node
CLUSTER CONFIGURATION
Shared
Multiple User Access
Only available in Premium. Supports Python, SQL
No Isolation Shared
Multiple User Access
Supports Python, SQL
Custom
Legacy Configuration
CLUSTER CONFIGURATION
Single/ Multi Node Databricks Runtime
Scala, Java, Ubuntu GPU
Spark Delta Lake
Access Mode Python, R Libraries Libraries
Other Databricks Services
Databricks Runtime
Databricks Runtime ML
Everything from Popular ML Libraries (PyTorch, Keras,
Databricks runtime TensorFlow, XGBoost etc)
Photon Runtime
Everything from
Photon Engine
Databricks runtime
Databricks Runtime
Databricks Runtime
Auto Termination
Auto Termination
Compute Optimized
Auto Scaling
Storage Optimized
Cluster VM Type/ Size
General Purpose
GPU Accelerated
CLUSTER CONFIGURATION
Single/ Multi Node
Access Mode
Databricks Runtime
Auto Termination
Auto Scaling
Cluster Policy
CLUSTER CONFIGURATION
Single/ Multi Node
Access Mode
Databricks Runtime
Auto Termination
Auto Scaling
Cluster Policy
CLUSTER CONFIGURATION
Single/ Multi Node
Access Mode
Simplifies the user interface
Databricks Runtime
Auto Scaling
Achieves cost control
Cluster VM Type/ Size
Only available on premium tier
Cluster Policy