Cisco ACI

Key benefits of Cisco ACI

• Automation of IT workflows and application deployment agility

• Open APIs and a programmable SDN fabric, with 65-plus ecosystem partners
• Security through allow lists, policy enforcement, microsegmentation, and analytics
• Workload mobility at scale for physical and virtual load
 Cisco ACI vs Traditional Network

Loop free Environment

• Since there is IP reachability between leaf and spine switches, there is no need for STP, and you do not have to block any port to
avoid the Layer 2 loops
 Cisco ACI vs Traditional Network

More Secure

• From security perspective, in a traditional network device, you usually allow all the traffic by default, or you explicitly configure
the device to block the traffic. However, in Cisco ACI, an allow list model is used. By default, everything is blocked, unless you
explicitly allow the traffic.
 Cisco ACI vs Traditional Network

Centralized Management
• Instead of using SSH to each and every device to configure and build the Cisco ACI fabric there is a centralized controller called
APIC.

• In Cisco ACI, there are typically three Cisco APIC controllers, and they form an APIC cluster. If you lose one of them, you can
still change and add new configurations through the remaining two controllers.

• Another very important point in the Cisco APIC is that it enables access via Cisco API.
Cisco ACI Topology and Hardware
 Cisco ACI vs Traditional Network

Topology Simplicity

• In the Cisco ACI, a spine-leaf topology is used . There is no leaf-to-leaf and no spine-to-spine connectivity.

• The physical devices that are used in Cisco ACI Fabric are Cisco Nexus 9000 switches. ACI operating system Not NXOS
 Cisco ACI fabric

The Cisco ACI fabric is composed of the Cisco APIC and the Cisco Nexus 9000 Series spine and leaf switches.
The leaf switches are connected to the spine switches, but never to each other. The spine switches are attached
only to the leaf switches. The Cisco APIC and all other endpoints and devices in the data center are connected to
the leaf switches only as seen from the following figure.
 Spine-Leaf Topology Benefits

In summary, the advantages of the spine-leaf topology include:

Simple and consistent topology
Scalability for connectivity and bandwidth
Symmetry for optimization of forwarding behavior
Least-cost design for high bandwidth
 IS-IS Fabric Infrastructure Routing

The fabric applies a densely tuned Intermediate System-to-Intermediate System (IS-IS) environment utilizing Level
1 connections within the topology for advertising loopback addresses. Loopback addresses are the Virtual Extensible LAN
(VXLAN) Tunnel Endpoints (TEPs) that are used in the integrated overlay and advertised to all other nodes
in the fabric for overlay tunnel use.
IS-IS is responsible for infrastructure connectivity in Cisco ACI:
IS-IS provides IP reachability among TEP addresses
Automatically deployed, no user intervention is required
No IS-IS knowledge is required
 Main components of Cisco ACI:

ACI is made up of 3 main components

• Spine Switches

1. Represent the backbone of the ACI fabric

2. Connected to leaf switches
• Leaf Switches

3. Represent connection point for end devices, including APIC

4. Connected to spine switches
• Application Policy Infrastructure Controllers (APIC)

5. Unified point of policy enforcement, health monitoring, and management for the Cisco ACI fabric
6. Not involved in data plane forwarding
 Cisco ACI Fabric

• Spine And Leaf Architecture

• 40 Gbps Cable Connects Every Spine with Every Leaf
• Can Be upgraded to 100 Gbps

• Spine Switches (Nexus 9500 – Nexus 3360)

• At least 2 spines for redundancy. Spine
• Used to interconnect the leafs
40 Gbps
QSFP
• Leaf Switches Nexus 9300
• End Host connected here
Leaf
• Leaf should be connected to spine only

• APIC Controller
• Minimum of 3 clusters
• Can scale up to 7 Cluster
• Odd number of Nodes only allowed on
Cluster ,3,5,7
• Should be Connected to Leaf APIC
App DB
Server Server
 • Spine Leaf Underlay
• IS-IS
• Advertise VTEP Addresses
• The leaves and spines will exchange IS-IS routing
• Equal Cost Multipath.

• COOP Spine
• Each Leaf store Full ARP Entry into local station Table
40 Gbps
• Leaf will reports this information to one of the spine
QSFP
switches (chosen at random) using the
Council Of Oracles Protocol (COOP).
Leaf
• Spine will relay ARP information to other spines switches to
be stored at (Global Station Table)

MP-BGP
• Required for outside the fabric L3 Connectivity

APIC
App DB
Server Server
 ACI APIC
• APIC
Cisco APIC is a policy controller. It relays the intended state of the policy to the fabric. The APIC does not represent
the control plane and does not sit in the traffic path. The hardware consists of a cluster of three or more servers in a
highly redundant array

• APIC is the policy controller

• No Impact if APIC crashed Spine
• At least three APIC on the topology 40 Gbps
• APIC are Active/Active QSFP
• Database is shaded Between Cluster.
Leaf

10 Gbps

APIC APIC APIC

 Cisco APIC Redundancy

• APIC
• Cisco APIC is redundant on multiple levels, including interface-level and cluster-level.
• Cisco APICs use a bonded interface that is typically dual-homed to two leaf switches for connectivity to the Cisco ACI fabric, and
have the ability to use a bonded interface that can be dual-homed to the out-of-band management network. There are two bond
interfaces:
1. Bond0 is used to connect to the fabric itself (to connect to leaf switches that connect into the fabric). It is recommended to
connect two fabric uplinks, each to a separate leaf and/or vPC Leaf pairs.
2. Bond1 is used to connect to the out-of-band (OOB) segment (that allows setup of the APIC itself).
 Cisco APIC Cluster
• Cisco APIC is deployed in a cluster with a minimum of three controllers
• The APIC cluster uses a large database technology called sharding. The APIC configuration database is partitioned into logically
bounded subsets, called shards, and each shard has three replicas. The shards are evenly distributed across the APICs.
 Cisco Nexus 9000 Series Hardware

Cisco Nexus 9000 Series family consists of these elements:

• Cisco Nexus 9500 Series modular chassis
• 4-slot, 8-slot, and 16-slot
• Support 10GE, 40GE, 100GE, and 400GE modules
• Cisco Nexus 9500 Series line cards
• Cisco Nexus 9300 Series top-of-rack (ToR) leaf and spine switches
• Cisco ACI spine and leaf varieties
• 1/10/40/100/400 GE interface speeds
Cisco Nexus 9500 Platform Components
 Cisco Nexus 9300 ACI Fixed Spine Switches

Several Cisco Nexus 9300 models can be deployed as Cisco ACI spine switches:
 Cisco Nexus 9364C (64x 100/40GE)
 Cisco Nexus 9332C (32x 100/40GE)
 Cisco Nexus 9336PQ (36x 40GE)
 Cisco Nexus 9316D-GX (16x 400GE)
 Cisco Nexus 9300 ACI Fixed Spine Switches

Cisco ACI leaf spectrum includes:

 40/100-GE switch examples:
1. Cisco Nexus 9336C-FX2 (36x 40/100GE)
2. Cisco Nexus 93180LC-EX (24x 40/50GE, 6x 40/100GE)
3. Cisco Nexus 9332PQ (32x 40GE)
Cisco ACI Fabric Discovery
• Initializing Discovery
• Initializing Discovery
 • Fabric Discovery

Fabric Discovery
• LLDP is used to Discover Leaf (Neighbor Discovery)
• DHCP is used to assign IP Address for VTEP
• IFM messages used as heartbeat and for policy pushing
• IS-IS is automatically starts to create VTEP adjacency. Spine

40 Gbps
QSFP

Leaf

10 Gbps

APIC APIC APIC

• Initializing Discovery
Cisco ACI Access Policies
 Cisco ACI Access Policies

Access policies configure external-facing interfaces that connect to devices such as hypervisors,
hosts, network attached storage, routers, or FEX interfaces. Access policies enable the configuration of
port channels and virtual port channels, protocols such asLink Layer Discovery Protocol (LLDP),
Cisco Discovery Protocol (CDP),or Link Aggregation Control Protocol (LACP), and features such
as statistics gathering, monitoring, and diagnostics.
 Cisco ACI Access Policies

•Pools: Specify VLAN and multicast address pools.

•Interface profiles: Specify which access interfaces to configure and the interface configuration policy.
•Switch profiles: Specify which switches to configure and the switch configuration policy.
•Global policies: Enable the configuration of DHCP, QoS, and Attachable Access Entity Profile (AAEP).
•Physical and external domains: Define a domain that bundles a set of interfaces (AAEP) and encapsulations (VLAN pool)
to allow other components.
•Monitoring and troubleshooting policies: Specify what to monitor, the thresholds, how to handle faults and logs, and how
to perform diagnostics related to external facing interfaces.
 Interface Policies, Switch Policies, and Profiles

• With interface policies, you can perform these actions:

• Create protocol policies to configure interface protocols and parameters.
• Choose a port connection type, such as vPC with an interface policy group.
• Group ports with an interface profile.
•With switch policies, you can perform these actions:
• Create or modify protocol policies at switch level with a Switch Policy.
• Group one or more leaf switches as one switch group with Switch Profile Policy.
• Associate a switch profile with an interface profile.
 Interface Protocol Policies

Interface policies dictate interface behavior, and are later tied to interface policy groups
that bundles sets of interface policies as a template. For example, there should be a policy
that dictates if the Cisco Discovery Protocol is disabled or enabled.
Examples of the protocols that can be configured are:
• Link-Level
• LLDP
• Cisco Discovery Protocol
• LACP
• NetFlow
• Control Plane Policing (CoPP)
 Interface Policy Groups

Interface policy groups are templates that dictate interface level configurations via Interface Protocol
Policies and are associated to an AAEP.

Policy group gathers interface policies into a bundle:

 Cisco Discovery Protocol, LLDP, LACP ...
Policy group is linked to:
 Interface selector (specifies to which interface block the policies apply)
 An AAEP (Attachable Access Entity Profile) to be bundled under a domain so that a domain
can provide a set of interfaces and encapsulation resources
There are three basic types of policy groups:
 Access port
 Port Channel
 vPC
 Interface Profiles

Interface profiles help tie the pieces together. Interface profiles contain blocks of ports (interface selectors)
and are also tied to the interface policy groups that were described in the previous paragraphs.
A port such as e1/1 is arbitrary, and the profile must be associated to a specific
switch profile to configure the ports.
 Domains and Attachable Access Entity Profiles

A physical domain profile stores the physical resources (ports and port channels) and encapsulation
resources(VLAN/VXLAN) that should be used for endpoint group connections to the fabric
You can configure the following domain types:

VMM domain profiles: required for virtual machine hypervisor integration.

Physical domain profiles: typically used for bare metal server attachment and management access.
Bridged outside network domain profiles: typically used to connect a bridged external network trunk switch to a leaf switch in the
Cisco ACI fabric.
Routed outside network domain profiles: used to connect a router to a leaf switch in the Cisco ACI fabric.

An AAEP (or AEP) is a component that bundles group of interfaces through Interface Policy Groups,
which contain multiple interfaces that share same port level policies such as LLDP. An AAEP is attached to
a domain so that a domain can provide a group of interfaces (via AAEP) and VLANs (via VLAN pool) to
the logical resources such as EPG, L3OUT. An AAEP can be attached to more than one domain. AAEPs are
configured under global policies
 Relationship Between Access Policies

The relationship among the access policies can be summarized with the following figure:
Relationship Between Access Policies