Cisco Application Centric Infrastructure Multi-Site Lab v2: About This Demonstration

Lab Guide
Cisco dCloud
Cisco Application Centric Infrastructure Multi-Site Lab v2
Last Updated: 5-FEBRUARY-2019
About This Demonstration

This guide for the preconfigured demonstration includes:
 About This Demonstration
 Requirements
 About This Solution
 Topology
 Get Started
 Scenario 1: Create New Users and Sites
 Scenario 2: Day-0 Infrastructure Configuration
 Scenario 3: Create a MSC Tenants
 Scenario 4: ACI Multi-Site Use Cases
 Scenario 5: IP Mobility Across Sites (Stretched BD without BUM Flooding)
 Scenario 6: Import Tenant Configuration from a Site
© 2019 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 1 of 60
Lab Guide
Cisco dCloud
Limitations
The demonstration environment is a simulated environment and there is no actual data plane, therefore the fabrics will not establish
OSPF/BGP adjacencies. All configuration will be lost after a reboot of the APIC simulator
Customization Options
For streamlined client demos, the following customizations are suggested:
 If a customer is only interested in a demonstration of adding APIC Multi-Site to a brownfield environment, use the FixMyDemo
script (on the workstation desktop) and choose Option 4 to skip the setup and configuration scenarios. Then proceed immediately
to Scenario 6.
 To decrease the time it takes to run the lab, use the FixMyDemo script (on the workstation desktop) and choose Option 4 to
automatically perform the setup and configuration scenarios. Then proceed to Scenario 3.
Requirements
The table below outlines the requirements for this preconfigured demonstration.
Table 1. Requirements
Required Optional
Laptop Cisco AnyConnect®
Lab Guide
Cisco dCloud
About This Solution

As the newest advance in the Cisco ACI methods to interconnect networks, Cisco ACI Multi-Site is an architectural approach for
interconnecting and managing multiple sites, each serving as a single fabric and availability zone. As shown in the diagram, the Multi-
Site architecture has three main functional components:
 Two or more ACI fabrics built with Nexus 9000 switches deployed as leaf and spine nodes.
 One APIC cluster domain in each fabric.
 An inter-site policy manager, named Cisco ACI Multi-Site, which is used to manage the different fabrics and to define inter-site
policies.
Multi-Site has the following benefits:
 Complementary with Cisco APIC, in Multi-Site each site is an availability zone (APIC cluster domain), which can be configured to
be a shared or isolated change-control zone.
 MP-BGP EVPN is used as the control plane between sites, with data-plane VXLAN encapsulation across sites.
 The Multi-Site solution enables extending the policy domain end-to-end across fabrics. You can create policies in the Multi-Site
GUI and push them to all sites or selected sites. Alternatively, you can import tenants and their policies from a single site and
deploy them on other sites.
 Multi-Site enables a global view of site health.
 From the GUI of the Multi-Site Policy Manager, you can launch site APICs.
 Cross-site namespace normalization is performed by the connecting spine switches. This function requires Cisco Nexus 9000
Series switches with EX on the end of the name, or newer.
Lab Guide
Cisco dCloud
 Disaster recovery scenarios offering IP mobility across sites is one of the typical Multi-Site use cases.
For the Cisco ACI Multi-Site documentation, see http://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-

infrastructure-controller-apic/tsd-products-support-series-home.html.
Terminology
As a complementary product with Cisco ACI, much of the Cisco ACI Multi-Site terminology is shared with ACI and APIC (for example,
they both use the terms fabric, tenant, contract, application profile, EPG, bridge domain, and L3Out). For definitions of ACI terminology,
see Cisco Application Centric Infrastructure Fundamentals.
Micro-services architecture: In its first implementation, the Cisco ACI Multi-Site (inter-site policy manager) is represented by a cluster
of three Virtual Machines (VMs) running on ESXi hosts. These ESXi hosts do not need to be connected to the ACI leaf nodes, because
it is only required to establish IP connectivity between the VMs and the OOB IP addresses of the different APIC cluster nodes.
Namespace: Each fabric maintains separate data in its name space, including such objects as the TEP pools, Class-IDs (EPG
identifiers) and VNIDs (identifying the different Bridge Domains and the defined VRFs). The site-connecting spine switches (EX or later)
perform the necessary namespace translation (normalization) between sites.
Schema: Profile including the site-configuration objects that will be pushed to sites.
Site: APIC cluster domain or single fabric, treated as an ACI region and availability zone. It can be located in the same metro-area as
other sites, or spaced world-wide.
Stretched: Objects (tenants, VRFs, EPGs, bridge-domains, subnets or contracts) are stretched when they are deployed to multiple
sites.
Template: Child of a schema, a template contains configuration-objects that are shared between sites or site-specific.
Template Conformity: When templates are stretched across sites, their configuration details are shared and standardized across
sites. To maintain template conformity, it is recommended to only make changes in the templates, using the Multi-Site GUI and not in a
local site's APIC GUI.
Lab Guide
Cisco dCloud
Topology
This content includes preconfigured users and components to illustrate the scripted scenarios and features of the solution. Most
components are fully configurable with predefined administrative user accounts. You can see the IP address and user account
credentials to use to access a component by clicking the component icon in the Topology menu of your active session and in the
scenario steps that require their use.
Figure 1. dCloud Topology
Lab Guide
Cisco dCloud
Get Started
BEFORE PRESENTING
Cisco dCloud strongly recommends that you perform the tasks in this document with an active session before presenting in front of a
live audience. This will allow you to become familiar with the structure of the document and content.
It may be necessary to schedule a new session after following this guide in order to reset the environment to its original configuration.
PREPARATION IS KEY TO A SUCCESSFUL PRESENTATION.
FOR DCLOUD SCHEDULED DEMO
Follow the steps to schedule a session of the content and configure your presentation environment.
1. Initiate your dCloud session. [Show Me How]
NOTE: It may take up to 10 minutes for your session to become active.
2. For best performance, connect to the workstation with Cisco AnyConnect VPN [Show Me How] and the local RDP client on your
laptop [Show Me How]
 Workstation 1: 198.18.133.36, Username: DCLOUD\demouser, Password: C1sco12345
Lab Guide
Cisco dCloud
Scenario 1. Create New Users and Sites

The purpose of this scenario is to use the privileges of the admin user to create user profiles with different functions, and to add sites to
the Multi-Site Policy Manager. These additional users and sites will be used in subsequent scenarios. The user admin is predefined for
this lab.
NOTE: In the context of this guide, the terms Multi-Site Policy Manager, Multi-Site Manager and Multi-Site Controller (MSC) will
be used interchangeably.
Steps
Create a New User
1. Double-click the ACI MultiSite Controller icon on the workstation desktop, and log in (admin/C1sco12345!).
2. Select the Users icon in the vertical menu.
NOTE: If necessary, scroll to the bottom and close the blue screen at start up.
Lab Guide
Cisco dCloud
3. Click the Add User button.
4. Configure the user-specific account information as follows:
 Username: demouser
 Password: C1sco12345!
 First Name: Demo
 Last Name: User
 Email Address: [email protected]
 Phone Number: Enter any phone number
 Account Status: Active
NOTE: The User Roles screen shows a set of predefined roles in the Multi-Site Manager. The following user roles are available
in Cisco ACI Multi-Site.
Power User: A power user can perform all the operations as an admin user.
Lab Guide
Cisco dCloud
Site and Tenant Manager: A site and tenant manager can manage sites, tenants, and associations.
Schema Manager: A schema manager can manage all schemas regardless of tenant associations.
Schema Manager (Restricted): A restricted schema manager can manage schemas that contain at least one tenant to which the
user is explicitly associated.
User and Role Manager: A user and role manager can manage all the users, their roles, and passwords.
Admin User: In the initial configuration script, the admin account is configured and the admin is the only user when the system
starts. The initial password for the admin user is set by the system. The admin user is assigned the role of a Power User. The
admin user should be used for creating other users and for all other Day-0 configurations. The account status of the admin user
cannot be set to Inactive.
5. Scroll down and toggle the Site and Tenant Manager switch to ON, to associate to the newly created user with that role.
6. Click Save.
7. Log out and log back in (demouser/C1sco12345!).
Lab Guide
Cisco dCloud
8. In the Reset Password dialog, enter any new password that meets the requirements. The demouser account is not used again in
this lab.
9. Verify that only a subset of the functions exposed to the admin user are now available (specifically, only the possibility of adding
Sites and Tenants to the Multi-Site manager).
10. Log out of ACI Multi-Site and log back in as an administrator (admin/C1sco12345!).
Add Sites to the Multi-Site Policy Manager

The purpose of this section is to connect the ACI Multi-Site Policy Manager to two different APIC domains, in New York and San
Francisco, and to be able to start centrally managing those different fabrics.
The figure below shows the physical topology used for the lab.
Lab Guide
Cisco dCloud
1. On the ACI Multi-Site dashboard, select Sites in the vertical menu and select Add Site.
2. In the Add Site wizard, enter details for Site 1 as follows, and click Save. If an error is displayed, click Save again.
 Name: San Francisco
 APIC Controller URLs:

https://apic1-a.dcloud.cisco.com
 Username: admin
 Password: C1sco12345 (enter even if it appears the password is already pre-populated)
 APIC Site ID: 1
NOTE: Each fabric connected to the MSC must be assigned a unique Site-ID value. MSC will give an error when trying to
configure overlapping Site-IDs to separate fabrics.
Lab Guide
Cisco dCloud
3. Click Add Site again.
4. Enter details for Site 2 (New York) as follows, and click Save.
 Name: New York
 APIC Controller URLs:

https://apic1-b.dcloud.cisco.com
 Username: admin
 Password: C1sco12345 (enter even if it appears the password is already pre-populated)
 APIC Site ID: 2
5. Verify that both sites show green status and that the configuration URLs are correct.
Lab Guide
Cisco dCloud
Scenario 2. Day-0 Infrastructure Configuration

The purpose of this scenario is to perform the Day-0 configuration for interconnecting ACI fabrics (sites), which is managed from the
Multi-Site manager.
The following configuration tasks are managed from the MSC.
 Spine interface configuration (Interface selection, IP address assignment, MTU setting)
 Control Plane E-TEP configuration (Used for BGP EVPN peering between sites)
 Unicast Data plane E-TEP configuration
 Multicast Data plane E-TEP configuration
 OSPF area configuration for spine to IP network connections (area id, area type)
 BGP Autonomous System Number assignment (uses fabric ASN)
 BGP community configuration
 External L3 domain selection
The following configuration tasks are managed from the APIC at each site (site local configuration).
 Configuration of the access policies for the External L3 domain (Spine switch profile, interface profile, interface policy group,
attachable entity profile, external L3 domain)
 BGP Route Reflector Policy
The MSC will read in the BGP ASN and External L3 Domains from each site. Add these to each site from the respective APICs.
Steps
Add BGP Route Reflector (RR) Policy for Site 1

The purpose of this section is to enter an Autonomous System Number and an External Routed Domain for the San Francisco site.
These elements are required for BGP routing.
1. Double-click the APIC SF icon on the workstation desktop, and log in (admin/C1sco12345).
2. Close the What’s New banner.
NOTE: Ignore any warnings to update the license for APIC.
Lab Guide
Cisco dCloud
3. Select System > System Settings in the top menu and click BGP Route Reflector.
4. Enter 65001 in the Autonomous System Number field and click Submit.
5. At the Policy Usage Warning dialogue, click Submit Changes.
6. Click Fabric > Access Policies in the top menu and expand Physical and External Domains.
7. Click External Routed Domains.
8. Expand the Tools menu and select Create Layer 3 Domain from the menu.
Lab Guide
Cisco dCloud
9. Enter Multisite_External_L3_Domain in the Name field and click Submit.
Add BGP RR policy to APIC for Site 2
1. Double-click the APIC NY icon on the workstation desktop, and log in (admin/C1sco12345).
2. Close the What’s New banner.
3. Select System > System Settings in the top menu and click BGP Route Reflector.
4. Enter 65002 in the Autonomous System Number field and click Submit.
Lab Guide
Cisco dCloud
5. At the Policy Usage Warning dialogue, click Submit Changes.
6. Click Fabric > Access Policies in the top menu and expand Physical and External Domains.
7. Click External Routed Domains.
8. Expand the Tools menu and select Create Layer 3 Domain from the menu.
9. Enter Multisite_External_L3_Domain in the Name field and click Submit.
Lab Guide
Cisco dCloud
MSC Infra Configuration Steps

The purpose of this section is to use MSC to configure Infra for San Francisco (Site 1) and New York (Site 2). Perform the same
procedure for each site, noting the difference in IP addresses and other values.
1. Return to the Multi-Site Controller and log in (admin/C1sco12345!). Click Configure Infra to display the BGP and OSPF settings
page.
NOTE: The default setting for BGP is full mesh and uses standard BGP timer values. The default OSPF network type is point-to-
point.
2. The Fabric Connectivity Infra screen displays.
Lab Guide
Cisco dCloud
3. Select San Francisco in the side menu to add Infra settings for Site 1.
NOTE: If configuring Site 2, select New York.
4. Highlight Spine 1 and click Add Port.
5. Enter 5/32 in the Port ID field for Spine 1.
NOTE: These settings are the same for San Francisco and New York.
6. Enter the Spine 1 interface address in the IP Address field.
 Site 1 (San Francisco): 10.1.0.1/31
 Site 2 (New York): 10.2.0.1/31
7. Select inherit from the MTU drop-down and click Save.
Lab Guide
Cisco dCloud
8. Toggle the BGP Peering button to enable BGP Peering.
9. Enter the appropriate IP address in the BGP-EVPN ROUTER_ID field for Spine 1.
 Site 1: 10.1.100.1
 Site 2: 10.2.100.1
NOTE: Intersite control plane: Endpoint reachability information is exchanged across sites using a Multiprotocol-BGP (MP-BGP)
Ethernet VPN (EVPN) control plane. This approach allows the exchange of MAC and IP address information for the endpoints that
communicate across sites. MP-BGP EVPN sessions are established between the spine nodes deployed in separate fabrics .
10. Select Spine 2 and click Add Port.
11. Enter 5/32 in the Port ID field for Spine 2.
NOTE: These settings are the same for San Francisco and New York.
12. Enter the Spine 2 interface address in the IP Address field.
 Site 1: 10.1.0.3/31
 Site 2: 10.2.0.3/31
Lab Guide
Cisco dCloud
13. Select inherit from the MTU drop-down and click Save.
14. Toggle the BGP Peering button to enable BGP Peering.
15. Enter the appropriate IP address in the BGP-EVPN ROUTER_ID field for Spine 2.
 Site 1: 10.1.100.2
 Site 2: 10.2.100.2
Lab Guide
Cisco dCloud
16. Configure Unicast Data Plane ETEP for Site 1 as follows:
a. Select pod-1.
b. Enter the appropriate IP address in the OVERLAY UNICAST TEP field.
 San Francisco: 10.1.100.100
 New York: 10.2.100.100
c. Select the site box (San Francisco or New York) to bring up the pane to enable Multi-Site.
d. Configure the following:
i. Site is Mult-Site Enabled: On
ii. OVERLAY MULTICAST TEP: SF: 10.1.100.200 / NY: 10.2.100.200
iii. BGP Autonomous System Number: SF: 65001 / NY: 65002
iv. OSPF Area ID: 0
v. OSPF Area Type: regular
vi. External Routed Domain: Multisite_External_L3_Domain
Lab Guide
Cisco dCloud
NOTE: The BGP AS number will be prepopulated as this is read in from the APIC. The External Routed Domain drop down will
display the domain previously configured on the APIC (Multisite_External_L3_domain).
e. Click DEPLOY to push the Infra L3out configuration to the APIC.
17. Wait for the success message and close the Fabric Connectivity Intra window.
18. Add the Multipod Data Plane TEP configuration as follows.
a. In the APIC window for the site being configured, browse to Tenants > infra > Policies > Protocol > Fabric External
Connections Policies and click Fabric External Connection Policy.
b. Enter extended:as2-nn4:5:16 in the Community field.
c. In the work pane, double-click Pod ID 1.
Lab Guide
Cisco dCloud
d. Click + to add a subnet.
e. Enter the MultiPod Dataplane TEP in the Subnet field.
 San Francisco: 10.1.200.200/32
 New York: 10.2.200.200/32
f. Click Update, then click Close. Finally click Submit to confirm the changes.
Lab Guide
Cisco dCloud
19. Expand Tenant > infra > Networking > External Routed Networks and verify that an L3out called intersite has been configured
under the infra tenant. This indicates that Infra L3out has been successfully configured on APIC for Site 1.
NOTE: The object for the L3Out also includes a small cloud icon . All ACI objects configured by the MSC will include this icon.
20. Repeat all of the steps for the New York site, using the New York values indicated in the text.
Lab Guide
Cisco dCloud
Scenario 3. Creating a MSC Tenant

The purpose of this scenario is to create a tenant in the Multi-Site controller that will be deployed across both sites. This will be a
greenfield tenant that does not currently exist at any sites, and the configuration will be pushed to both sites at the same time.
Steps
1. In the Multi-Site Configuration window, select Tenants in the side menu and click Add Tenant.
2. Enter Tesla in the Display Name field.
3. Select both San Francisco and New York and click Save to push the Tenant configuration to APIC.
Lab Guide
Cisco dCloud
4. Return to the APIC SF and APIC NY windows. Click Tenants > ALL TENANTS in each window and verify that the Tesla tenant
has been created on both fabrics.
5. In either APIC SF or APIC NY, double-click Tesla to proceed to the APIC window for Tesla.
6. Note that the tenant object includes the cloud symbol, indicating that this object has been configured from the MSC. The
APIC GUI will also display a message to this effect.
Lab Guide
Cisco dCloud
Scenario 4. ACI Multi-Site Use Cases

This scenario contains four common Multi-Site use cases:
 Layer 3 Only Communication Across Sites
 Layer 3 Only Communications Across Sites and VRFs (Shared Services)
 IP Mobility Across Sites (Stretched BD Without BUM Flooding)
 Importing Tenant Configuration From a Site
Steps
Layer 3 only Communication across Sites (Intra-Tenant and Intra-VRF)

This use case will show Layer-3 communication across sites. The Tenant and VRF objects will be stretched across sites but the BD,
Subnet, and EPG configuration will be local to each site.
Lab Guide
Cisco dCloud
Configure a New Schema and Template

The purpose of this section is to configure a schema – schemas contain templates for tenant object configuration.
1. In the Multi-Site Configuration window, select Schemas from the vertical menu and click Add Schema.
2. Click Untitled Schema in the upper left corner to make the field editable, and enter L3-stretch-schema as the schema name.
NOTE: Schemas contain templates. The templates are associated to one or more sites and are used to define the objects that
will be stretched between sites or will remain site-local.
Lab Guide
Cisco dCloud
3. Since this configuration will stretch the Tenant and VRF, click Template1, then click the pencil icon and name the template SF
and NY Template.
4. Click the + symbol to add the tenant.
5. Select Tesla from the Select a Tenant drop-down.
6. Click + under VRF. For this use case the only stretched object (commonly defined across sites) will be the VRF.
7. Enter VRF1 in the Display Name field.
Lab Guide
Cisco dCloud
Associate the Template to the Sites

Since the objects in this template will be stretched across both sites, it must be associated with both sites.
1. Select the + symbol next to Sites in the side menu.
2. Select both San Francisco and New York and click Save.
NOTE: When configuration is added to the MSC there is a Save button and a Deploy to Sites button. Saving the template
configuration saves it to the MSC database but does not make any changes to the APICs. Only after selecting Deploy to Sites is
the configuration change pushed to the APICs. At this point in the configuration we have added a template and created a VRF
but have not saved nor deployed the configuration
3. Click Save to save the configuration to the MSC without deploying to APIC.
Lab Guide
Cisco dCloud
4. Click Deploy to Sites.
5. A window will appear showing which changes will be deployed and to which sites. Click Deploy, which will result in the creation of
the VRF in both sites.
6. Wait for the successful deployment message.
Add a Single-Site Tenant to Site 1

The purpose of this section is to add an SF Only Template and associate it to the Tesla tenant. This template will contain the BD,
Subnets, and EPGs that will be locally deployed at site 1.
1. In the Multi-Site Configuration window, select the + next to Templates.
2. Click on Template1, then on the pencil icon so you can rename it.
3. Enter SF Only in the Name field.
4. Select + to associate the tenant.

5. Select Tesla from the Select a Tenant drop-down.
Lab Guide
Cisco dCloud
Add Network Elements for Site 1

The purpose of this section is to add an Application Profile, EPG, Bridge domain, and subnet for Site 1 (San Francisco).
1. Select + next to Application Profile.
2. Enter Webapp in the Display Name field.
3. Click Add EPG and enter Web in the Display Name field.
Lab Guide
Cisco dCloud
4. Select the dropdown for the Bridge Domain to associate the Web EPG to a bridge domain.
5. Enter Web-BD in the Bridge Domain field. Since Web-BD does not currently exist, the option to create the object is one of the
choices on the drop-down.
6. Scroll down and select the Web-BD. The default BD settings will appear on the right side pane. This BD will not be stretched
across sites. Uncheck the L2STRETCH box.
7. Acknowledge the Warning by clicking Yes.
NOTE: When the L2STRETCH box is unchecked the option to add a BD subnet is removed. This is because the BD becomes a
site local configuration. The site local configuration will be covered in a few more steps
8. On the Virtual Routing and Forwarding drop-down, select VRF1 (the VRF created in the SF and NY Template).
Lab Guide
Cisco dCloud
9. Select + next to Sites, use the drop-down to add the SF Only template to San Francisco. This associates the template to only
San Francisco Site.
10. Click Save to close the Add Sites window.
11. At the top of the screen, click Save.
NOTE: You can save and deploy to sites in one step by just selecting deploy to sites. The configuration will be saved to the MSC.
12. Click Deploy To Sites.
Lab Guide
Cisco dCloud
13. Click Deploy. The MSC will show that the changes are only being pushed to San Francisco.
14. Wait for the Successfully deployed message.
NOTE: Site-local configuration changes are not displayed in the Template view. They are visible only in the site view.
15. Select San Francisco SF Only in the vertical menu to view the site-local changes.
Lab Guide
Cisco dCloud
16. Select Web-BD and click + Subnet to add a subnet.
17. Add 10.1.1.254/24 in the Gateway IP field and click Save.
Lab Guide
Cisco dCloud
18. At the top, click Save then click Deploy to Sites to deploy the changes to the site, then click Deploy.
NOTE: EPGs are also associated to domains (physical or VMM domains). The domain association and static path binding
configuration is also done from the MSC. This will always be a site local configuration task and will be configured from selecting
the site just as what was done for the BD subnet. In this lab we will not be configuring the domain but be aware that this
configuration is always site local.
Create a Single-Site Tenant & Network Elements for Site 2

1. Repeat the previous section to create a NY Only template and associate it to the Tesla tenant with the following changes:
 Create a NY Only template
 Application Profile Name: Webapp
 EPG Name: App
 Bridge Domain Name: App-BD
 Add the NY Only template to Site 2 (New York).
 Add the BD subnet for App-BD. Remember to select the NY Site on the left pane for site local configuration.
 Gateway IP: 10.1.2.254/24
At this point both sites have been configured with a tenant called Tesla and an application profile called Webapp. A Web EPG and BD
has been configured in San Francisco and an App EPG and BD has been configured in New York. There is no communication at this
time between sites, and the Web BD subnet in San Francisco is not known to New York and vice versa for the App BD in New York. A
contract is required in order to allow communication between sites and to advertise endpoint IP address information between sites.
Since this contract will be used by both EPGs we will configure it under the SF and NY Template.
Lab Guide
Cisco dCloud
Add a Contract Between Sites

The purpose of this section is to add a contract between the Web EPG at Site 1 (San Francisco) and the App EPG at
Site 2 (New York). This will permit traffic to flow between the EPGs at the different sites.
1. Select SF and NY Template in the vertical menu.
2. Click the + under Contracts.
3. Enter web-to-app in the Display Name field.
4. Scroll down to Filter and click +. Enter any in the Display Name field.
Lab Guide
Cisco dCloud
5. Click + Entry to add an entry for the filter.
6. Enter any in the Name field and click Save.
7. Scroll up and click the web-to-app contract.
8. Click + next to Filter to add a filter.
Lab Guide
Cisco dCloud
9. Select any from the Name drop-down.
10. Select none in the Directive field to configure the Filter Chain.
11. Click Save, then Deploy to Sites.
12. Click Deploy to deploy contract to both San Francisco and New York sites.
Lab Guide
Cisco dCloud
Add Contracts to Single-Site Templates

The purpose of this section is to add a web-to-app provider contract to the Web EPG in the SF Only template, and a web-to-app
consumer contract to the App EPG in the NY Only template. This will allow the application tiers to communicate between sites as long
as they are deployed in the same tenant.
1. Select SF Only in the vertical menu.
2. Click the Web EPG.
3. Click + Contract to add a contract to the EPG.
4. Select web-to-app in the Contract drop-down.
5. Select provider in the Type drop-down. Click Save.
Lab Guide
Cisco dCloud
6. Select NY Only in the vertical menu.
7. Click the App EPG.
8. Click + Contract to add a contract to the EPG.
9. Select web-to-app from the Contract drop-down.
10. Select consumer from the Type drop-down.
11. Click Save.
12. Click Save, then Deploy to Sites on the main page.
13. Click Deploy to deploy the NY Only template to the New York site.
Lab Guide
Cisco dCloud
14. Click SF Only in the vertical menu.
15. Click Deploy to Sites.
16. Click Deploy to deploy the SF Only template to the San Francisco site.
Verify configurations are pushed to both sites

When the contract is applied to the Web and App EPGs the Web EPG and BD are pushed to the NY site and the App EPG and BD are
pushed to the SF site. Verify the configuration in the APIC.
1. Open either APIC NY or APIC SF if they are not already open.
2. Click Tenants and double-click Tesla.
3. Expand Tenant Tesla > Application Profiles > Webapp > Application EPGs and verify the presence of the App and Web
EPGs.
4. Expand Tenant Tesla > Networking > Bridge Domains and verify the presence of the App-BD and Web-BD bridge domains.
Lab Guide
Cisco dCloud
5. Click the Application Profiles > Webapp application profile.
6. Click Topology tab and show the presence of the contract that allows the EPGs to communicate.
Layer 3 only Communication across Sites and VRFs (Shared Services)

This use case is similar to the one just completed to enable L3 communication across sites. The only difference is that this use case
seeks to provide shared services access to resources that are deployed in a site as part of a dedicated Shared VRF/BD/EPG object, as
shown below.
The purpose of this scenario is to create the inter-site policies required to enable the Web endpoints to access the shared DNS
services. If necessary, refer to the previous use case for specifics on each step.
Lab Guide
Cisco dCloud
1. In ACI MultiSite Controller, select Schemas > L3-stretch-schema > and the SF Only Template.
2. Under VRF, select + to create a new VRF. Enter VRF2 as the name.
3. Still in the SF Only Template, select Web-BD, and choose VRF2 from the Virtual Routing and Forwarding drop down menu.
4. Select the NY Only template, click + below VRF to create a new VRF, enter Shared VRF as the Display Name.
Lab Guide
Cisco dCloud
5. Still in the NY Only template, in the Bridge Domain field, select +, and create a new Bridge Domain with the following attributes:
a. Display Name: Shared-BD
b. Virtual Routing & Forwarding: Shared VRF
6. In the left menu pane, under Templates, select NY Only.
7. Select the Shared-BD, and select +Subnet.
8. Create a new subnet with the following attributes, and click Save:
a. Gateway IP: 10.10.1.254/24
b. Shared between VRFs: Selected
Lab Guide
Cisco dCloud
9. Scroll up a little way to the AP section, and click +Add EPG to create a new EPG.
10. Enter the following properties for the new EPG:
a. Display Name: DNS
b. Bridge Domain: Shared-BD
11. Since this EPG will provide shared services to other EPGs, it is critical to remember to add the IP subnet information under the
EPG (at the site level). Still in the work pane for the new DNS EPG, select +Subnet.
12. Create a new subnet with the following attributes and click Save:
a. Gateway IP: 10.10.1.254/24
b. Shared between VRFs: Selected
c. No Default SVI Gateway: Selected
Lab Guide
Cisco dCloud
NOTE: Notice that the sole purpose of defining the IP subnet information under the provider EPG is enabling the necessary VRF
route-leaking functionalities between the ‘Shared VRF’ and the other VRFs accessing the shared services. The default gateway
services are provided by the IP subnet configured at the BD, so it is important here to select the NO DEFAULT SVI GATEWAY
flag.
13. Create the web-to-dns contract and ensure that it is provided by the DNS EPG. In the left menu pane, under Templates, select NY
Only.
14. In the Contracts field select + to create a new Contract. Enter the following attributes:
a. Display Name: web-to-dns
b. Scope: tenant
c. Filter: any (select + to select the any filter)
NOTE: By default the contracts are created with VRF scope. In this case, the contract must be consumed by the ‘Web’ EPG that
is part of a different VRF, so it is essential to modify the scope of the contract to ‘tenant’ or ‘global’.
Lab Guide
Cisco dCloud
15. Select the DNS EPG, and +Contract.
Lab Guide
Cisco dCloud
16. Select the web-to-dns contract and set as a provider. Then Save.
17. Select Save and then Deploy to Sites. Click Deploy again to push out the configuration to New York.
18. Now configure the Web EPG to consume the web-to-dns contract. From the left pane, under Templates, select SF Only, and the
Web EPG.
19. Select +Contract.
Lab Guide
Cisco dCloud
20. Select the web-to-dns contract and configure it as a consumer. Then select Save.
21. Select Save and Deploy to Sites. Click Deploy again to push out the new configuration.
22. Verify the configuration has been deployed to San Francisco (apic-1a). Login to the APIC, and browse to Tenants > Tesla >
Application Profiles > Webapp.
23. Click Topology.
Lab Guide
Cisco dCloud
24. Verify the configuration has been deployed to New York (apic-1b). Login to the APIC, and browse to Tenants > Tesla >
Application Profiles > Webapp.
25. Click Topology.
Lab Guide
Cisco dCloud
Scenario 5. IP Mobility across Sites (Stretched BD without BUM

Flooding)
In this use case, the requirement is to have IP mobility across sites (same BD and subnet) but without enabling BUM flooding.
This use case is typical for disaster recovery sites where there is not a requirement for mobility (vMotion) across sites, but allows the
application to be brought up at a DR site without having to re-IP the application servers.
The suppression of BUM flooding across sites provides more resiliency across sites, since a problem (like a broadcast storm) hitting
site 1 won’t be able to propagate to the other sites.
A new BD, called DB-BD, will exist on both the San Francisco and New York sites and will enable this use case on the Tesla tenant.
Steps
1. In ACI MSC, browse to Schemas > L3-stretch-schema > SF and NY Template.
2. In the Bridge Domain field select +, and configure with the following:
a. Display Name: DB-BD
b. Virtual Routing & Forwarding: VRF1
c. INTERSITE BUM TRAFFIC ALLOW: Deselected (Click Yes on the Warning dialogue)
d. Add Subnet: 10.3.1.254/24
Lab Guide
Cisco dCloud
NOTE: At this point there are no EPGs that are shared across sites so the Webapp Application profile has not been configured
under the SF and NY template.
3. Click Save.
4. Ensure the SF and NY Template is selected. Click +Application Profile and enter Webapp as the Display Name. (The
Application Profile name is case sensitive, so be consistent with the name used previously).
5. Click +Add EPG to create a new EPG with the following configuration:
a. Display Name: DB
b. Bridge Domain: DB-BD
6. Click Save and Deploy to Sites. Click Deploy to push out the configuration.
Lab Guide
Cisco dCloud
7. Verify the configuration has been deployed to San Francisco (apic-1a) and New York (apic-2a). Login to the APIC, and browse to
Tenants > Tesla > Application Profiles > Webapp.
8. Select the Topology tab.
Lab Guide
Cisco dCloud
Scenario 6. Importing Tenant Configuration from a Site

This is an important use case as it will be deployed in migration scenarios where ACI has already been deployed in a fabric and now
there is a requirement to connect the existing fabric to a new greenfield ACI environment using Multi-Site.
This implies that the ACI Multi-Site Policy Manager will be inserted in the deployment and both sites will be added to it (as described as
part of the Add Sites to the Multi-Site Policy Manager’ section).
The purpose of this section is to import the existing configuration for a tenant from a brownfield ACI fabric, and stretch the objects
associated to that tenant (application profile with corresponding EPGs, BDs and VRFs) toward one (or more) greenfield ACI fabrics. In
the context of this lab, the brownfield site is San Francisco and the new greenfield fabric is New York.
Steps
1. In the APIC for the San Francisco (apic-1a) screen, click Tenants > Brownfield.
2. Click Tenant Brownfield from the menu.
3. Note that this tenant has the following configuration:
 VRF: Brownfield-VRF
 Bridge Domain: Brownfield-BD
 Application Profile: AP
 End Point Group: Brownfield-EPG
Lab Guide
Cisco dCloud
4. Return to ACI Multisite Controller and select Tenants and Add Tenant.
5. Enter the Display Name Brownfield and select both San Francisco and New York as the associated sites, and Save.
NOTE: It is essential that the name of the tenant created on MSC matches the name of the tenant in the brownfield fabric from
where the configuration should be imported. The newly created tenant should then be associated to both existing sites, since
the configuration will be imported from one and stretched toward the other.
6. Click Schemas > Add Schema
7. Create a new schema called Migration-Schema. This new schema will be used to perform the import of the configuration from
‘San Francisco’ into a new ‘Migration-Template’. Select the Brownfield tenant.
Lab Guide
Cisco dCloud
8. Click Import and select San Francisco to import the Brownfield tenant configuration into the Multi-Site manager.
9. In the resulting window, select the Application Profile Brownfield-AP and make sure the Include Relations toggle is ON to import
all the objects associated to the Brownfield-AP application profile.
10. Hover over AP to see the sources. Click Import and click Save.
11. Select the Brownfield-BD bridge domain to verify that the configuration is not stretched (which is expected, since it was imported
from a specific site).
12. Click the L2Stretch check box to allow to stretch it to the Greenfield site. Click Yes to acknowledge the warning.
13. Check the Intersite BUM Traffic Allow checkbox to ensure that BUM traffic is allowed, as this is required in a real life scenario to
be able to migrate workloads from brownfield to greenfield leveraging live migration technologies (for example, vMotion in a
vSphere environment).
Lab Guide
Cisco dCloud
14. Now that the configuration has been imported to the Multi-Site manager, it is required to push the objects toward the Greenfield
ACI fabric (New York site). First rename the Template1 to Migration-Template.
15. Now associate the Migration-Template to the New York site. Under Sites, click +, and select New York and Save.
Lab Guide
Cisco dCloud
16. Click Save again.
17. Click Deploy to Sites to push the configuration to the Greenfield site.
18. Click Deploy again. This will push the objects imported from the brownfield site toward the Greenfield ACI fabric.
19. Verify that the configuration is now displayed correctly in the New York APIC controller.
Printed in USA Cxx-xxxxxx-xx 05/17

Cisco Application Centric Infrastructure Multi-Site Lab v2: About This Demonstration

Uploaded by

Copyright:

Available Formats

Cisco Application Centric Infrastructure Multi-Site Lab v2: About This Demonstration

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cisco Application Centric Infrastructure Multi-Site Lab v2: About This Demonstration

Uploaded by

Copyright:

Available Formats

What are the main components of the ACI Multi-Site architecture?

What are the main components of the ACI Multi-Site architecture?

What are the benefits of the ACI Multi-Site solution?

What are the benefits of the ACI Multi-Site solution?

Lab Guide

Cisco Application Centric Infrastructure Multi-Site Lab v2

Last Updated: 5-FEBRUARY-2019

About This Demonstration

 About This Demonstration

 About This Solution

 Scenario 1: Create New Users and Sites

 Scenario 2: Day-0 Infrastructure Configuration

 Scenario 3: Create a MSC Tenants

 Scenario 4: ACI Multi-Site Use Cases

 Scenario 5: IP Mobility Across Sites (Stretched BD without BUM Flooding)

 Scenario 6: Import Tenant Configuration from a Site

About This Solution

 One APIC cluster domain in each fabric.

Multi-Site has the following benefits:

 Multi-Site enables a global view of site health.

For the Cisco ACI Multi-Site documentation, see http://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-

Figure 1. dCloud Topology

PREPARATION IS KEY TO A SUCCESSFUL PRESENTATION.

FOR DCLOUD SCHEDULED DEMO

1. Initiate your dCloud session. [Show Me How]

NOTE: It may take up to 10 minutes for your session to become active.

 Workstation 1: 198.18.133.36, Username: DCLOUD\demouser, Password: C1sco12345

Scenario 1. Create New Users and Sites

Create a New User

2. Select the Users icon in the vertical menu.

3. Click the Add User button.

4. Configure the user-specific account information as follows:

 First Name: Demo

 Last Name: User

 Email Address: [email protected]

 Phone Number: Enter any phone number

 Account Status: Active

7. Log out and log back in (demouser/C1sco12345!).

Add Sites to the Multi-Site Policy Manager

 Name: San Francisco

 APIC Controller URLs:

 Password: C1sco12345 (enter even if it appears the password is already pre-populated)

 APIC Site ID: 1

3. Click Add Site again.

 Name: New York

 APIC Controller URLs:

 Password: C1sco12345 (enter even if it appears the password is already pre-populated)

 APIC Site ID: 2

Scenario 2. Day-0 Infrastructure Configuration

The following configuration tasks are managed from the MSC.

 Spine interface configuration (Interface selection, IP address assignment, MTU setting)

 Unicast Data plane E-TEP configuration

 Multicast Data plane E-TEP configuration

 BGP Autonomous System Number assignment (uses fabric ASN)

 BGP community configuration

 External L3 domain selection

 BGP Route Reflector Policy

Add BGP Route Reflector (RR) Policy for Site 1

2. Close the What’s New banner.