MIST EDGE GENERAL CONFIG GUIDE

DOCUMENT OWNERS:

Robert Young – [email protected]

Slava Dementyev – [email protected]

Jan Van de Laer – [email protected]

1
Table of Contents
Solution Overview 3
How it works 6

Configuration Steps 7

Setup Mist Edge 7

Configure and prepare the SSID 19

Create a Site for Campus, Branches 21

Mist Edge Insights 22

Troubleshooting 24

Packet Captures on the Mist Edge 24

2
Solution Overview
Mist solution leverages Mist Edge for cases that need to retain the Centralized Datapath
Architecture for Campus/Branch deployments. Mist AP can form L2TPv3 Tunnel to extend one or
multiple vlan from one or multiple Mist Edge located in Campus, DC or DMZ simultaneously. AP can
support both local and Centralized Datapath together.

3
WIth Mist Edge solution customers can retain their centralized Datapath, providing the same level of
redundancy and access to corporate resources, while extending visibility into user network
experience and streamlining IT operations.

What are the benefits of the Mist Edge Centralized architecture solution with Mist Edge compared
to all the other alternatives?

Agility:

● Zero Touch Provisioning - no AP pre-staging required, support for same AP to tunnel to

multiple locations and any number of cluster support
● Exceptional support with minimal effort - leverage Mist SLEs and Marvis Actions
● No firmware dependency between AP and Mist Edge , Mist Edge services can be updated
independently and takes 3 seconds at max to update.

Security:

● Traffic Isolation - same level of traffic control as original WLC architecture.

● Automated Security - machine-driven site deployment, no credential exposure.
● Support Dot1x and MPSK support.

Flexibility:

● Full re-usability of hardware

● Can add additional Mist Edges for horizontal scale increase of APs or users.

Scalability:

● Can scale from few branches to Thousands of them.

4
 ● Can support Campus with a few hundred APs to Thousands of them.
● A single Mist Edge can support 10000 AP and 100000 Clients.

The components of the Centralized Architecture solution include the following:

● Mist AP
● Mist Edge Appliance:

Appliance VM

Key Metrics Mist Edge Mist Edge Mist Edge – Mist Edge – Mist Edge -
–X1 –X5 X5-M X10 VM

# AP 500 5000 5000 10000 500

# Clients 5000 50000 50000 100,000 5000

Throughput 2 Gbps 20 Gbps 40 Gbps 40 Gbps 2 Gbps

● Mist WiFi Assurance subscription (1x per AP) where X is 1,3 or 5 Years of service:

SUB-1S-<X>Y

● Mist Edge subscription (1x per AP), where X is 1, 3 or 5 years service:

SUB-ME-1S-<X>Y

Recommended additional components:

● Mist Marvis subscription (1x per AP) where X is 1, 3 or 5 years of service:

SUB-1S-<X>Y

Note : Mist Edge VM has part number ME-VM that needs to be used for quotes. 1 ME-VM license
allows any number of Mist Edge VM per org for a 1000 AP limit.

5
How it works
Mist solution leverages Mist Edge for extending centralized corporate/production/guest network
vlan to APs using L2TPv3 tunnel. Mist Cloud orchestrates the Tunnel , Datapath continues to work
even if Mist Edge or AP’s Cloud connectivity to Mist Cloud is lost.

Mist Edge is based on multi service architecture , so individual services can be upgraded as and when
required and takes a maximum of 3 seconds and does not require a Mist Edge reboot.

AP firmware and Mist Edge service version are decoupled , upgrading a Mist Edge does not warrant
an AP firmware upgrade.

APs can form Multiple Tunnels to different Mist Edge Cluster on Site, DMZ and Datacenter , user
traffic can be mapped to be tunneled or local bridged based on Radius attribute returned for Dot1x
authenticated wireless LAN.

APs can support Tunneled and Local Bridged Wireless LAN together and are not mutually exclusive.

Mist cloud-driven AI provides unprecedented user experience visibility via Service Level
Expectations (SLE) framework, AI-driven Marvis engine with natural language processing for
troubleshooting and root cause analysis and Marvis actions, which IT can leverage for remote
troubleshooting of user issues without spending any additional resources.

6
Configuration Steps
The configuration process is very straightforward and consists of the following steps. Once the initial
configuration is done, no pre-staging of the Access Points is required, they can be shipped directly to
the branch or Campus and brought online.

Setup Mist Edge

Mist Edge typically resides in the DMZ/DC/Campus with one arm connected to the Internet and
another arm going into a trusted corporate network. First, it is necessary to understand physical port
connections before proceeding to the actual configuration.

Connect Cables - Physical Port Connections:

The following snippet outlines Mist Edge port configuration requirements:

Note : OOBM IP and Tunnel IP are different IP addresses and need to be from different subnets.

Request to Keep the Switch ports to which Data ports (ge0,ge1) on ME-X1 or (Xe0,Xe1) on
ME-X5 and (Xe0,Xe1,Xe2,Xe3) on ME-X5-M and ME-X10 are connected shut down until Mist Edge
is configured for Tunnel IP and Mist Tunnel details.

1. Out of Band Management Port:

7
Connect Out-Of-Band-Management Port (OOBM) of the Mist Edge to an untagged interface of your
switch. OOBM port is used by the Mist Edge to communicate to the Mist Cloud:

Note: OOBM port on the Mist Edge appliance is marked as “MIST”. By default OOBM port is
configured to obtain an IP address via DHCP, it can be later changed to use static IP configuration.

The following figures shows OOBM port on X1 Mist Edge appliance:

Mist Edge comes pre-loaded with a custom debian linux installed. To configure static IP on the
OOBM port, add the following lines to the interfaces config. Use iDrac interface or connect keyboard
and monitor to the appliance for the OOBM initial staging if DHCP is not available. The default
username and password for Mist Edge appliance is mist / <Claim-code>, default root (su -) password is
<Claim-code>. Note the right interface id based on your MistEdge Appliance Model:

nano /etc/network/interfaces

iface eno1 inet static

address 192.168.50.50/24
gateway 192.168.50.1
dns-nameservers 8.8.8.8 8.8.4.4

8
Please update the DNS entries as well.

nano /etc/resolv.conf

nameserver 8.8.8.8
nameserver 8.8.4.4

After saving the file, reboot the Mist Edge to apply the settings.

OOBM Interface ID per Mist Edge (ME) model:

Mist Edge Appliance Model Interface Id

X1 eno1

X5 eno3

X5-M / X10 enp59s0f0 (for Deb9 based ME) , ens1f0 (for

Deb-10 based ME)

Note: All ZTP capable Mist Edge shipped with claim code are Debian 10 based.

The ‘OOBM IP’ received through DHCP or assigned static while bringing up the Mist Edge is
different from ‘Tunnel IP’ that is entered in the Mist Edge details on Mist Dashboard (Mist UI

So 2 IP addresses need to be set aside for Mist Edge , one for OOBM and other for Tunnel IP, they
should be from different subnets.

In order for the Mist Edge to communicate to the Mist Cloud the following FQDNs and ports must be
allowed for the OOBM interface.

For US cloud environment:

ep-terminator.mistsys.net : TCP port 443

For GCP cloud environment:

ep-terminator.gc1.mist.com : TCP port 443

For EU cloud environment:

ep-terminator.eu.mistsys.net : TCP port 443

9
2. Tunnel IP or Downstream Port:

Connect your Downstream port to the untrusted side of your network that typically goes to
your firewall. Downstream Port must be connected to the untagged interface.

Make sure that your router/FW either does Port Forwarding to the Tunnel Interface IP
address (UDP port 1701) This is the interface/IP to which APs from a site or multiple site will
be talking to in order to establish a L2TPv3 tunnel:

Note: Tunnel IP SVI on Mist Edge is a protected interface , so even it its not connected to
Firewall , it is only open for ports UDP: 1701 (L2TPv3) , 500 and 4500 (IPsec) and TCP port
2083 for Radsec.

For Remote worker use case alone Mist Edge will be using UDP port 500,4500 and TCP port
2083 , all other Campus and Branch use cases it will just use UDP port 1701

3. Upstream Data Port:

Connect your Upstream port to the trusted side of the network. This interface would
typically connect to your core/agg switch with all the necessary user VLANs tagged.

10
Now after all interfaces have been connected to the correct ports, it is time to register and configure
Mist Edge in the Mist Cloud Dashboard.

11
Mist Edge Claim on the Mist Dashboard:

On the Mist Dashboard navigate to Organization → Mist Edges and Click ‘Claim Mist Edge’:

Enter the claim code received in PO or present on the service Tag:

Claim Code can be found on the service Tag of Mist Edge located below the power button as shown
below. Service Tag can be pulled out:

Ensure Mist Edge is powered on and the Power button shows Green.

12
After the Mist Edge is claimed it will show up as Disconnected and Registered, select it to edit
settings:

13
Mist Edge will download Tunnel terminator service and Reboot in 3 minutes to show connected.

This reboot is only the first time when Mist Edge is brought online, future service upgrades only take
3 seconds and does not require Reboot.

In case of Mist Edge not showing connected even after 5 minutes , one can SSH to the Mist Edge
appliance using the Out-Of-Band management IP address that we have configured in the previous
step. The default username and password for Mist Edge appliance is mist /<Claim-code>, default root
password is <Claim-code>. Make sure you drop into root (su -) to issue a few debug commands .
Issue the following commands to check connectivity to Mist Cloud:

a. Issue ‘mxagent info’ , which will show output similar to below:

mxagent info

Status: Registered

IP: 10.2.10.224

Mist Reachable: Yes

Org ID: <Current-Org-Id to which it is Registered>

b. ping ep-terminator.mistsys.net

If Ping is successful , request to ensure 443 outbound to ep-terminator.mistsys.net is

allowed , which should ensure Mist Edge shows up connected.

Note: Above URL will be ep-terminator.gc1.mist.com or ep-terminator.eu.mistsys.net if its GCP or EU

cloud instance that Mist Edge is being claimed against.

14
In the setting page first enable “Separate Upstream and Downstream Traffic”. Assign correct
interface IDs to the correct interfaces. In the below example we are using X1 Mist Edge, where ge0
interface is connected to the public untrusted side and ge1 interface is connected to the corporate
network with all the user VLANs tagged:

Note: a. Upstream Port VLAN ID is optional and should only be used whenever the upstream
switchport is configured as an access port with a single VLAN untagged.

b. The ‘OOBM IP’ received through DHCP or assigned static while bringing up the Mist Edge is
different from ‘Tunnel IP’ that is entered in the Mist Edge details on Mist Dashboard (Mist UI).

So 2 IP addresses need to be set aside for Mist Edge , one for OOBM and other for Tunnel IP, they
need to be from different subnets.

Based on your Mist Edge model the interface IDs might be different. Please use the image below that
show individual model port mappings:

Note: Request to keep the data ports on switch side , that is corresponding ports to ge0,ge1 or
xe0,xe1 or xe0,xe1,xe2,xe3 shutdown until Mist Edge is configured with Tunnel IP and ‘Mist Tunnel
vlan’.

15
 X1

X5

X5-M
and

X10

Note:

1. Instead of a Dual arm port config, that is separate port for Downstream and Upstream , one can
useSingle arm - one port or multiple ports in the Port channel , where the corresponding switch port
is trunk with Tunnel IP being native/untagged , rest of the client vlans are tagged.

If multiple ports are used they will be part of the port channel , listed below are those settings.

Mist Edge auto detects port channel.

16
Single Arm, Single port used Single Arm, Multiple ports in Port channel

2. For a ME-X5-M and ME-X10 , one can do a port channel for Downstream and Upstream, where
Downstream port channel is untagged Tunnel IP vlan and Upstream port channel is all tagged client
vlan.

17
3. For ME-X5, ME-X5-M and ME-X10 fiber ports;

a. Request to use SFP Cable OM3 50/125μm / OM4.

OS2 version is not supported.

If the OS2 version is used, frequent LACP flap, packet errors, port not coming up will be observed.

The above issues get resolved once the SFP cable is replaced with OM3/OM4 type.

b. If for ME-X5-M and ME-X10 , copper ports are required for OOBM (management port
labelled as mist) , a Juniper Fiber to converter can be requested.

Create a Mist Edge Cluster:

Now that all the necessary services have been provisioned let’s create a Mist Edge Cluster and add
Mist Edge in there:

Under Mist Edge Cluster configuration, we will need to set our Cluster IP address(es) or FQDNs for
the remote APs to communicate to.

In case your Firewall/Router is not doing a port forward to the Tunnel IP interface, IP specified on
the Mist dge cluster will be the same IP configured on Mist Edge.

In case your Firewall/Router is doing a port forward to the Tunnel IP interface, you will need to
specify the external IP address of your Firewall/Router that translates to Tunnel IP . In case multiple
Mist Edges are part of the cluster, their respective IP addresses should be listed there, comma
separated:

18
Time to move to the next step and create a Mist Tunnel.

Setup the Mist Tunnel

Navigate to Organization → Mist Tunnels and Create a new Tunnel. Typically this is where you would
list all your user VLANs (Client vlan) that you would like to extend from your corporate network to
the APs . The VLAN list should be comma separated:

Once you create a Mist Tunnel, specify all user VLANs required to be tunneled back, assign the
tunnel to the Mist Edge Cluster (s) we have created earlier, leave the rest of the setting as it is:

Configure and prepare the SSID

19
The best way to provision your corporate SSID to extend vlan is to leverage Config Templates.

Navigate to Organization → Config Templates.

Create config template and use template assignment for either

a) Specific Sites or Site-Group, where individual sites will be mapped into a Site Group

or

b) Entire Org with actual office Sites added as exceptions. For example the following template will be
assigned to all Sites, except Sites “HQ”, “BranchA”, and “BranchB”.

Config Template assigned to Site/Site group Entire Org with some site exclusion.

SSID settings would depend upon particular customer requirements, but below are the most
important parts with regards to vlan data tunneling back to the corporate network.

20
Note: Configuring one Config template per Wireless LAN makes it easier to manage SSIDs that need
to be broadcast at a given site by using site and Site-groups.

Create a Site for Campus, Branches

Sites can be created using UI under Organization → Site Configuration

Please note the following guidelines:

● For all APs, use AP firmware version 0.8.21301

APs mapped to Tunneled Wireless LAN will form L2TPv3 tunnel and status can be confirmed on AP
table as well as Mist Edge inventory.

21
Mist Edge Insights

One can launch Mist Edge Insights from Mist Edge details.

Mist Edge Insights provide Insights into Tunnel trend, Mist Edge Events (service restarts, config
changes, upgrade, Mist Edge reboots).

It also shows the Time series and list view of Data ports.

Provides LACP status and LACP neighbor info, makes it easier to verify the upstream switch port
connections.

22
23
Troubleshooting
To see established L2TPv3 tunnels from the MistEdge perspective:

Note: Ensure that ‘curl http://localhost:9110/debug/l2tp’ shows a listener as

shown below, if there isn’t a listener that indicates Mist Cluster not
updated with Tunnel IP.

root@ME-Production-005:~# curl http://localhost:9110/debug/l2tp

1 tunnels, 1 listeners.

Tunnels by state:

State established-with-sessions: 1

tunnel between 10.137.70.14:43346 - Branch-3160-AP-001 - router-id

176.133.250.183 and 10.1.2.22:1701

state established-with-sessions

listener at 10.1.2.22:1701

Packet Captures on the Mist Edge

Currently the packet capture facility on the Mist Edge is local to the appliance only, but it can be very
useful to troubleshoot datapath at different entry points (inbound physical port, l2tpv3 tunnel, drop
etc). In order to enable packet captures into the cli shell, it is necessary to instal tshark:

apt-get install tshark

After the tshark is installed you could use port debug command to list all the interfaces you can
capture on:

root@ME-Production-005:~# curl http://localhost:9110/debug/ports

Port 0 "port0":

PCI address: "0000:13:00.0"

MAC: 00-0c-29-22-a4-d1

PMD: "net_vmxnet3"

link: true, duplex: true, Speed: 10000 Mbps

state: Forwarding

24
 Rx: 314267822 bytes, 2253968 packets, 0+0 errors

Tx: 282976995 bytes, 1947497 packets, 0 errors

rx_good_packets: 2253968

tx_good_packets: 1947497

rx_good_bytes: 314267822

tx_good_bytes: 282976995

rx_q0packets: 2253968

rx_q0bytes: 314267822

tx_q0packets: 1947497

tx_q0bytes: 282976995

Port 1 "port1":

PCI address: "0000:1b:00.0"

MAC: 00-0c-29-22-a4-db

PMD: "net_vmxnet3"

link: true, duplex: true, Speed: 10000 Mbps

state: Forwarding

Rx: 640727016 bytes, 1387326 packets, 0+79 errors

Tx: 279571047 bytes, 1783598 packets, 0 errors

rx_good_packets: 1387326

tx_good_packets: 1783598

rx_good_bytes: 640727016

tx_good_bytes: 279571047

rx_missed_errors: 79

rx_q0packets: 1387326

rx_q0bytes: 640727016

tx_q0packets: 1783598

tx_q0bytes: 279571047

25
Bridge port vlans:

[0] port0, PVID 1, Inactive Vlans [1]

[1] port1, Inactive Vlans [100]

[4] kni0, Inactive Vlans [1]

[11] L2TP session "mxtunnel" with 192.168.51.122:0 (d4-20-b0-02-63-5e),

Active Vlans [100]

Based on the example above, below are some sample packet capture syntax commands (more info
available at tt-pcap --help)

root@ME-Production-005:~# tt-pcap -port=1 udp port 67 | tshark -nr -

Running as user "root" and group "root". This could be dangerous.

1 0.000000000 0.0.0.0 ? 255.255.255.255 DHCP 346 DHCP Discover -

Transaction ID 0x12e3913a

2 0.103353790 192.168.100.1 ? 192.168.100.135 DHCP 337 DHCP Offer -

Transaction ID 0x12e3913a

3 1.521102670 0.0.0.0 ? 255.255.255.255 DHCP 346 DHCP Request -

Transaction ID 0x12e3913a

4 2.133698590 192.168.100.1 ? 192.168.100.135 DHCP 337 DHCP ACK -

Transaction ID 0x12e3913a

26