MikroTik - RouterOSv7 First Look - L3 ASIC Performance Testing
MikroTik - RouterOSv7 First Look - L3 ASIC Performance Testing
net
When MikroTik announced the CRS3xx series switches a few years ago, one of the most exciting
aspects of that news release was the prospect of L3 forwarding in hardware on very inexpensive
devices.
A quick review of the Marvell Prestera ASIC family showed a number of advanced routing,
switching, MPLS and VxLAN capabilites.
Fast forward to 2020, where MikroTik has started to enable some of those features in RouterOS
v7 beta.
Now we can finally take some of the CRS3xx switches and test their capabilities with L3
forwarding performance in hardware
Here is a chart from MikroTik that outlines ACL rule count, Unicast FDB entries and MTU size.
https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-performance-testing/ 1/16
29/09/2022 10:58 MikroTik – RouterOSv7 first look – L3 ASIC performance testing – StubArea51.net
MIkroTik has been working on the development of the features listed below to offload into
hardware.
For the tests in this article, we’ll be using IPv4 Unicast and Inter-VLAN routing.
https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-performance-testing/ 2/16
29/09/2022 10:58 MikroTik – RouterOSv7 first look – L3 ASIC performance testing – StubArea51.net
(https://iparchitechs.com/contact-us/)
https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-performance-testing/ 3/16
29/09/2022 10:58 MikroTik – RouterOSv7 first look – L3 ASIC performance testing – StubArea51.net
MikroTik Configuration
https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-performance-testing/ 4/16
29/09/2022 10:58 MikroTik – RouterOSv7 first look – L3 ASIC performance testing – StubArea51.net
/interface bridge
add mtu=9216 name=bridge1 vlan-filtering=yes
add name=lo0
/interface ethernet
set [ find default-name=sfp-sfpplus15 ] comment="Proxmox - ens2f0" l2mtu=10218 mtu=92
16
set [ find default-name=sfp-sfpplus16 ] comment="Proxmox - ens2f1" l2mtu=10218 mtu=92
16
/interface vlan
add interface=bridge1 mtu=9216 name=vlan103 vlan-id=103
add interface=bridge1 mtu=9216 name=vlan104 vlan-id=104
/interface ethernet switch
set 0 l3hw=yes
/ip pool
add name=dhcp_pool0 ranges=10.255.34.11-10.255.34.254
add name=dhcp_pool1 ranges=10.255.35.2-10.255.35.254
/ip dhcp-server
add address-pool=dhcp_pool0 disabled=no interface=vlan103 name=dhcp1
add address-pool=dhcp_pool1 disabled=no interface=vlan104 name=dhcp2
/ip vrf
add list=all name=main
/interface bridge port
add bridge=bridge1 interface=sfp-sfpplus15 pvid=103
add bridge=bridge1 interface=sfp-sfpplus16 pvid=104
/interface bridge vlan
add bridge=bridge1 tagged=bridge1 untagged=sfp-sfpplus15 vlan-ids=103
add bridge=bridge1 tagged=bridge1 untagged=sfp-sfpplus16 vlan-ids=104
/ip address
add address=10.255.34.1/24 interface=vlan103 network=10.255.34.0
add address=10.255.35.1/24 interface=vlan104 network=10.255.35.0
/ip dhcp-server network
add address=10.255.34.0/24 dns-server=9.9.9.9 gateway=10.255.34.1
add address=10.255.35.0/24 dns-server=9.9.9.9 gateway=10.255.35.1
https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-performance-testing/ 5/16
29/09/2022 10:58 MikroTik – RouterOSv7 first look – L3 ASIC performance testing – StubArea51.net
iperf3 results
iperf3 results
https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-performance-testing/ 6/16
29/09/2022 10:58 MikroTik – RouterOSv7 first look – L3 ASIC performance testing – StubArea51.net
iperf3 results
Conclusions
The initial results are very promising. Getting close to 10G sustained L3 throughput using an ASIC
on a device that lists for $399 USD is unheard of.
The most noteworthy items for improvement are the number of TCP retransmits in the iperf
https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-performance-testing/ 7/16
29/09/2022 10:58 MikroTik – RouterOSv7 first look – L3 ASIC performance testing – StubArea51.net
Normally, most ASIC based platforms will push 96 bytes through as fast as 1500 bytes. The
retransmits suggest that more work is needed in the way RouterOS interfaces with the switching
buffers for L3 HW offload
This is also on a beta version of RouterOS that will still go through many revisions before going
into prod so I would expect to see the performance to improve as the code matures.
All things considered though, things are looking great to take the CRS3xx series and be able to
deploy them as a true L3 switch in prod sometime in 2021.
PUBUDEUX
December 13, 2020 at 6:46 pm (https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-
performance-testing/#comment-123760)
I tried using your exact config on a CRS317 with 7.1beta3 and for some strange
reason Level 3 hw offloading does not seem to be working. Inter-vlan routing goes
through the CPU no matter what I try.
Do you know if there’s something that I might need to do outside of the config?
Another thing I noticed is there is no option for l3hw=fw as it says there should be in
the docs. Maybe support was removed in beta3?
S://STUBAREA51.NET/2020/10/12/MIKROTIK-ROUTEROSV7-FIRST-LOOK-L3-ASIC-PERFORMANCE-TESTING/?REPLYTOCOM=123760#RESPOND)
SHAKTI PLASMOSIS
February 19, 2021 at 8:23 am (https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-
performance-testing/#comment-128837)
L3 offload is broken in 7.1beta3; Beta 4 has been released, fixing this and offering a
more versatile implementation of L3 offload which now allows enabling the feature
per-port.
/STUBAREA51.NET/2020/10/12/MIKROTIK-ROUTEROSV7-FIRST-LOOK-L3-ASIC-PERFORMANCE-TESTING/?REPLYTOCOM=128837#RESPOND)
https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-performance-testing/ 8/16
29/09/2022 10:58 MikroTik – RouterOSv7 first look – L3 ASIC performance testing – StubArea51.net
Tried it on a CRS305 with 7.1b6. Got a little over 200Mb/s. Yep, bits. Here is the
/export
/interface bridge
add mtu=9216 name=bridge1 vlan-filtering=yes
add name=lo0
/interface ethernet
set [ find default-name=sfp-sfpplus3 ] comment=”Proxmox – ens2f0″ l2mtu=10218 \
mtu=9216
set [ find default-name=sfp-sfpplus4 ] comment=”Proxmox – ens2f1″ l2mtu=10218 \
mtu=9216
/interface vlan
add interface=bridge1 mtu=9216 name=vlan103 vlan-id=103
add interface=bridge1 mtu=9216 name=vlan104 vlan-id=104
/interface ethernet switch
set 0 l3-hw-offloading=yes
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp_pool0 ranges=10.255.34.11-10.255.34.254
add name=dhcp_pool1 ranges=10.255.35.2-10.255.35.254
/ip dhcp-server
add address-pool=dhcp_pool0 disabled=no interface=vlan103 name=dhcp1
add address-pool=dhcp_pool1 disabled=no interface=vlan104 name=dhcp2
/interface bridge port
add bridge=bridge1 interface=sfp-sfpplus3 pvid=103
add bridge=bridge1 interface=sfp-sfpplus4 pvid=104
/interface bridge vlan
add bridge=bridge1 tagged=bridge1 untagged=sfp-sfpplus3 vlan-ids=103
add bridge=bridge1 tagged=bridge1 untagged=sfp-sfpplus4 vlan-ids=104
/ip address
add address=10.255.34.1/24 interface=vlan103 network=10.255.34.0
add address=10.255.35.1/24 interface=vlan104 network=10.255.35.0
/ip dhcp-server network
add address=10.255.34.0/24 dns-server=9.9.9.9 gateway=10.255.34.1
add address=10.255.35.0/24 dns-server=9.9.9.9 gateway=10.255.35.1
/system routerboard settings
set boot-os=router-os
https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-performance-testing/ 9/16
29/09/2022 10:58 MikroTik – RouterOSv7 first look – L3 ASIC performance testing – StubArea51.net
Any ideas very welcome. We would love to use the CRS series, but lack of wire speed
an issue.
Thanks
S://STUBAREA51.NET/2020/10/12/MIKROTIK-ROUTEROSV7-FIRST-LOOK-L3-ASIC-PERFORMANCE-TESTING/?REPLYTOCOM=131514#RESPOND)
MikroTik just enabled L3 forwarding for all 3xx series switches (vs. just a few) with
the latest release so it’s possible there may be some bugs. Your config looks correct.
If it’s not working, I would post in the v7 beta forum at forum.mikrotik.com
/STUBAREA51.NET/2020/10/12/MIKROTIK-ROUTEROSV7-FIRST-LOOK-L3-ASIC-PERFORMANCE-TESTING/?REPLYTOCOM=131565#RESPOND)
RONALD HALEY
June 10, 2021 at 3:47 pm (https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-
performance-testing/#comment-131672)
Kevin,
Took your advice and posted on MT forum. Apparently L3 forwarding doesn’t support
large MTU’s yet, although it appeared to work for you. Perhaps they changed the coed
when they expanded it to all CRS.
Like your work btw:)
S://STUBAREA51.NET/2020/10/12/MIKROTIK-ROUTEROSV7-FIRST-LOOK-L3-ASIC-PERFORMANCE-TESTING/?REPLYTOCOM=131672#RESPOND)
https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-performance-testing/ 10/16
29/09/2022 10:58 MikroTik – RouterOSv7 first look – L3 ASIC performance testing – StubArea51.net
Gotcha, that makes sense. There are definitely a lot of things changing weekly in
ROSv7 so this isn’t surprising. The routing filters are being re-written because they
were too complex.
Thanks for the feedback on my posts – always great to hear that people find it
useful
/STUBAREA51.NET/2020/10/12/MIKROTIK-ROUTEROSV7-FIRST-LOOK-L3-ASIC-PERFORMANCE-TESTING/?REPLYTOCOM=131688#RESPOND)
LEAVE A REPLY
Your email address will not be published. Required fields are marked *
Comment *
Name *
Email *
Website
POST COMMENT
MikroTik – RouterOSv7 first look – Dynamic routing with IPv6 and OSPFv3/BGP
(https://stubarea51.net/2020/12/30/mikrotik-routerosv7-first-look-dynamic-routing-with-ipv6-and-ospfv3-bgp/)
https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-performance-testing/ 11/16
29/09/2022 10:58 MikroTik – RouterOSv7 first look – L3 ASIC performance testing – StubArea51.net
FOLLOW US
ABOUT US
Links (https://stubarea51.net/links/)
RECENT COMMENTS
RECENT POSTS
https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-performance-testing/ 12/16
29/09/2022 10:58 MikroTik – RouterOSv7 first look – L3 ASIC performance testing – StubArea51.net
Search…
ARCHIVES
https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-performance-testing/ 14/16
29/09/2022 10:58 MikroTik – RouterOSv7 first look – L3 ASIC performance testing – StubArea51.net
CATEGORIES
BGP (https://stubarea51.net/category/bgp/)
BRAS (https://stubarea51.net/category/bras/)
Cisco (https://stubarea51.net/category/cisco/)
FISP (https://stubarea51.net/category/fisp/)
IPv6 (https://stubarea51.net/category/ipv6/)
Juniper (https://stubarea51.net/category/juniper/)
MikroTIk (https://stubarea51.net/category/mikrotik/)
MPLS (https://stubarea51.net/category/mpls/)
NFV (https://stubarea51.net/category/nfv/)
OSPF (https://stubarea51.net/category/ospf/)
PPPoE (https://stubarea51.net/category/pppoe/)
RouterOS v7 (https://stubarea51.net/category/routeros-v7/)
routing (https://stubarea51.net/category/routing/)
Security (https://stubarea51.net/category/security/)
Switching (https://stubarea51.net/category/switching/)
Uncategorized (https://stubarea51.net/category/uncategorized/)
https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-performance-testing/ 15/16
29/09/2022 10:58 MikroTik – RouterOSv7 first look – L3 ASIC performance testing – StubArea51.net
Virtualization (https://stubarea51.net/category/virtualization/)
Whitebox (https://stubarea51.net/category/whitebox/)
WISP (https://stubarea51.net/category/wisp/)
META
Log in (https://stubarea51.net/wp-login.php)
WordPress.org (https://wordpress.org/)
(http://www.iparchitechs.com)
https://stubarea51.net/2020/10/12/mikrotik-routerosv7-first-look-l3-asic-performance-testing/ 16/16