14/8/24, 15:33 [EX/QFX] How to calculate and to optimize TCAM usage in firewall filters

[EX/QFX] How to calculate and to optimize TCAM usage in firewall filters

Article ID KB30953 Created 2016-05-27 Last Updated 2019-05-07

Description

This article explains how to calculate the number of TCAM (Ternary Content Addressable Memory) entries a particular firewall filter term will take up. Based on this, one can
optimize the matching conditions to use less TCAM space.

Symptoms

To understand how firewall filters are programmed into the TCAM and therefore be better able to optimize them to reduce overall TCAM usage.

Solution
The TCAM needs to create a separate entry for every possible combination of matching items within a firewall filter term. Below are examples of how the TCAM entries are
calculated, how to check your TCAM usage, and related TCAM KB articles.
Examples showing how TCAM entries are calculated
Example 1

In the following configuration, this term will only take one rule in the TCAM because there is only one of each type of matching condition. When they are multiplied
together, this makes for on TCAM entry: 1 x 1 x 1 x 1 = 1 TCAM entry.

{master:0}[edit firewall family inet filter FOO]

user@switch# show
term one {
from {
source-address {
10.10.10.0/24;
}
destination-address {
20.20.20.0/24;
}
source-port 65000;
destination-port ssh;
}
then accept;
}

Example 2

https://supportportal.juniper.net/s/article/EX-QFX-How-to-calculate-and-to-optimize-TCAM-usage-in-firewall-filters 1/10
14/8/24, 15:33 [EX/QFX] How to calculate and to optimize TCAM usage in firewall filters
In the following configuration, multiple types of terms yields a different result. Here, there are two options for source-address and two options for destination-address.
This creates 4 possible ways to match on this term: 2 x 2 x 1 x 1 = 4 TCAM entries.

{master:0}[edit firewall family inet filter FOO]

user@switch# show
term one {
from {
source-address {
10.10.10.0/24;
30.30.30.0/24;
}
destination-address {
20.20.20.0/24;
40.40.40.0/24;
}
source-port 65000;
destination-port ssh;
}
then accept;
}

The 4 TCAM entries are:

term1.1: match [Source IP A, Destination IP C, Source Port E, Destination Port F]
term1.2: match [Source IP A, Destination IP D, Source Port E, Destination Port F]
term1.3: match [Source IP B, Destination IP C, Source Port E, Destination Port F]
term1.4: match [Source IP B, Destination IP D, Source Port E, Destination Port F]
Notice that since there is only one source-port and one destination-port, these two matching conditions do not increase the total number of TCAM entries needed.

Example 3

There are some things to keep in mind. Most switches will not automatically summarize continuous network address prefixes, so the following configuration will result in
4 TCAM entries.

{master:0}[edit firewall family inet filter FOO]

user@switch# show
term one {
from {
source-address {
10.0.0.0/24;
10.0.1.0/24;
10.0.2.0/24;
10.0.3.0/24;
}
}
then accept;
}

However, these 4 subnets can be summarized, so if you configure them the following way, then you will have just 1 TCAM entry used for this term.

{master:0}[edit firewall family inet filter FOO]

user@switch# show
term one {
from {
source-address {
10.0.0.0/22;
}
}
then accept;
}

This can make a huge difference when you have many entries for both source and destination prefixes.
Caution: Care should be taken when summarizing network prefixes, as depending on the summarization done, you may be matching against a larger set of IP addresses
than you were when using a series of smaller, more specific subnets. In that case, you will need to weigh what is more important: security (with permitting more
addresses) or TCAM space reduction.

Example 4

https://supportportal.juniper.net/s/article/EX-QFX-How-to-calculate-and-to-optimize-TCAM-usage-in-firewall-filters 2/10
14/8/24, 15:33 [EX/QFX] How to calculate and to optimize TCAM usage in firewall filters
The following example has 8 source-address subnets, 8 destination-address subnets, 2 source-ports and 2 destination-ports. This makes for a total of 8 x 8 x 2 x 2 = 256
TCAM entries.
{master:0}[edit firewall family inet filter FOO]
user@switch# show
term one {
from {
source-address {
10.0.0.0/24;
10.0.1.0/24;
10.0.2.0/24;
10.0.3.0/24;
10.0.4.0/24;
10.0.5.0/24;
10.0.6.0/24;
10.0.7.0/24;
}
destination-address {
20.0.0.0/24;
20.0.1.0/24;
20.0.2.0/24;
20.0.3.0/24;
20.0.4.0/24;
20.0.5.0/24;
20.0.6.0/24;
20.0.7.0/24;
}
source-port [ 100 200 ];
destination-port [ ssh bgp ];
}
then accept;
}

If the configuration is changed to summarize the source and destination prefixes, we can produce a term that only uses 4 TCAM entries :

{master:0}[edit firewall family inet filter FOO]

user@switch# show
term one {
from {
source-address {
10.0.0.0/21;
}
destination-address {
20.0.0.0/21;
}
source-port [ 100 200 ];
destination-port [ ssh bgp ];
}
then accept;
}

This is an extreme example but illustrates an important point. If you summarize source and destination prefixes it can have a large impact on TCAM space needed.

Caution: Care should be taken when summarizing network prefixes, as depending on the summarization done, you may be matching against a larger set of IP addresses
than you were when using a series of smaller, more specific subnets. In that case, you will need to weigh what is more important: security (with permitting more
addresses) or TCAM space reduction.

Note: Support for automatically summarizing contiguous prefixes varies between switches. Some switches support doing this automatically upon programming the term
conditions in the TCAM, whereas many will require manual optimization for most efficient TCAM usage.

Example 5

https://supportportal.juniper.net/s/article/EX-QFX-How-to-calculate-and-to-optimize-TCAM-usage-in-firewall-filters 3/10
14/8/24, 15:33 [EX/QFX] How to calculate and to optimize TCAM usage in firewall filters
The behavior is different with source-port and destination-port. The switches will optimize contiguous port numbers as much as is possible by using bit mask ranges. The
same principle as network address summarization.

In the following configuration, one would expect 100 TCAM entries, but the optimization manages to reduce the number of TCAM entries to only 7.

{master:0}[edit firewall family inet filter FOO]

user@switch# show
term one {
from {
source-port 101-200;
}
then accept;
}

When you compare this to the following term, which uses the same number of port numbers, but they are not contiguous so the switch cannot optimize using bit masks.
This one takes 100 TCAM Entries .

{master:0}[edit firewall family inet filter FOO]

user@switch# show
term one {
from {
source-port [ 000 010 020 030 040 050 060 070 080 090 100 110 120 130 140 150 160 170 180 190 200
210 220 230 240 250 260 270 280 290 300 310 320 330 340 350 360 370 380 390 400 410 420 430 440 450 460 470
480 490 500 510 520 530 540 550 560 570 580 590 600 610 620 630 640 650 660 670 680 690 700 710 720 730 740
750 760 770 780 790 800 810 820 830 840 850 860 870 880 890 900 910 920 930 940 950 960 970 980 990 ];
then accept;
}

Tip: Keep in mind that that if your firewall filter does not contain an explicit default term, then the implicit default term that discards all traffic will add one extra TCAM
entry to your firewall filter.

Checking your TCAM usage

Checking PFE usage requires going into the PFE and the commands used depend on the model:
For EX4300, EX4600, QFX3500, QFX5100

https://supportportal.juniper.net/s/article/EX-QFX-How-to-calculate-and-to-optimize-TCAM-usage-in-firewall-filters 4/10
14/8/24, 15:33 [EX/QFX] How to calculate and to optimize TCAM usage in firewall filters
On QFX5100, run the following CLI command to get an overview of TCAM usage:

{master:0}
user@QFX5100> show pfe filter hw summary

Slot 0

Group Group-ID Allocated Used Free

---------------------------------------------------------------------------
> Ingress filter groups:
iRACL group 14 512 21 491
> Egress filter groups:

On broadcom chipsets in general you can check the TCAM usage directly in the PFE:

https://supportportal.juniper.net/s/article/EX-QFX-How-to-calculate-and-to-optimize-TCAM-usage-in-firewall-filters 5/10
14/8/24, 15:33 [EX/QFX] How to calculate and to optimize TCAM usage in firewall filters
{master:0}
user@EX4300> start shell
user@EX4300:RE:0% vty fpc0

BSD platform (QorIQ P202H processor, 0MB memory, 0KB flash)

(vty)# show filter

Program Filters:
---------------
Index Dir Cnt Text Bss Name
-------- ------ ------ ------ ------ --------

Term Filters:
------------
Index Semantic Name
-------- ----------------
1 Classic test --> A firewall filter with name 'test' has hw index 1
17000 Classic __default_arp_policer__
57008 Classic __cfm_filter_shared_lc__
16777216 Classic fnp-filter-level-all
46137360 Classic pfe-cos-cl-557-5-2
46137361 Classic pfe-cos-cl-558-5-2
46137362 Classic pfe-cos-cl-559-5-2

Resolve Filters:
---------------
Index
--------

(vty)# show filter hw 1 show_term_info --> The number used here is the hw index number discovered via the previous command
======================
Filter index : 1
======================

- Filter name : test

+ Hardware Instance : 1
+ Hardware key (struct pfe_bcm_dfw_hw_key_t):
- Type : IRACL
- Vlan id : 0
- Direction : ingress
- Protocol : 2 (IPv4)
- Port class id : 0
- Class id : 1
- Loopback : 0
- Vlan tag : 0
+ FP usage info (struct pfe_bcm_dfw_fp_t):
- Group : IFP iRACL group (3)
- List of tcam entries : [ total: 152; 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86
87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116
117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143
144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170
171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197
198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 ]
- List of ranges : [ total: 0; ]
+ Misc info (struct pfe_bcm_dfw_misc_info_t):
+ Bind point info (union pfe_bcm_dfw_bind_point_info_t):
+ Class id : 1
- Vlans : [4093 (total:1/4096)]
+ AE intf match list:
+ Programmed: YES
+ Total TCAM entries available: 7016
+ Total TCAM entries installed : 152
+ Term Expansion:
- Term 1: will expand to 151 terms: Name "term-one"
- Term 2: will expand to 1 term : Name "final"
+ Term TCAM entry requirements:
- Term 1: needs 151 TCAM entries: Name "term-one" --> Total number of TCAM entries needed per term
- Term 2: needs 1 TCAM entry : Name "final"
+ Total TCAM entries available: 7016 --> Total number of TCAM entries available
+ Total TCAM entries installed : 152 --> Total number of TCAM entries used
Total hardware instances: 1

https://supportportal.juniper.net/s/article/EX-QFX-How-to-calculate-and-to-optimize-TCAM-usage-in-firewall-filters 6/10
14/8/24, 15:33 [EX/QFX] How to calculate and to optimize TCAM usage in firewall filters
(vty)# exit

% exit

{master:0}
user@EX4300>

For EX2200, EX3200, EX3300, EX4200, EX4500, EX4550, EX8200:

{master:0}
user@EX4200> start shell
% vty fpc0

BSD platform (MPC 8544 processor, 48MB memory, 0KB flash)

PFEM0(vty)# show tcam vendor 1 rules
Number of rules as Ingress PACL: 0
Number of rules as Ingress VACL: 0
Number of rules as Ingress RACL: 152 ---> This particular switch has one layer 3 firewall filter using a total of 152 terms.
Number of rules as Egress PCL: 0
152 Ingress RACL rules

HW-index Page_id Entry_id rule_size fw_id fmt Rule

--------------------------------------------------------------------------------
472 118 0 2 5 4 term-one.ext.0 --> each line in TCAM used shows the corresponding term name
474 118 2 2 5 4 term-one.ext.1
.
.
.
13894 3473 2 2 5 4 term-one.ext.149
14112 3528 0 2 5 4 term-one.ext.150
14114 3528 2 2 5 4 final.ext.0

TCAM utilization: 152(used), 6886(free), 7038(total) --> Here it shows TCAM space used, free and total available.

PFEM0(vty)# exit
% exit
{master:0}
user@EX4200>

For QFX5200:

https://supportportal.juniper.net/s/article/EX-QFX-How-to-calculate-and-to-optimize-TCAM-usage-in-firewall-filters 7/10
14/8/24, 15:33 [EX/QFX] How to calculate and to optimize TCAM usage in firewall filters
To calculate the TCAM entries available and Total TCAM entries needed in QFX5200, do the following:
1. Run 'show filter hw groups' on the QFX5200.
2. Look for the IFP iRACL group. See the blue highlighted output below. Entries used are 520, and Max entries are 768. The remaining entries are 248 (768 - 520 = 248).

FPC0(FRA vty)# show filter hw groups

Unit:0 Group Information:
VFP groups:
IFP groups:
BA classifier dynamic group id: 21. Pipe: 0 Entries: 76 Max Entries(total_available): 256( 256) Pri: 2 Slice: 1 Def Entries: 0
BA classifier dynamic group id: 22. Pipe: 1 Entries: 76 Max Entries(total_available): 256( 256) Pri: 2 Slice: 1 Def Entries: 0
BA classifier dynamic group id: 23. Pipe: 2 Entries: 76 Max Entries(total_available): 256( 256) Pri: 2 Slice: 1 Def Entries: 0
BA classifier dynamic group id: 24. Pipe: 3 Entries: 76 Max Entries(total_available): 256( 256) Pri: 2 Slice: 1 Def Entries: 0

iRACL group id: 33. Pipe: 0 Entries: 520 Max Entries(total_available): 768( 768) Pri: 6 Slice: 9 Def Entries: 0 <<<<<<
iRACL group id: 34. Pipe: 1 Entries: 520 Max Entries(total_available): 768( 768) Pri: 6 Slice: 9 Def Entries: 0 <<<<<<<
iRACL group id: 35. Pipe: 2 Entries: 520 Max Entries(total_available): 768( 768) Pri: 6 Slice: 9 Def Entries: 0 <<<<<<<
iRACL group id: 36. Pipe: 3 Entries: 520 Max Entries(total_available): 768( 768) Pri: 6 Slice: 9 Def Entries: 0 <<<<<<<

Dynamic group id: 17. Pipe: 0 Entries: 146 Max Entries(total_available): 256( 256) Pri: 4 Slice: 2 Def Entries: 0
Dynamic group id: 18. Pipe: 1 Entries: 146 Max Entries(total_available): 256( 256) Pri: 4 Slice: 2 Def Entries: 0
Dynamic group id: 19. Pipe: 2 Entries: 146 Max Entries(total_available): 256( 256) Pri: 4 Slice: 2 Def Entries: 0
Dynamic group id: 20. Pipe: 3 Entries: 146 Max Entries(total_available): 256( 256) Pri: 4 Slice: 2 Def Entries: 0

3. Run the command ' show filter hw 1 show_term_info '. The "Total TCAM entries available" listed is actually the remaining entries.

https://supportportal.juniper.net/s/article/EX-QFX-How-to-calculate-and-to-optimize-TCAM-usage-in-firewall-filters 8/10
14/8/24, 15:33 [EX/QFX] How to calculate and to optimize TCAM usage in firewall filters
FPC0(FRA vty)# show filter hw 1 show_term_info
======================
Filter index : 1
======================

- Filter name : control-plane-protection-v4

+ Hardware Instance : 1
+ Hardware key (struct brcm_dfw_hw_key_t):
- Type : IRACL
- Vlan id : 0
- Direction : ingress
- Protocol : 2 (IPv4)
- Port class id : 0
- Class id : 0
- Loopback : 1
- Port : 0(xe-1)
- Vlan tag : 0
- Non-overflow : 1
+ FP usage info (struct brcm_dfw_fp_t):
- Group : IFP iRACL group (33)
- My Mac : 00:00:00:00:00:00
- Loopback Reference Count : 00000001
- IFL Type : unknown (0)
+ List of tcam entries : [ total: 520; ]
- Pipe: 0; [1033 1037 1041 1045 1049 1053 1057 1061 1065 1069 1073 1077 1081 1085 1089 1093 1097 1101
1105 1109 1113 1117 1121 1125 1129 1133 1137 1141 1145 1149 1153 1157 1161 1165 1169 1173 1177 1181 1185 1189
1193 1197 1201 1205 1209 1213 1217 1221 1225 1229 <<SNIP>> 2964 2968 2972 2976 2980 2984 2988 2992 2996 3000
3004 3008 3012 3016 3020 3024 3028 3032 3036 3040 3044 3048 3052 3056 3060 3064 3068 3072 3076 3080 3084 3088
3092 3096 3100 3104 3108 3112 ]
+ List of ranges : [ total: 0; ]
- Pipe: 0 []
- Pipe: 1 []
- Pipe: 2 []
- Pipe: 3 []
+ List of interface match entries : [ total: 0; ]
- Pipe: 0 []
- Pipe: 1 []
- Pipe: 2 []
- Pipe: 3 []
+ List of dot1q-tag match entries : [ total: 0; ]
- Pipe: 0 []
- Pipe: 1 []
- Pipe: 2 []
- Pipe: 3 []
- List of l3 ifl index entries : [ total: 0; ]
+ List of vfp tcam entries : [ total: 0; ]
- Pipe: 0 []
- Pipe: 1 []
- Pipe: 2 []
- Pipe: 3 []
+ Misc info (struct brcm_dfw_misc_info_t):
- List of <anlz_id, entry_id> : [ total: 0; ]
+ Bind point info (union brcm_dfw_bind_point_info_t):
+ Loopback : CPU Traffic
+ Programmed: YES
+ BD ID : 223
+ Total TCAM entries available: 248
+ Total TCAM entries needed : 520
+ Term Expansion:
- Term 1: will expand to 15 terms: Name "snmp"
- Term 2: will expand to 15 terms: Name "snmp-frags"
- Term 3: will expand to 15 terms: Name "http"
- Term 4: will expand to 15 terms: Name "ssh"
- Term 5: will expand to 1 term : Name "icmp"
- Term 6: will expand to 11 terms: Name "ntp"
- Term 7: will expand to 11 terms: Name "ntp-back"
- Term 8: will expand to 3 terms: Name "dns"
- Term 9: will expand to 7 terms: Name "gre"
- Term 10: will expand to 7 terms: Name "bgp"
- Term 11: will expand to 7 terms: Name "bgp-back"
- Term 12: will expand to 1 term : Name "rsvp"
- Term 13: will expand to 1 term : Name "vrrp"
- Term 14: will expand to 1 term : Name "ospf"

https://supportportal.juniper.net/s/article/EX-QFX-How-to-calculate-and-to-optimize-TCAM-usage-in-firewall-filters 9/10
14/8/24, 15:33 [EX/QFX] How to calculate and to optimize TCAM usage in firewall filters
- Term 15: will expand to 3 terms: Name "bfd"
- Term 16: will expand to 3 terms: Name "bfd-back"
- Term 17: will expand to 2 terms: Name "dhcp"
- Term 18: will expand to 2 terms: Name "dhcp-back"
- Term 19: will expand to 9 terms: Name "traceroute"
- Term 20: will expand to 1 term : Name "default-term"
+ Term TCAM entry requirements:
- Term 1: needs 60 TCAM entries: Name "snmp"
- Term 2: needs 60 TCAM entries: Name "snmp-frags"
- Term 3: needs 60 TCAM entries: Name "http"
- Term 4: needs 60 TCAM entries: Name "ssh"
- Term 5: needs 4 TCAM entries: Name "icmp"
- Term 6: needs 44 TCAM entries: Name "ntp"
- Term 7: needs 44 TCAM entries: Name "ntp-back"
- Term 8: needs 12 TCAM entries: Name "dns"
- Term 9: needs 28 TCAM entries: Name "gre"
- Term 10: needs 28 TCAM entries: Name "bgp"
- Term 11: needs 28 TCAM entries: Name "bgp-back"
- Term 12: needs 4 TCAM entries: Name "rsvp"
- Term 13: needs 4 TCAM entries: Name "vrrp"
- Term 14: needs 4 TCAM entries: Name "ospf"
- Term 15: needs 12 TCAM entries: Name "bfd"
- Term 16: needs 12 TCAM entries: Name "bfd-back"
- Term 17: needs 8 TCAM entries: Name "dhcp"
- Term 18: needs 8 TCAM entries: Name "dhcp-back"
- Term 19: needs 36 TCAM entries: Name "traceroute"
- Term 20: needs 4 TCAM entries: Name "default-term"
+
Total TCAM entries available: 248 <<<<<<<<<<<<< Remaining Entries.
+ Total TCAM entries needed : 520

Related KB articles
For more TCAM information refer to the following KB articles:
KB28925 - TCAM filter space allocation and verification in QFX devices from Junos OS 12.2X50-D20 onward
-> Information on how memory slices are reserved in the TCAM when using PACLs, RACLs and VACLs

KB25927 - [QFX] How to check the TCAM Utilization on QFX-3500

-> Shows how to check TCAM utilization on a QFX3500

KB25106 - Calculate the TCAM utilization by loopback Firewall Filter on the QFX3500 switch
-> Caveats regarding the application of firewall filters to the loopback interface (i.e. they use more TCAM space)

KB30804 - QFX5100 failed to program firewall filters with multiple port range options.
-> Proper use of the source-port-range-optimize and destination-port-range-optimize options for QFX5100

Modification History

2019-05-07: QFX5200 information added.

AFFECTED PRODUCT SERIES / FEATURES

https://supportportal.juniper.net/s/article/EX-QFX-How-to-calculate-and-to-optimize-TCAM-usage-in-firewall-filters 10/10