User Guide VyOS Wiki
User Guide VyOS Wiki
User Guide
From VyOS Wiki
Linux vyatta 3.3.8-1-amd64-vyatta #1 SMP Wed Oct 30 22:54:40 CET 2013 x86_64
Welcome to VyOS.
This system is open-source software. The exact distribution terms for
each module comprising the full system are described in the individual
files in /usr/share/doc/*/copyright.
Last login: Wed Dec 11 17:33:09 2013
vyos@vyos:~$
Unlike general purpose Linux distributions, VyOS uses "image installation" that mimics the user experience of
traditional hardware routers and allows you to keep multiple VyOS versions on the same machine and switch to a
previous version if something breaks after upgrade. Every version is contained in its own squashfs image that is
mounted in a union filesystem together with a directory for mutable data (configs etc.).
Note: older versions used to support non-image installation ("install system" command). It's been deprecated since
the time image installation was introduced (long before the fork), and does not provide any version management
capabilities. You should not use it for new installations even if it's still available in new versions. You should not
worry about older systems installed that way though, they can be upgraded with "add system image".
https://wiki.vyos.net/wiki/User_Guide 2/45
5/11/2018 User Guide - VyOS Wiki
How big of a root partition should I create? (1000MB - 2147MB) [2147]MB: [return]
Which drive should GRUB modify the boot partition on? [sda]: [return]
Setting up grub: OK
Done!
vyos@vyos:~$
After the installation is complete, remove the Live CD and reboot the system:
vyos@vyos:~$ reboot
Proceed with reboot? (Yes/No) [No] Yes
vyos@vyos:~$
https://wiki.vyos.net/wiki/User_Guide 3/45
5/11/2018 User Guide - VyOS Wiki
Operational mode allows for commands to perform operational system tasks and view system and service status,
while configuration mode allows for the modification of system configuration. The command tree page lists
available commands and their functions.
The CLI provides a built-in help system. In the CLI the [?] key may be used to display available commands. The
[tab] key can be used to auto-complete commands and will present the help system upon a conflict or unknown
value.
For example typing sh followed by the [tab] key will complete to show. Pressing [tab] a second time will display
the possible sub-commands of the show command.
vyos@vyos:~$ s[tab]
set show
vyos@vyos:~$
When the output of a command results in more lines than can be displayed on the terminal screen the output is
paginated as indicated by a : prompt.
vyos@vyos:~$ configure
[edit]
https://wiki.vyos.net/wiki/User_Guide 4/45
5/11/2018 User Guide - VyOS Wiki
vyos@vyos:~#
Note that the prompt changes from $ to #. To exit configuration mode, type exit.
vyos@vyos:~# exit
exit
vyos@vyos:~$
See the configuration section of this document for more information on configuration mode.
vyos@vyos$ configure
vyos@vyos#
https://wiki.vyos.net/wiki/User_Guide 5/45
5/11/2018 User Guide - VyOS Wiki
set service dns forwarding name-server '8.8.8.8'
set service dns forwarding name-server '8.8.4.4'
vyos@vyos# commit
vyos@vyos# save
Saving configuration to '/config/config.boot'...
Done
vyos@vyos# exit
vyos@vyos$
Configuration Overview
VyOS makes use of a unified configuration file for all system configuration: config.boot. This allows for easy
template creation, backup, and replication of system configuration.
The current configuration can be viewed using the show configuration command.
https://wiki.vyos.net/wiki/User_Guide 6/45
5/11/2018 User Guide - VyOS Wiki
loopback lo {
}
}
service {
ssh {
port 22
}
}
system {
config-management {
commit-revisions 20
}
console {
device ttyS0 {
speed 9600
}
}
login {
user vyos {
authentication {
encrypted-password ****************
}
level admin
}
}
ntp {
server 0.pool.ntp.org {
}
server 1.pool.ntp.org {
}
server 2.pool.ntp.org {
}
}
syslog {
global {
facility all {
level notice
}
facility protocols {
level debug
}
}
}
}
vyos@vyos:~$
Because configuration changes are made using set and delete commands, the commands to generate the active
configuration can also be displayed using the show configuration commands command.
Configuration changes made do not take effect until committed using the commit command in configuration mode.
vyos@vyos# commit
[edit]
https://wiki.vyos.net/wiki/User_Guide 7/45
5/11/2018 User Guide - VyOS Wiki
vyos@vyos# exit
Warning: configuration changes have not been saved.
vyos@vyos:~$
In order to preserve configuration changes upon reboot, the configuration must also be saved once applied. This is
done using the save command in configuration mode.
vyos@vyos# save
Saving configuration to '/config/config.boot'...
Done
[edit]
vyos@vyos#
The show command within configuration mode will show the current configuration indicating line changes with a
+ for additions and a - for deletions.
vyos@vyos:~$ configure
[edit]
Configuration mode can not be exited while uncommitted changes exist. To exit configuration mode without
applying changes, the exit discard command can be used.
vyos@vyos# exit
Cannot exit: configuration modified.
Use 'exit discard' to discard the changes and exit.
[edit]
vyos@vyos# exit discard
exit
vyos@vyos:~$
VyOS also maintains backups of previous configurations. To compare configuration revisions in configuration
mode, use the compare command:
https://wiki.vyos.net/wiki/User_Guide 8/45
5/11/2018 User Guide - VyOS Wiki
Revisions:
0 2013-12-17 20:01:37 root by boot-config-loader
1 2013-12-13 15:59:31 root by boot-config-loader
2 2013-12-12 21:56:22 vyos by cli
3 2013-12-12 21:55:11 vyos by cli
4 2013-12-12 21:27:54 vyos by cli
5 2013-12-12 21:23:29 vyos by cli
6 2013-12-12 21:13:59 root by boot-config-loader
7 2013-12-12 16:25:19 vyos by cli
8 2013-12-12 15:44:36 vyos by cli
9 2013-12-12 15:42:07 root by boot-config-loader
10 2013-12-12 15:42:06 root by init
[edit]
vyos@vyos#
You can rollback configuration using the rollback command, however this command will currently trigger a
system reboot.
vyos@vyos# compare 1
[edit system]
>host-name vyos-1
[edit]
vyos@vyos# rollback 1
Proceed with reboot? [confirm][y]
VyOS also supports saving and loading configuration remotely using SCP, FTP, or TFTP.
https://wiki.vyos.net/wiki/User_Guide 9/45
5/11/2018 User Guide - VyOS Wiki
Network Interfaces
Configured interfaces on a VyOS system can be displayed using the show interfaces command.
A specific interface can be shown using the show interfaces <type> <name> command.
Different network interfaces provide type-specific configuration. Ethernet interfaces, for example, allow the
configuration of speed and duplex.
Many services, such as network routing, firewall, and traffic policy also maintain interface-specific configuration.
These will be covered in their respective sections.
Ethernet Interfaces
Ethernet interfaces allow for the configuration of speed, duplex, and hw-id (MAC address). Below is an example
configuration:
Resulting in:
ethernet eth1 {
address 192.168.0.1/24
https://wiki.vyos.net/wiki/User_Guide 10/45
5/11/2018 User Guide - VyOS Wiki
description INSIDE
duplex auto
hw-id 00:0c:29:44:3b:19
smp_affinity auto
speed auto
}
In addition, Ethernet interfaces provide the extended operational commands show interfaces ethernet <name>
physical and show interfaces ethernet <name> statistics. Statistics available are driver dependent.
802.1Q VLAN interfaces are represented as virtual sub-interfaces in VyOS. The term used for this is vif.
Configuration of a tagged sub-interface is accomplished using the configuration command set interfaces ethernet
<name> vif <vlan-id>.
Resulting:
ethernet eth1 {
address 192.168.0.1/24
description INSIDE
duplex auto
hw-id 00:0c:29:44:3b:19
smp_affinity auto
speed auto
https://wiki.vyos.net/wiki/User_Guide 11/45
5/11/2018 User Guide - VyOS Wiki
vif 100 {
address 192.168.100.1/24
description "VLAN 100"
}
}
Bridging
Interfaces in VyOS can be bridged together to provide software switching of Layer-2 traffic.
A bridge is created when a bridge interface is defined. In the example below we will be creating a bridge for
VLAN 100 and assigning a VIF to the bridge.
Interfaces assigned to a bridge-group do not have address configuration. An IP address can be assigned to the
bridge interface itself, however, like any normal interface.
Example Result:
bridge br100 {
address 192.168.100.1/24
}
[...]
ethernet eth1 {
[...]
vif 100 {
bridge-group {
bridge br100
}
}
}
In addition to normal IP interface configuration, bridge interfaces support Spanning-Tree Protocol. STP is disabled
by default.
NOTE: Please use caution when introducing spanning-tree protocol on a network as it may result in topology
changes.
To enable spanning-tree use the set interfaces bridge <name> stp true command:
https://wiki.vyos.net/wiki/User_Guide 12/45
5/11/2018 User Guide - VyOS Wiki
STP priority, forwarding-delay, hello-time, and max-age can be configured for the bridge-group. The MAC
aging time can also be configured using the aging directive.
For member interfaces, the bridge-group priority and cost can be configured.
The show bridge operational command can be used to display configured bridges:
If spanning-tree is enabled, the show bridge <name> spanning-tree command can be used to show STP
configuration:
eth1.100 (1)
port id 8001 state forwarding
designated root 0000.000c29443b19 path cost 4
designated bridge 0000.000c29443b19 message age timer 0.00
designated port 8001 forward delay timer 0.00
designated cost 0 hold timer 0.00
flags
The MAC address-table for a bridge can be displayed using the show bridge <name> macs command:
Bonding
You can combine (aggregate) 2 or more physical interfaces into a single logical one. It's called bonding, or LAG, or
etherchannel, or portchannel.
https://wiki.vyos.net/wiki/User_Guide 13/45
5/11/2018 User Guide - VyOS Wiki
For example:
You may want to set IEEE 802.3ad Dynamic link aggregation (802.3ad) AKA LACP (don't forget to setup it on the
other end of these links):
After a commit you may treat bond0 as almost a physical interface (you can't change its` duplex, for example) and
assign IPs or VIFs on it.
Tunnel Interfaces
https://wiki.vyos.net/wiki/User_Guide 14/45
5/11/2018 User Guide - VyOS Wiki
Routing
VyOS is a "router first" network operating system. It supports static routing, policy routing, and dynamic routing
using standard protocols (RIP, OSPF, and BGP).
Static
A typical use for a static route is a static default route for systems that do not make use of DHCP or dynamic
routing protocols:
static {
route 0.0.0.0/0 {
next-hop 203.0.113.1 {
distance 1
}
}
}
Another common use of static routes is to blackhole (drop) traffic. In the example below, RFC 1918 private IP
networks are set as blackhole routes. This does not prevent networks within these segments from being used, since
the most specific route is always used. It does, however, prevent traffic to unknown private networks from leaving
the router. Commonly refereed to as leaking.
Resulting configuration:
route 10.0.0.0/8 {
blackhole {
distance 254
}
}
route 172.16.0.0/12 {
blackhole {
distance 254
}
}
route 192.168.0.0/16 {
https://wiki.vyos.net/wiki/User_Guide 15/45
5/11/2018 User Guide - VyOS Wiki
blackhole {
distance 254
}
}
Note that routes with a distance of 255 are effectively disabled and not installed into the kernel.
RIP
Node 1:
Node 2:
Resulting configuration:
Node 1:
rip {
network 192.168.0.0/24
redistribute {
connected {
}
}
}
Node 2:
rip {
network 192.168.0.0/24
redistribute {
connected {
}
}
}
OSPF
IPv4
A typical configuration using 2 nodes, redistribute loopback address and the node 1 sending the default route:
Node 1:
https://wiki.vyos.net/wiki/User_Guide 16/45
5/11/2018 User Guide - VyOS Wiki
Node 2:
Resulting configuration:
Node 1:
ospf {
area 0 {
network 192.168.0.0/24
}
default-information {
originate {
always
metric 10
metric-type 2
}
}
log-adjacency-changes {
}
parameters {
router-id 1.1.1.1
}
redistribute {
connected {
metric-type 2
route-map CONNECT
}
}
}
route-map CONNECT {
rule 10 {
action permit
match {
interface lo
}
https://wiki.vyos.net/wiki/User_Guide 17/45
5/11/2018 User Guide - VyOS Wiki
}
}
Node 2:
ospf {
area 0 {
network 192.168.0.0/24
}
log-adjacency-changes {
}
parameters {
router-id 2.2.2.2
}
redistribute {
connected {
metric-type 2
route-map CONNECT
}
}
}
route-map CONNECT {
rule 10 {
action permit
match {
interface lo
}
}
}
IPv6
Node 1:
Node 2:
Resulting configuration:
Node 1:
protocols {
ospfv3 {
area 0.0.0.0 {
interface eth1
range 2001:db8:1::/64 {
https://wiki.vyos.net/wiki/User_Guide 18/45
5/11/2018 User Guide - VyOS Wiki
}
}
parameters {
router-id 192.168.1.1
}
redistribute {
connected {
}
}
}
}
Node 2:
protocols {
ospfv3 {
area 0.0.0.0 {
interface eth1
range 2001:db8:2::/64 {
}
}
parameters {
router-id 192.168.2.1
}
redistribute {
connected {
}
}
}
}
BGP
IPv4
Node 1:
Node 2:
Don’t forget, the CIDR declared in the network statement MUST exist in your routing table (dynamic or static),
the best way to make sure that is true is creating a static route:
https://wiki.vyos.net/wiki/User_Guide 19/45
5/11/2018 User Guide - VyOS Wiki
Node 1:
Node 2:
Resulting configuration:
Node 1:
bgp 65536 {
neighbor 192.168.0.2 {
ebgp-multihop 2
remote-as 65537
update-source 192.168.0.1
}
network 1.0.0.0/16 {
}
parameters {
router-id 192.168.0.1
}
}
static {
route 1.0.0.0/16 {
blackhole {
distance 254
}
}
}
Node 2:
bgp 65537 {
neighbor 192.168.0.1 {
ebgp-multihop 2
remote-as 65536
update-source 192.168.0.2
}
network 2.0.0.0/16 {
}
parameters {
router-id 192.168.0.2
}
}
static {
route 2.0.0.0/16 {
blackhole {
distance 254
}
}
}
https://wiki.vyos.net/wiki/User_Guide 20/45
5/11/2018 User Guide - VyOS Wiki
IPv6
Node 1:
Node 2:
Don’t forget, the CIDR declared in the network statement MUST exist in your routing table (dynamic or static), the
best way to make sure that is true is creating a static route:
Node 1:
Node 2:
Resulting configuration:
Node 1:
bgp 65536 {
neighbor 2a00:100::2 {
address-family {
ipv6-unicast {
}
}
ebgp-multihop 2
remote-as 65537
update-source 2a00:100::1
}
network 2a00:100:1::/48 {
}
parameters {
router-id 1.1.1.1
}
}
static {
route6 2a00:100:1::/48 {
https://wiki.vyos.net/wiki/User_Guide 21/45
5/11/2018 User Guide - VyOS Wiki
blackhole {
distance 254
}
}
}
Node 2:
bgp 65537 {
neighbor 2a00:100::1 {
address-family {
ipv6-unicast {
}
}
ebgp-multihop 2
remote-as 65536
update-source 2a00:100::2
}
network 2a00:100:2::/48 {
}
parameters {
router-id 1.1.1.2
}
}
static {
route6 2a00:100:2::/48 {
blackhole {
distance 254
}
}
}
Route Filtering
Node1:
Node2:
https://wiki.vyos.net/wiki/User_Guide 22/45
5/11/2018 User Guide - VyOS Wiki
set policy prefix-list AS65536-OUT rule 10 prefix '1.0.0.0/16'
set policy prefix-list6 AS65536-IN rule 10 action 'permit'
set policy prefix-list6 AS65536-IN rule 10 prefix '2a00:100:1::/48'
set policy prefix-list6 AS65536-OUT rule 10 action 'deny'
set policy prefix-list6 AS65536-OUT rule 10 prefix '2a00:100:1::/48'
set policy route-map AS65536-IN rule 10 action 'permit'
set policy route-map AS65536-IN rule 10 match ip address prefix-list 'AS65536-IN'
set policy route-map AS65536-IN rule 10 match ipv6 address prefix-list 'AS65536-IN'
set policy route-map AS65536-IN rule 20 action 'deny'
set policy route-map AS65536-OUT rule 10 action 'deny'
set policy route-map AS65536-OUT rule 10 match ip address prefix-list 'AS65536-OUT'
set policy route-map AS65536-OUT rule 10 match ipv6 address prefix-list 'AS65536-OUT'
set policy route-map AS65536-OUT rule 20 action 'permit'
set protocols bgp 65537 neighbor 2a00:100::1 route-map export 'AS65536-OUT'
set protocols bgp 65537 neighbor 2a00:100::1 route-map import 'AS65536-IN'
We could expand on this and also deny link local and multicast in the rule 20 action deny.
Policy Routing
VyOS supports Policy Routing, allowing traffic to be assigned to a different routing table. Traffic can be matched
using standard 5-tuple matching (source address, destination address, protocol, source port, destination port).
The following example will show how VyOS can be used to redirect web traffic to an external transparent proxy:
This creates a route policy called FILTER-WEB with one rule to set the routing table for matching traffic (TCP port
80) to table ID 100 instead of the default routing table.
To create routing table 100 and add a new default gateway to be used by traffic matching our route policy:
This can be confirmed using the show ip route table 100 operational command.
Finally, to apply the policy route to ingress traffic on our LAN interface, we use:
The route policy functionality in VyOS can also be used to rewrite TCP MSS using the set policy route <name>
rule <rule> set tcp-mss <value> directive, modify DSCP value using [...] set dscp <value>, or mark the
traffic with an internal ID using [...] set mark <value> for further processing (e.g. QOS) on a per-rule basis for
matching traffic.
In addition to 5-tuple matching, additional options such as time-based rules, are available. See the built-in help for
a complete list of options.
Firewall
VyOS makes use of Linux netfilter (http://netfilter.org/) for packet filtering.
https://wiki.vyos.net/wiki/User_Guide 23/45
5/11/2018 User Guide - VyOS Wiki
The firewall supports the creation of groups for ports, addresses, and networks (implemented using netfilter ipset)
and the option of interface or zone based firewall policy.
Important note on usage of terms: The firewall makes use of the terms in, out, and local for firewall policy.
Users experienced with netfilter often confuse in to be a reference to the INPUT chain, and out the OUTPUT chain
from netfilter. This is not the case. These instead indicate the use of the FORWARD chain and either the input or output
interface. The INPUT chain, which is used for local traffic to the OS, is a reference to as local with respect to its
input interface.
As an alternative to applying policy to an interface directly, a zone-based firewall can be created to simplify
configuration when multiple interfaces belong to the same security zone. Instead of applying to rulesets to
interfaces they are applied to source zone-destination zone pairs.
An introduction can to zone-based firewalls can be found here. For an example see Zone-policy example.
Groups
Firewall groups represent collections of IP addresses, networks, or ports. Once created, a group can be referenced
by firewall rules as either a source or destination. Members can be added or removed from a group without
changes to or the need to reload individual firewall rules. Note that groups can also be referenced by NAT
configuration.
While network groups accept IP networks in CIDR notation, specific IP addresses can be added as a 32-bit prefix.
If you foresee the need to add a mix of addresses and networks, the network group is recommended.
Here is an example of a network group for the IP networks that make up the internal network:
A port group represents only port numbers, not the protocol. Port groups can be referenced for either TCP or UDP.
It is recommended that TCP and UDP groups are created separately to avoid accidentally filtering unnecessary
ports. Ranges of ports can be specified by using a -.
Rule-Sets
A rule-set is a named collection of firewall rules that can be applied to an interface or zone. Each rule is numbered,
has an action to apply if the rule is matched, and the ability to specify the criteria to match.
https://wiki.vyos.net/wiki/User_Guide 24/45
5/11/2018 User Guide - VyOS Wiki
set firewall name INSIDE-OUT rule 1010 state established enable
set firewall name INSIDE-OUT rule 1010 state related enable
set firewall name INSIDE-OUT rule 1020 action drop
set firewall name INSIDE-OUT rule 1020 state invalid enable
Once a rule-set is created, it can be applied to an interface. Note only one rule-set can be applied to each interface
for in, out, or local traffic for each protocol (IPv4 and IPv6).
A named rule-set can also be applied to a zone relationship (note, zones must first be created):
firewall {
all-ping enable
broadcast-ping disable
config-trap disable
group {
network-group BAD-NETWORKS {
network 1.2.3.0/24
network 1.2.4.0/24
}
network-group GOOD-NETWORKS {
network 4.5.6.0/24
network 4.5.7.0/24
}
port-group BAD-PORTS {
port 65535
}
}
name FROM-INTERNET {
default-action accept
description "From the Internet"
rule 10 {
action accept
description "Authorized Networks"
protocol all
source {
group {
network-group GOOD-NETWORKS
}
}
}
rule 11 {
action drop
description "Bad Networks"
protocol all
source {
group {
network-group BAD-NETWORKS
}
}
}
rule 30 {
action drop
description "BAD PORTS"
https://wiki.vyos.net/wiki/User_Guide 25/45
5/11/2018 User Guide - VyOS Wiki
destination {
group {
port-group BAD-PORTS
}
}
log enable
protocol all
}
}
}
interfaces {
ethernet eth1 {
address dhcp
description OUTSIDE
duplex auto
firewall {
in {
name FROM-INTERNET
}
}
}
}
NAT
Source NAT
Source NAT is typically referred to simply as NAT. To be more correct, what most people refer to as NAT is
actually the process of Port Address Translation (PAT), or NAT Overload: The process of having many internal
host systems communicate to the Internet using a single or subset of IP addresses.
In the example used for the Quick Start configuration above, we demonstrate the following configuration:
rule 100 {
outbound-interface eth0
source {
address 192.168.0.0/24
}
translation {
address masquerade
}
}
https://wiki.vyos.net/wiki/User_Guide 26/45
5/11/2018 User Guide - VyOS Wiki
In this example, we use masquerade as the translation address instead of an IP address. The masquerade target is
effectively an alias to say "use whatever IP address is on the outgoing interface", rather than a statically configured
IP address. This is useful if you use DHCP for your outgoing interface and do not know what the external address
will be.
When using NAT for a large number of host systems it recommended that a minimum of 1 IP address is used to
NAT every 256 host systems. This is due to the limit of 65,000 port numbers available for unique translations and a
reserving an average of 200-300 sessions per host system.
Example: For an ~ 8,000 host network a source NAT pool of 32 IP addresses is recommended.
A pool of addresses can be defined by using a - in the set nat source rule [n] translation address statement.
Linux netfilter will not NAT traffic marked as INVALID. This often confuses people into thinking that Linux (or
specifically VyOS) has a broken NAT implementation because non-NATed traffic is seen leaving an external
interface. This is actually working as intended, and a packet capture of the "leaky" traffic should reveal that the
traffic is either an additional TCP "RST", "FIN,ACK", or "RST,ACK" sent by client systems after Linux netfilter
considers the connection closed. The most common is the additional TCP RST some host implementations send
after terminating a connection (which is implementation-specific).
In other words, connection tracking has already observed the connection be closed and has transition the flow to
INVALID to prevent attacks from attempting to reuse the connection.
You can avoid the "leaky" behavior by using a firewall policy that drops "invalid" state packets.
Having control over the matching of INVALID state traffic, e.g. the ability to selectively log, is an important
troubleshooting tool for observing broken protocol behavior. For this reason, VyOS does not globally drop invalid
state traffic, instead allowing the operator to make the determination on how the traffic is handled.
A typical problem with using NAT and hosting public servers is the ability for internal systems to reach an internal
server using it's external IP address. The solution to this is usually the use of split-DNS to correctly point host
systems to the internal address when requests are made internally. Because many smaller networks lack DNS
infrastructure, a work-around is commonly deployed to facilitate the traffic by NATing the request from internal
hosts to the source address of the internal interface on the firewall. This technique is commonly reffered to as NAT
Reflection, or Hairpin NAT.
In this example, we will be using the example Quick Start configuration above as a starting point.
To setup a NAT reflection rule, we need to create a rule to NAT connections from the internal network to the same
internal network to use the source address of the internal interface.
https://wiki.vyos.net/wiki/User_Guide 27/45
5/11/2018 User Guide - VyOS Wiki
rule 110 {
description "NAT Reflection: INSIDE"
destination {
address 192.168.0.0/24
}
outbound-interface eth1
source {
address 192.168.0.0/24
}
translation {
address masquerade
}
}
Destination NAT
DNAT is typically referred to as a Port Forward. When using VyOS as a NAT router and firewall, a common
configuration task is to redirect incoming traffic to a system behind the firewall.
In this example, we will be using the example Quick Start configuration above as a starting point.
In our example, we will be forwarding web server traffic to an internal web server on 192.168.0.100.
nat {
destination {
rule 10 {
description "Port Forward: HTTP to 192.168.0.100"
destination {
port 80
https://wiki.vyos.net/wiki/User_Guide 28/45
5/11/2018 User Guide - VyOS Wiki
}
inbound-interface eth0
protocol tcp
translation {
address 192.168.0.100
}
}
}
}
Note that if forwarding traffic to a different port than it is arriving on, you may also configure the translation port
using set nat destination rule [n] translation port.
This establishes our Port Forward rule, but if we created a firewall policy it will likely block the traffic.
It is important to note that when creating firewall rules that the DNAT translation occurs before traffic traverses the
firewall. In other words, the destination address has already been translated to 192.168.0.100.
So in our firewall policy, we want to allow traffic coming in on the outside interface, destined for TCP port 80 and
the IP address of 192.168.0.100.
rule 20 {
action accept
destination {
address 192.168.0.100
port 80
}
protocol tcp
state {
new enable
}
}
Note: If you have configured the INSIDE-OUT policy, you will need to add additional rules to permit inbound
NAT traffic.
1-to-1 NAT
For a 1-to-1 NAT configuration, both DNAT and SNAT are used to NAT all traffic from an external IP address to
an internal IP address and vice-versa.
Typically, a 1-to-1 NAT rule omits the destination port (all ports) and replaces the protocol with either all or ip.
Then a corresponding SNAT rule is created to NAT outgoing traffic for the internal IP to a reserved external IP.
This dedicates an external IP address to an internal IP address and is useful for protocols which don't have the
notion of ports, such as GRE.
https://wiki.vyos.net/wiki/User_Guide 29/45
5/11/2018 User Guide - VyOS Wiki
Here's an extract of a simple 1-to-1 NAT configuration with one internal and one external interface:
Firewall rules are written as normal, using the internal IP address as the source of outbound rules and the
destination of inbound rules.
NPTv6 (RFC6296)
VPN
OpenVPN
Traditionally hardware routers implement IPsec exclusively due to relative ease of implementing it in hardware
and insufficient CPU power for doing encryption in software. Since VyOS is a software router, this is less of a
concern. OpenVPN has been widely used on UNIX platform for a long time and is a popular option for remote
access VPN, though it's also capable of site-to-site connections.
It uses a single TCP or UDP connection and does not rely on packet source addresses, so it will work even
through a double NAT: perfect for public hotspots and such
It's easy to setup and offers very flexible split tunneling
There's a variety of client GUI frontends for any platform
It's slower than IPsec due to higher protocol overhead and the fact it runs in user mode while IPsec, on
Linux, is in kernel mode
None of the operating systems have client software installed by default
In the VyOS CLI, a key point often overlooked is that rather than being configured using the set vpn stanza,
OpenVPN is configured as a network interface using set interfaces openvpn.
OpenVPN Site-To-Site
While many are aware of OpenVPN as a Client VPN solution, it is often overlooked as a site-to-site VPN solution
due to lack of support for this mode in many router platforms.
https://wiki.vyos.net/wiki/User_Guide 30/45
5/11/2018 User Guide - VyOS Wiki
Site-to-site mode supports x.509 but doesn't require it and can also work with static keys, which is simpler in many
cases. In this example, we'll configure a simple site-to-site OpenVPN tunnel using a 2048-bit pre-shared key.
First, one one of the systems generate the key using the operational command generate openvpn key <filename>.
This will generate a key with the name provided in the /config/auth/ directory. Once generated, you will need to
copy this key to the remote router.
In our example, we used the filename openvpn-1.key which we will reference in our configuration.
The public IP address of the local side of the VPN will be 198.51.100.10
The remote will be 203.0.113.11
The tunnel will use 10.255.1.1 for the local IP and 10.255.1.2 for the remote.
OpenVPN allows for either TCP or UDP. UDP will provide the lowest latency, while TCP will work better
for lossy connections; generally UDP is preferred when possible.
The official port for OpenVPN is 1194, which we reserve for client VPN; we will use 1195 for site-to-site
VPN.
The persistent-tunnel directive will allow us to configure tunnel-related attributes, such as firewall policy
as we would on any normal network interface.
If known, the IP of the remote router can be configured using the remote-host directive; if unknown, it can
be omitted. We will assume a dynamic IP for our remote router.
Local Configuration:
Remote Configuration:
The configurations above will default to using 128-bit Blowfish in CBC mode for encryption and SHA-1 for
HMAC authentication. These are both considered weak, but a number of other encryption and hashing algorithms
are available:
For Encryption:
https://wiki.vyos.net/wiki/User_Guide 31/45
5/11/2018 User Guide - VyOS Wiki
aes192 AES algorithm with 192-bit key
aes256 AES algorithm with 256-bit key
For Hashing:
If you change the default encryption and hashing algorithms, be sure that the local and remote ends have matching
configurations, otherwise the tunnel will not come up.
Static routes can be configured referencing the tunnel interface; for example, the local router will use a network of
10.0.0.0/16, while the remote has a network of 10.1.0.0/16:
Local Configuration:
Remote Configuration:
Firewall policy can also be applied to the tunnel interface for local, in, and out directions and function identically
to ethernet interfaces.
If making use of multiple tunnels, OpenVPN must have a way to distinguish between different tunnels aside from
the pre-shared-key. This is either by referencing IP address or port number. One option is to dedicate a public IP to
each tunnel. Another option is to dedicate a port number to each tunnel (e.g. 1195,1196,1197...)
OpenVPN status can be verified using the show openvpn operational commands. See the built-in help for a
complete list of options.
OpenVPN Server
Multi-client server is the most popular OpenVPN mode on routers. It always x.509 authentication and therefore
requires a PKI setup. This guide assumes you have already setup a PKI and have a CA certificate, a server
certificate and key, a certificate revokation list, a Diffie-Hellman key exchange parameters file. You do not need
client certificates and keys for the server setup.
In this example we will use the most complicated case: a setup where each client is a router that has its own subnet
(think HQ and branch offices), since simpler setups are subsets of it.
Suppose you want to use 10.23.1.0/24 network for client tunnel endpoints and all client subnets belong to
10.23.0.0/20. All clients need access to the 192.168.0.0/16 network.
First we need to specify the basic settings. 1194/UDP is the default. The persistent-tunnel option is recommended,
it prevents the TUN/TAP device from closing on connection resets or daemon reloads.
https://wiki.vyos.net/wiki/User_Guide 32/45
5/11/2018 User Guide - VyOS Wiki
Then we need to specify the location of the cryptographic materials. Suppose you keep the files in
/config/auth/openvpn
Now we need to specify the server network settings. In all cases we need to specify the subnet for client tunnel
endpoints. Since we want clients to access a specific network behind out router, we will use a push-route option for
installing that route on clients.
Since it's a HQ and branch offices setup, we will want all clients to have fixed addresses and we will route traffic to
specific subnets through them. We need configuration for each client to achieve this. Note that clients are identified
by the CN field of their x.509 certificates, in this example the CN is client0:
OpenVPN will not automatically create routes in the kernel for client subnets when they connect and will only use
client-subnet association internally, so we need to create a route to the 10.23.0.0/20 network ourselves:
Example for configuring a simple L2TP over IPsec VPN for remote access (works with native Windows and Mac
VPN clients):
https://wiki.vyos.net/wiki/User_Guide 33/45
5/11/2018 User Guide - VyOS Wiki
If a local firewall policy is in place on your external interface you will need to open:
In addition when NAT is detected by the VPN client ESP is encapsulated in UDP for NAT-traversal:
Example:
Also note that if you wish to allow the VPN to be used for external access you will need to add the appropriate
source NAT rules to your configuration.
To be able to resolve when connected to the VPN, the following DNS rules are needed as well.
Established sessions can be viewed using the show vpn remote-access operational command.
Site-to-Site IPsec
Example:
https://wiki.vyos.net/wiki/User_Guide 35/45
5/11/2018 User Guide - VyOS Wiki
# server side
set nat source rule 10 destination address '10.0.0.0/24'
set nat source rule 10 'exclude'
set nat source rule 10 outbound-interface 'eth1'
set nat source rule 10 source address '192.168.0.0/24'
To allow traffic to pass through to clients, you need to add the following rules. (if you used the default
configuration at the top of this page)
# server side
set firewall name OUTSIDE-LOCAL rule 32 action 'accept'
set firewall name OUTSIDE-LOCAL rule 32 source address '10.0.0.0/24'
DMVPN
One common use of traffic policy is to limit bandwidth for an interface. In the example below we limit bandwidth
for our LAN connection to 200M download and out WAN connection to 50M upload:
traffic-policy {
shaper WAN-OUT {
bandwidth 50Mbit
default {
bandwidth 50%
ceiling 100%
queue-type fair-queue
}
}
https://wiki.vyos.net/wiki/User_Guide 36/45
5/11/2018 User Guide - VyOS Wiki
shaper LAN-OUT {
bandwidth 200Mbit
default {
bandwidth 50%
ceiling 100%
queue-type fair-queue
}
}
}
Once defined, a traffic policy can be applied to each interface using the interface-level traffic-policy directive:
Note that a traffic policy can also be defined to match specific traffic flows using class statements.
DHCP Server
For more information, refer to the DHCP server page.
VyOS provides a DHCP server. The following example provides open pool addressing for a LAN network:
This will provide dynamic addressing using a 24-hour lease and the range of 128-254 for host addressing,
reserving the first half of the network for static assignment. Note that the DHCP service will NOT be authoritative
for the subnet unless explicitly configured.
Static mappings can also be configured for hosts which need a fixed address via DHCP:
https://wiki.vyos.net/wiki/User_Guide 37/45
5/11/2018 User Guide - VyOS Wiki
set service dhcp-server shared-network-name 'LAN' subnet '192.168.0.0/24' static-mapping 'SERVER' ip-address '192.168.0.10'
set service dhcp-server shared-network-name 'LAN' subnet '192.168.0.0/24' static-mapping 'SERVER' mac-address '00:53:00:00:00:
Failover
set service dhcp-server shared-network-name 'LAN' subnet '192.168.0.0/24' failover local-address '192.168.0.1'
set service dhcp-server shared-network-name 'LAN' subnet '192.168.0.0/24' failover name 'foo'
set service dhcp-server shared-network-name 'LAN' subnet '192.168.0.0/24' failover peer-address '192.168.0.2'
The primary and secondary statements determines whether the server is primary or secondary
set service dhcp-server shared-network-name 'LAN' subnet '192.168.0.0/24' failover status 'primary'
or
set service dhcp-server shared-network-name 'LAN' subnet '192.168.0.0/24' failover status 'secondary'
DHCPv6 server
DHCPv6 server shares most of the configuration options with IPv4 DHCP, with one major exception: in IPv6,
DHCP is never used for default gateway configuration. If default gateway is configured automatically, it is always
done via router advertisment (RA).
Since router advertisment cannot be deactivated even if DHCPv6 is in use, first you need to set the "managed flag"
in RA, which instructs clients to request an IPv6 address via DHCPv6 instead of using SLAAC or EUI-64.
set service dhcpv6-server shared-network-name LAN subnet 2001:db8:ff::/64 address-range prefix '2001:db8:ff::/64'
set service dhcpv6-server shared-network-name LAN subnet 2001:db8:ff::/64 address-range start 2001:db8::1000 stop '2001:db8::f
set service dhcpv6-server shared-network-name LAN subnet 2001:db8:ff::/64 name-server '2001:db8:aa::bb'
DHCP-Relay
VyOS can act as a DHCP relay, sending DHCP requests from locally connected networks to a central DHCP
server, or servers:
https://wiki.vyos.net/wiki/User_Guide 38/45
5/11/2018 User Guide - VyOS Wiki
set service dhcp-relay server 192.168.0.4
set service dhcp-relay server 192.168.0.5
An important note that dhcp-relay currently requires specifying the interface the DHCP server replies come from in
addition to interfaces to listen on. An upstream bug exists for this: https://bugs.debian.org/cgi-bin/bugreport.cgi?
bug=648401
By default the ISC dhcp-relay process will relay relay packets; as this is not expected behavior, disabling this
functionality is recommended using the following directive:
Additional protections can be configured matching hop-limit and max-size for DHCP packets.
DNS Forwarder
VyOS provides a DNS forwarder. This example shows the forwarder being used to listen for DNS request on
interface eth1 and use Google Public DNS for name servers with caching disabled:
Alternatively, DNS servers can be specified to be the system DNS servers using the set service dns forwarding
system directive, or to those obtained via DHCP using the [...] dhcp <interface> directive.
By default, the forwarder will also make use of the local hosts on the system; this can be disable using the ignore-
host-file directive.
To add entries to the local hosts file for the system, the set system static-host-mapping directive can be used:
Dynamic DNS
VyOS provides a dynamic DNS update agent which can be used to update DNS providers when dynamic
addressing is used.
mDNS Repeater
https://wiki.vyos.net/wiki/User_Guide 39/45
5/11/2018 User Guide - VyOS Wiki
Multicast DNS uses the 224.0.0.51 address, which is "administratively scoped" and does not leave the subnet. It re-
broadcast mDNS packets from one interface to other interfaces. This enables support for e.g. Airplay devices
across multiple VLANs.
Will re-broadcast all mDNS packets from eth0 to eth1 and vice versa.
mDNS repeater can be disabled using set service mdns repeater disable
Every UDP port which will be forward requires one unique ID. Currently we support 99 IDs!
Each broadcast relay instance can be disabled using set service broadcast-relay id <n> disable.
In addition you can disable the whole service without removing the configuration by set service broadcast-relay
disable.
System Configuration
System Users
As of 1.1.8 release and 1.2.0 release candidates, VyOS support two user privilege levels: admin and operator.
However, the operator level is a legacy feature and was proved insecure: any user can escape the restricted shell in
multiple easy way. That feature will be removed in future releases, and we do not recommend using it.
https://wiki.vyos.net/wiki/User_Guide 40/45
5/11/2018 User Guide - VyOS Wiki
Create user account jsmith, with admin level access and the password "mypassword"
The command:
user jsmith {
authentication {
encrypted-password $6$0OQHjuQ8M$AYXVn7jufdfqPrSk4/XXsDBw99JBtNsETkQKDgVLptXogHA2bU9BWlvViOFPBoFxIi.iqjqrvsQdQ./cfiiPT.
plaintext-password ""
}
full-name "Johan Smith"
level admin
}
The following command will load the public key dev.pub for user jsmith
Note: this requires uploading the dev.pub public key to the vyos firewall first. As an alternative you can also load
the ssh public key directly from a remote system:
IPv6
Advanced IPv6 configuration examples are available on the IPv6 page.
Clustering
VyOS supports multicast and unicast clustering. Multicast is default and to use the unicast method you can add the
peer directive to the interface with the ip of the other cluster member.
In the example below SSH is clustered between two nodes with the unicast method.
https://wiki.vyos.net/wiki/User_Guide 41/45
5/11/2018 User Guide - VyOS Wiki
cluster {
dead-interval 20000
group cluster {
auto-failback false
primary vyos
secondary vyos2
service ssh
service 192.168.0.123/24/eth0
}
interface eth0 {
peer 192.168.0.121
}
keepalive-interval 5000
monitor-dead-interval 20000
pre-shared-secret S3cr#t
}
/
/boot
/boot/grub
/boot/VyOS_999.hydrogen.11291501
The image directory contains the system kernel, a compressed image of the root filesystem for the OS, and a
directory for persistent storage, such as configuration.
On boot, the system will extract the OS image into memory and mount the appropriate live-rw sub-directories to
provide persistent storage system configuration.
This process allows for a system to always boot to a known working state, as the OS image is fixed and non-
persistent. It also allows for multiple releases of VyOS to be installed on the same storage device.
The image can be selected manually at boot if needed, but the system will otherwise boot the image configured to
be the default.
The default boot image can be set using the set system image default-boot command in operational mode.
A list of available images can be shown using the show system image command in operational mode.
vyos@vyos:~$
Images no longer needed can be removed using the delete system image command.
https://wiki.vyos.net/wiki/User_Guide 42/45
5/11/2018 User Guide - VyOS Wiki
Finally, new system images can be added using the add system image command. The add image command will
extract the image from the release ISO (either on the local filesystem or remotely if a URL is provided). The image
install process will prompt you to use the current system configuration and SSH security keys, allowing for the
new image to boot using the current configuration.
Note that configuration is associated to each image, and each image has a unique copy of its configuration. This is
different than a traditional network router where the configuration is shared across all images.
If you need some files from a previous images - take a look inside a /live directory.
Troubleshooting
Sometimes things break or don't work as expected. This section describes several troubleshooting tools provided
by VyOS that can help when something goes wrong.
Verifying connectivity can be done with the familiar ping and traceroute commands. The options for each are
shown (the options for each command were displayed using the built-in help as described in the Using the
Command-Line Interface section and are omitted from the output here):
vyos@vyos:~$ ping
Possible completions:
<hostname> Send Internet Control Message Protocol (ICMP) echo request
<x.x.x.x>
<h:h:h:h:h:h:h:h>
vyos@vyos:~$
vyos@vyos:~$ traceroute
Possible completions:
<hostname> Track network path to specified node
<x.x.x.x>
<h:h:h:h:h:h:h:h>
https://wiki.vyos.net/wiki/User_Guide 43/45
5/11/2018 User Guide - VyOS Wiki
ipv4 Track network path to <hostname|IPv4 address>
ipv6 Track network path to <hostname|IPv6 address>
vyos@vyos:~$
However, another tool, mtr, is available which combines ping and traceroute into a single tool. An example of its
output is shown:
The output of mtr consumes the screen and will replace your command prompt:
My traceroute [v0.75]
vyos (0.0.0.0) Fri Feb 24 14:28:08 2017
Keys: Help Display mode Restart statistics Order of fields quit
Packets Pings
Host Loss% Snt Last Avg Best Wrst StDev
1. 10.11.110.4 0.0% 34 0.5 0.5 0.4 0.8 0.1
2. 10.62.255.184 0.0% 34 1.1 1.0 0.9 1.4 0.1
3. 10.62.255.71 0.0% 34 1.4 1.4 1.3 2.0 0.1
4. 10.62.212.12 0.0% 34 1.6 1.6 1.6 1.7 0.0
Several options are available for changing the display output. Press h to invoke the built in help system. To quit,
just press q and you'll be returned to the VyOS command prompt.
It's possible to monitor network traffic, either at the flow level or protocol level. This can be useful when
troubleshooting a variety of protocols and configurations. The following interface types can be monitored:
To monitor traffic flows, issue the monitor interfaces <type> <name> flow command, replacing <type> and
<name> with your desired interface type and name, respectively. Output looks like the following:
https://wiki.vyos.net/wiki/User_Guide 44/45
5/11/2018 User Guide - VyOS Wiki
<= 680b 136b 136b
????????????????????????????????????????????????????????????????????????????????????????????????????
TX: cumm: 26.7KB peak: 40.6Kb rates: 23.2Kb 21.4Kb 21.4Kb
RX: 67.5KB 63.6Kb 54.6Kb 54.0Kb 54.0Kb
TOTAL: 94.2KB 104Kb 77.8Kb 75.4Kb 75.4Kb
Several options are available for changing the display output. Press h to invoke the built in help system. To quit,
just press q and you'll be returned to the VyOS command prompt.
To monitor interface traffic, issue the monitor interfaces <type> <name> traffic command, replacing <type> and
<name> with your desired interface type and name, respectively. This command invokes the familiar tshark utility
and the following options are available:
To quit monitoring, press Ctrl-c and you'll be returned to the VyOS command prompt. The detail keyword
provides verbose output of the traffic seen on the monitored interface. The filter keyword accepts valid PCAP filter
expressions (http://www.tcpdump.org/manpages/pcap-filter.7.html), enclosed in single or double quotes (e.g. 'port
25' or "port 161 and udp"). The save keyword allows you to save the traffic dump to a file. The unlimited keyword
is used to specify that an unlimited number of packets can be captured (by default, 1,000 packets are captured and
you're returned to the VyOS command prompt).
https://wiki.vyos.net/wiki/User_Guide 45/45