Open Shortest Path First (OSPF) is a routing protocol for Internet Protocol (IP) networks. Because I have site-to-site OpenVPN and IPSEC tunnels between my primary and remote sites I ended up defining lots of static routes. The environment started to be complex enough to justify the use of OSPF that is supported by EdgeRouter.
The current setup of my environment looks like:
Primary Site:
ER-8 (with load-balancing WAN1 and WAN 2):
– WAN 1: eth0 linked to the ISP 1 through a Hitron cable modem in bridge mode. Thus the ER-8 gets the IP from the ISP.
– WAN 2: eth1 not linked.
– LAN 7: eth7 to internal LAN 2
– LAN 11: eth2 internal LAN 11
D-Link DWR-921 LTE:
– WAN 1: LTE link to Mobile service ISP.
EdgeRouter POE:
– WAN1: etho, IP = 192.168.7.10 linked to EdgeRouter ER-8 eth7 with gateway 192.168.7.1
– WAN2: eth1, IP = 192.168.0.50 link to D-Link DWR-921 LTE eth4 with gateway 192.168.0.1
– LAN 2: switch0, all the internal LAN
Remote Site:
UPC Cable Modem:
– WAN 1: eth0 linked to the ISP 1
EdgeRouter Lite:
– WAN 1: eth0, link to UPC Cable Modem eth1 with gateway 192.168.0.1
– LAN 9: eth1, local service LAN
– LAN 10: eth2, local management LAN
OpenVPN site-to-site between primary site (EdgeRouter POE) and remote site (EdgeRouter Lite). The tunnel is used to communicate between networks 192.168.2.0/24 from primary site and 192.168.9.0/24 from remote site.
IPSEC site-to-site with vti between primary site (ER-8) and remote site (EdgeRouter Lite).
By using IPSEC site-to-site with vti see https://blog.voina.org/edgerouter-ipsec-site-to-site-with-virtual-tunnel-interface/ I have the advantage of being able to tret the IPSEC associated virtual interface as any other routable interface, so OSPF can be applied for IPSEC also.
By using OSPF I am able to get rid of all the static routes necessary to be able to route through OpenVPN and IPSEC tunnels. This will make it very easy to add new local network on primary and remote sites without the need to add additional static routes.
Note that OSPF works by discovering routes from the neighbour. In the digram of the network we can see we have a loop made of the three EdgeRouter routers. Very important in this loop is the direction in which the OSPF will discover routes.
(EdgeRouter POE)(eth0) —uplink—> (eth7)(ER-8)(vti0) —IPSEC—> (vti0)(EdgeRouter Lite)(vtun0) —OpenVPN—> (vtun0)(EdgeRouter POE)
I am choosing the above direction because I want some networks to be routed through IPSEC connection that is much faster than the OpenVPN. Because OpenVPN is not using any hardware acceleration on EdgeRouter has much lower bandwidth than IPSEC.
The configuration will be done in the same order.
STEP 1: Configure OSPF on EdgeRouter POE
STEP 1.1: Allow OSPF traffic through the firewall
Add a new allow rule to the firewall for OSPF protocol:
name WAN_LOCAL {
default-action drop
description "WAN to router"
rule 1 {
action accept
description "Allow established/related"
state {
established enable
related enable
}
}
rule 2 {
action accept
description "Allow icmp"
log disable
protocol icmp
}
rule 3 {
action accept
description "Allow ospf"
log disable
protocol ospf
}
rule 4 {
action drop
description "Drop invalid state"
state {
invalid enable
}
}
}
STEP 1.2: Modify interfaces for OSPF
We have to declare in each interface definition that will be involved with OSPF, the OSPF parameters. Note that currently we will not use vtun0 for OSPF as is not necessary to “close” the loop. To route directly through the OpenVPN tunnel we will use static routes. See the static route used to force going to 192.168.9.0/24 from the remote site.
...
static {
interface-route 192.168.9.0/24 {
next-hop-interface vtun0 {
}
}
....
}
So we add to the eth0 interface the ospf routing parameters:
interfaces {
ethernet eth0 {
address 192.168.7.10/24
description Internet
dhcp-options {
default-route update
default-route-distance 210
name-server no-update
}
duplex auto
firewall {
in {
name WAN_IN
}
local {
name WAN_LOCAL
}
}
ip {
ospf {
dead-interval 40
hello-interval 10
priority 1
retransmit-interval 5
transmit-delay 1
}
}
poe {
output off
}
speed auto
}
...
STEP 1.3: Advertise networks for ospf
Under protocols add an ospf section:
protocols {
ospf {
area 0.0.0.0 {
network 192.168.2.0/24
network 192.168.7.0/24
}
parameters {
abr-type cisco
router-id 192.168.7.10
}
}
...
}
Note:
1. network 192.168.2.0/24 = with this we advertise a network for ospf. You can add any network that is connected to the router.
2. router-id 192.168.7.10 = this is important to be the router IP that is linked to the next neighbour. The IP of the immediate neighbour will be an IP from the same network (in our case 192.168.7.1 the IP on eth7 on the ER-8).
STEP 1.4: Commit changes
After we commit the changes osfp daemon will be automatically started and the router will start route discovery. Nothing will happen yet as there is no active neighbour.
STEP 2: Configure OSPF on ER-8
STEP 2.1: Allow ospf traffic through the firewall
Add a new allow rule to the firewall for OSPF protocol:
name WAN_LOCAL {
default-action drop
description "WAN to router"
rule 1 {
action accept
description "Allow established/related"
state {
established enable
related enable
}
}
rule 2 {
action accept
description "Allow IPSEC ESP"
log disable
protocol esp
}
rule 3 {
action accept
description "Allow IPSEC IKE"
destination {
port 500
}
log disable
protocol udp
}
rule 4 {
action accept
description "Allow IPSEC NAT-T"
destination {
port 4500
}
log disable
protocol udp
}
rule 5 {
action accept
description "Allow icmp"
log disable
protocol icmp
}
rule 6 {
action accept
description "Allow ospf"
log disable
protocol ospf
}
rule 7 {
action drop
description "Drop invalid state"
state {
invalid enable
}
}
}
STEP 2.2: Modify interfaces for OSPF
We have to declare in each interface definition that will be involved with ospf, the ospf parameters. This is the middle device so we have two interfaces that advertise for ospf: eth7 and vti0
So we add to the eth7 and vti0 interface the ospf routing parameters:
Note that vti0 has a special setting: “network point-to-point” to indicate that this is a special kind of OSPF.
interfaces {
...
ethernet eth7 {
address 192.168.7.1/24
description Internal
duplex auto
ip {
ospf {
dead-interval 40
hello-interval 10
priority 1
retransmit-interval 5
transmit-delay 1
}
}
speed auto
}
...
vti vti0 {
address 40.0.0.1/30
ip {
ospf {
dead-interval 40
hello-interval 10
network point-to-point
priority 1
retransmit-interval 5
transmit-delay 1
}
}
mtu 1436
}
...
STEP 2.3: Advertise networks for OSPF
Under protocols add an ospf section:
protocols {
ospf {
area 0.0.0.0 {
network 40.0.0.0/30
network 192.168.7.0/24
network 192.168.11.0/24
}
parameters {
abr-type cisco
router-id 192.168.7.1
}
}
...
}
Note:
1. network 192.168.7.0/24 = with this we advertise a network for ospf. You can add any network that is connected to the router. Note that we must add the vti0 network 40.0.0.0/30
2. router-id 192.168.7.1 = this is important to be the router IP that is linked to the previous neighbour. The IP of the immediate neighbour will be an IP from the same network (in our case 192.168.7.10 the IP on eth0 on the EdgeRouter-POE).
3. because the vti0 interface is declared as having ospf as “network point-to-point” we are able to see the next neighbour directly through this interface. In this case it does not matter that we have “router-id 192.168.7.1”.
STEP 2.4: Commit changes
After we commit the changes OSPF daemon will be automatically started and the router will start route discovery.
STEP 3: Configure OSPF on ER-Lite
STEP 3.1: Allow ospf traffic through the firewall
Add a new allow rule to the firewall for OSPF protocol:
name WAN_LOCAL {
default-action drop
description "WAN to router"
rule 1 {
action accept
description "Allow established/related"
state {
established enable
related enable
}
}
rule 2 {
action accept
description "Allow IPSEC ESP"
log disable
protocol esp
}
rule 3 {
action accept
description "Allow IPSEC IKE"
destination {
port 500
}
log disable
protocol udp
}
rule 4 {
action accept
description "Allow IPSEC NAT-T"
destination {
port 4500
}
log disable
protocol udp
}
rule 5 {
action accept
description "Allow icmp"
log disable
protocol icmp
}
rule 6 {
action accept
description "Allow ospf"
log disable
protocol ospf
}
rule 7 {
action drop
description "Drop invalid state"
state {
invalid enable
}
}
}
STEP 3.2: Modify interfaces for OSPF
We have to declare in each interface definition that will be involved with ospf, the ospf parameters. This is the last device so we have one interface that advertises for ospf: vti0
So we add to the vti0 interface the ospf routing parameters:
Note that vti0 has a special setting: “network point-to-point” to indicate that this is a special kind of OSPF.
interfaces {
...
vti vti0 {
address 40.0.0.2/30
ip {
ospf {
dead-interval 40
hello-interval 10
network point-to-point
priority 1
retransmit-interval 5
transmit-delay 1
}
}
mtu 1436
}
...
STEP 3.3: Advertise networks for OSPF
Under protocols add an ospf section:
protocols {
ospf {
area 0.0.0.0 {
network 40.0.0.0/30
network 192.168.9.0/24
network 192.168.10.0/24
}
parameters {
abr-type cisco
router-id yy.yy.yy.yy
}
}
static {
interface-route 192.168.2.0/24 {
next-hop-interface vtun0 {
}
}
}
}
}
Note:
1. network 192.168.9.0/24 = with this we advertise a network for ospf. You can add any network that is connected to the router. We must add the vti0 network 40.0.0.0/30
2. router-id yy.yy.yy.yy = this is the external WAN IP or the WAN IP. Because our immediate neighbour is the neighbour through the vti0 interface this router-id is not used to detect the neighbour. Because the vti0 interface is declared as having ospf as “network point-to-point” we are able to see the next neighbour directly through this interface.
STEP 3.4: Commit changes
After we commit the changes OSPF daemon will be automatically started and the router will start route discovery.
STEP 4:
Because the OSPF daemons were started in the order the configurations were done not everything was discovered. Let’s clear the ip tables and force discovery.
Execute on all the routers:
clear ip ospf process
Other useful ospf commands:
Show IP information for OSPF:
show ip ospf
Show IP database for OSPF network:
show ip ospf database show ip ospf neighbor
Show OSPF interface :
show ip ospf interface eth0 show ip ospf interface vti0
STEP 5: List the discovered routes on all routers
Note that there are a lot of routes discovered across networks and routers. There are even multiple routes for different networks from the remote site, through vtun0 and also through vti0.
The distance of OSPF detected routes is 110 so lower than static defined routes. This means that we can overwrite at any time an OSPF route with a static one.
Router 1: EdgeRouter POE
ubnt@ubnt:~$ show ip route
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
> - selected route, * - FIB route, p - stale info
IP Route Table for VRF "default"
S *> 0.0.0.0/0 [1/0] via 192.168.7.1, eth0
S 0.0.0.0/0 [200/0] via 192.168.0.1, eth1
C *> 10.99.99.1/32 is directly connected, vtun0
C *> 10.99.99.2/32 is directly connected, vtun0
O *> 40.0.0.0/30 [110/11] via 192.168.7.1, eth0, 21:23:23
C *> 127.0.0.0/8 is directly connected, lo
C *> 192.168.0.0/24 is directly connected, eth1
C *> 192.168.2.0/24 is directly connected, switch0
C *> 192.168.7.0/24 is directly connected, eth0
S *> 192.168.9.0/24 [1/0] is directly connected, vtun0
O 192.168.9.0/24 [110/12] via 192.168.7.1, eth0, 21:23:23
O *> 192.168.10.0/24 [110/12] via 192.168.7.1, eth0, 21:23:23
O *> 192.168.11.0/24 [110/11] via 192.168.7.1, eth0, 21:23:23
Router 2: ER-8
ubnt@ubnt:~$ show ip route
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
> - selected route, * - FIB route, p - stale info
IP Route Table for VRF "default"
S *> 0.0.0.0/0 [210/0] via 78.97.202.1, eth0
C *> 40.0.0.0/30 is directly connected, vti0
C *> xx.xx.xx.xx/23 is directly connected, eth0
C *> 127.0.0.0/8 is directly connected, lo
O *> 192.168.2.0/24 [110/11] via 192.168.7.10, eth7, 21:20:28
C *> 192.168.7.0/24 is directly connected, eth7
O *> 192.168.9.0/24 [110/11] via 40.0.0.2, vti0, 21:20:48
O *> 192.168.10.0/24 [110/11] via 40.0.0.2, vti0, 21:20:48
C *> 192.168.11.0/24 is directly connected, eth2
Router 3: EdgeRouter Lite
ubnt@ubnt:~$ show ip route
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
> - selected route, * - FIB route, p - stale info
IP Route Table for VRF "default"
S *> 0.0.0.0/0 [210/0] via 192.168.0.1, eth0
S 0.0.0.0/0 [1/0] via 10.0.0.1 inactive
C *> 10.99.99.1/32 is directly connected, vtun0
C *> 10.99.99.2/32 is directly connected, vtun0
C *> 40.0.0.0/30 is directly connected, vti0
C *> 127.0.0.0/8 is directly connected, lo
C *> 192.168.0.0/24 is directly connected, eth0
S *> 192.168.2.0/24 [1/0] is directly connected, vtun0
O 192.168.2.0/24 [110/21] via 40.0.0.1, vti0 inactive, 21:25:10
O *> 192.168.7.0/24 [110/20] via 40.0.0.1, vti0, 21:25:35
C *> 192.168.9.0/24 is directly connected, eth1
C *> 192.168.10.0/24 is directly connected, eth2
O *> 192.168.11.0/24 [110/20] via 40.0.0.1, vti0, 21:25:35