As you can see (from left to right), there is 1 SRX 240 acting as the core firewall, 1 core EX4200 switch, 2 SRX 240's acting as next hops, both of which have VPN connections terminated to them from another SRX 240 at a remote site. The VPN connections are traversing an MPLS backbone which does not consist of Juniper gear, nor will it be discussed in this post. Our concern is regarding the fact that there are two possible paths (dual VPNs) between the core and the remote SRX.
In this scenario, lets assume we want to use static routing, but also allow for route failover in the event of a VPN outage.
There are multiple options that we can utilize to provide route failover in this type of scenario. Some of these options include:
- Bi-directional Forwarding Detection
- Real-time Network Performance with IP Monitoring
- VPN Monitoring
I went with VPN Monitoring due to the fact that we are using VPNs. If this scenario didn't include VPNs, then I would've went with BFD or RPM and IP Monitoring. Here is what our configurations will look like:
Core SRX Configuration (10.0.0.1):
Configure the interface that will connect to the core switch:
set interfaces ge-0/0/0 unit 0 description "*** CORE ***"
set interfaces ge-0/0/0 unit 0 family inet address 10.0.0.1/24
Configure a route for the remote site:
set routing-options static route 10.0.99.0/24 next-hop 10.0.0.2
Core EX Configuration (10.0.0.5):
Configure the Routed VLAN Interface:
set interfaces vlan unit 0 description "*** CORE ***"
set interfaces vlan unit 0 family inet address 10.0.0.5/24
set vlans CORE vlan-id 3
set vlans CORE l3-interface vlan.0
Configure the interfaces on the switch:
set interfaces ge-0/0/0 unit 0 family ethernet-switching vlan members CORE
set interfaces ge-0/0/1 unit 0 family ethernet-switching vlan members CORE
set interfaces ge-0/0/2 unit 0 family ethernet-switching vlan members CORE
etc...
VPN1 SRX Configuration (10.0.0.2):
Configure the Routed VLAN Interface:
set interfaces vlan unit 0 description "*** CORE ***"
set interfaces vlan unit 0 family inet address 10.0.0.2/24
set vlans CORE vlan-id 3
set vlans CORE l3-interface vlan.0
Configure the interfaces that will connect to the core switch:
set interfaces ge-0/0/13 unit 0 family ethernet-switching vlan members CORE
set interfaces ge-0/0/14 unit 0 family ethernet-switching vlan members CORE
Configure the interface that will act as the WAN interface for our MPLS connection:
set interfaces ge-0/0/15 unit 0 description "*** MPLS CONNECTION TO REMOTE SITE ***"
set interfaces ge-0/0/15 unit 0 family inet address 1.1.1.1/30
Configure the interface that will be used for the VPN:
set interfaces st0 unit 1 description "*** CONNECTION TO REMOTE SITE ***"
set interfaces st0 unit 1 family inet address 100.100.100.1/30
Configure a default route:
set routing-options static route 0.0.0.0/0 next-hop 10.0.0.1
Configure a preferred route to the remote site via 10.0.0.2, and then a backup route via 10.0.0.3:
set routing-options static route 10.0.99.0/24 next-hop st0.1
set routing-options static route 10.0.99.0/24 qualified-next-hop 10.0.0.3 preference 6
Configure the route-based VPN:
set security ike policy IKE-POLICY-REMOTE-SITE mode main
set security ike policy IKE-POLICY-REMOTE-SITE proposal-set standard
set security ike policy IKE-POLICY-REMOTE-SITE pre-shared-key ascii-text testing123
set security ike gateway GW-REMOTE-SITE ike-policy IKE-POLICY-REMOTE-SITE
set security ike gateway GW-REMOTE-SITE address 1.1.1.2
set security ike gateway GW-REMOTE-SITE external-interface ge-0/0/15.0
set security ipsec policy IPSEC-POLICY-REMOTE-SITE perfect-forward-secrecy keys group2
set security ipsec policy IPSEC-POLICY-REMOTE-SITE proposal-set standard
set security ipsec vpn VPN-REMOTE-SITE bind-interface st0.1
set security ipsec vpn VPN-REMOTE-SITE ike gateway GW-REMOTE-SITE
set security ipsec vpn VPN-REMOTE-SITE ike ipsec-policy
set security ipsec vpn VPN-REMOTE-SITE establish-tunnels immediately
set security flow tcp-mss ipsec-vpn mss 1350
Configure VPN monitoring:
set security ipsec vpn-monitor-options interval 2
set security ipsec vpn-monitor-options threshold 10
set security ipsec vpn VPN-REMOTE-SITE vpn-monitor optimized
Create security zones and policies:
NOTE: For testing purposes, I opened up everything to my zones and policies. I always start this way so as to make things easier on myself. It is more effective to get what you want working first and then go back and secure things as needed (granted this is in a lab environment).
set security zones security-zone CORE host-inbound-traffic system-services all
set security zones security-zone CORE host-inbound-traffic protocols all
set security zones security-zone CORE interfaces vlan.0
set security zones security-zone MPLS host-inbound-traffic system-services all
set security zones security-zone MPLS host-inbound-traffic protocols all
set security zones security-zone MPLS interfaces ge-0/0/15.0
set security zones security-zone VPN host-inbound-traffic system-services all
set security zones security-zone VPN host-inbound-traffic protocols all
set security zones security-zone VPN interfaces st0.1
[edit security policies]
set interfaces ge-0/0/0 unit 0 description "*** CORE ***"
set interfaces ge-0/0/0 unit 0 family inet address 10.0.0.1/24
Configure a route for the remote site:
set routing-options static route 10.0.99.0/24 next-hop 10.0.0.2
Core EX Configuration (10.0.0.5):
Configure the Routed VLAN Interface:
set interfaces vlan unit 0 description "*** CORE ***"
set interfaces vlan unit 0 family inet address 10.0.0.5/24
set vlans CORE vlan-id 3
set vlans CORE l3-interface vlan.0
Configure the interfaces on the switch:
set interfaces ge-0/0/0 unit 0 family ethernet-switching vlan members CORE
set interfaces ge-0/0/1 unit 0 family ethernet-switching vlan members CORE
set interfaces ge-0/0/2 unit 0 family ethernet-switching vlan members CORE
etc...
VPN1 SRX Configuration (10.0.0.2):
Configure the Routed VLAN Interface:
set interfaces vlan unit 0 description "*** CORE ***"
set interfaces vlan unit 0 family inet address 10.0.0.2/24
set vlans CORE vlan-id 3
set vlans CORE l3-interface vlan.0
Configure the interfaces that will connect to the core switch:
set interfaces ge-0/0/13 unit 0 family ethernet-switching vlan members CORE
set interfaces ge-0/0/14 unit 0 family ethernet-switching vlan members CORE
Configure the interface that will act as the WAN interface for our MPLS connection:
set interfaces ge-0/0/15 unit 0 description "*** MPLS CONNECTION TO REMOTE SITE ***"
set interfaces ge-0/0/15 unit 0 family inet address 1.1.1.1/30
Configure the interface that will be used for the VPN:
set interfaces st0 unit 1 description "*** CONNECTION TO REMOTE SITE ***"
set interfaces st0 unit 1 family inet address 100.100.100.1/30
Configure a default route:
set routing-options static route 0.0.0.0/0 next-hop 10.0.0.1
Configure a preferred route to the remote site via 10.0.0.2, and then a backup route via 10.0.0.3:
set routing-options static route 10.0.99.0/24 next-hop st0.1
set routing-options static route 10.0.99.0/24 qualified-next-hop 10.0.0.3 preference 6
Configure the route-based VPN:
set security ike policy IKE-POLICY-REMOTE-SITE mode main
set security ike policy IKE-POLICY-REMOTE-SITE proposal-set standard
set security ike policy IKE-POLICY-REMOTE-SITE pre-shared-key ascii-text testing123
set security ike gateway GW-REMOTE-SITE ike-policy IKE-POLICY-REMOTE-SITE
set security ike gateway GW-REMOTE-SITE address 1.1.1.2
set security ike gateway GW-REMOTE-SITE external-interface ge-0/0/15.0
set security ipsec policy IPSEC-POLICY-REMOTE-SITE perfect-forward-secrecy keys group2
set security ipsec policy IPSEC-POLICY-REMOTE-SITE proposal-set standard
set security ipsec vpn VPN-REMOTE-SITE bind-interface st0.1
set security ipsec vpn VPN-REMOTE-SITE ike gateway GW-REMOTE-SITE
set security ipsec vpn VPN-REMOTE-SITE ike ipsec-policy
VPN-REMOTE-SITE
set security flow tcp-mss ipsec-vpn mss 1350
Configure VPN monitoring:
set security ipsec vpn-monitor-options interval 2
set security ipsec vpn-monitor-options threshold 10
set security ipsec vpn VPN-REMOTE-SITE vpn-monitor optimized
set security ipsec vpn VPN-REMOTE-SITE vpn-monitor source-interface ge-0/0/15.0
set security ipsec vpn VPN-REMOTE-SITE vpn-monitor destination-ip 1.1.1.2
NOTE: For testing purposes, I opened up everything to my zones and policies. I always start this way so as to make things easier on myself. It is more effective to get what you want working first and then go back and secure things as needed (granted this is in a lab environment).
set security zones security-zone CORE host-inbound-traffic system-services all
set security zones security-zone CORE host-inbound-traffic protocols all
set security zones security-zone CORE interfaces vlan.0
set security zones security-zone MPLS host-inbound-traffic system-services all
set security zones security-zone MPLS host-inbound-traffic protocols all
set security zones security-zone MPLS interfaces ge-0/0/15.0
set security zones security-zone VPN host-inbound-traffic system-services all
set security zones security-zone VPN host-inbound-traffic protocols all
set security zones security-zone VPN interfaces st0.1
[edit security policies]
set from-zone CORE to-zone VPN policy CORE-TO-VPN match source-address any
set from-zone CORE to-zone VPN policy CORE-TO-VPN match destination-address any
set from-zone CORE to-zone VPN policy CORE-TO-VPN match application any
set from-zone CORE to-zone VPN policy CORE-TO-VPN then permit
set from-zone VPN to-zone CORE policy VPN-TO-CORE match source-address any
set from-zone VPN to-zone CORE policy VPN-TO-CORE match destination-address any
set from-zone VPN to-zone CORE policy VPN-TO-CORE match application any
set from-zone VPN to-zone CORE policy VPN-TO-CORE then permit
set from-zone VPN to-zone VPN policy VPN-TO-VPN match source-address any
set from-zone VPN to-zone VPN policy VPN-TO-VPN match destination-address any
set from-zone VPN to-zone VPN policy VPN-TO-VPN match application any
set from-zone VPN to-zone VPN policy VPN-TO-VPN then permit
set from-zone CORE to-zone MPLS policy CORE-TO-MPLS match source-address any
set from-zone CORE to-zone MPLS policy CORE-TO-MPLS match destination-address any
set from-zone CORE to-zone MPLS policy CORE-TO-MPLS match application any
set from-zone CORE to-zone MPLS policy CORE-TO-MPLS then permit
set from-zone MPLS to-zone CORE policy MPLS-TO-CORE match source-address any
set from-zone MPLS to-zone CORE policy MPLS-TO-CORE match destination-address any
set from-zone MPLS to-zone CORE policy MPLS-TO-CORE match application any
set from-zone MPLS to-zone CORE policy MPLS-TO-CORE then permit
set from-zone VPN to-zone MPLS policy VPN-TO-MPLS match source-address any
set from-zone VPN to-zone MPLS policy VPN-TO-MPLS match destination-address any
set from-zone VPN to-zone MPLS policy VPN-TO-MPLS match application any
set from-zone VPN to-zone MPLS policy VPN-TO-MPLS then permit
set from-zone MPLS to-zone VPN policy MPLS-TO-VPN match source-address any
set from-zone MPLS to-zone VPN policy MPLS-TO-VPN match destination-address any
set from-zone MPLS to-zone VPN policy MPLS-TO-VPN match application any
set from-zone MPLS to-zone VPN policy MPLS-TO-VPN then permit
set from-zone CORE to-zone CORE policy CORE-TO-CORE match source-address any
set from-zone CORE to-zone CORE policy CORE-TO-CORE match destination-address any
set from-zone CORE to-zone CORE policy CORE-TO-CORE match application any
set from-zone CORE to-zone CORE policy CORE-TO-CORE then permit
Configure a firewall filter so that VPN1 SRX only accepts IKE packets from specific devices:
NOTE: This can also be done with policy. When I was testing failover and simulating a down tunnel, I noticed that my VPN connection would attempt to renegotiate over my second VPN at the remote site. I simply set a firewall filter to ensure that this SRX only accepts IKE packets from the MPLS zone IP of the remote site.
set firewall family inet filter REJECT-IKE term T1 from source-address 0.0.0.0/0
set firewall family inet filter REJECT-IKE term T1 from source-address 1.1.1.2/32 except
set firewall family inet filter REJECT-IKE term T1 from protocol udp
set firewall family inet filter REJECT-IKE term T1 from destination-port 500
set firewall family inet filter REJECT-IKE term T1 then reject
set firewall family inet filter REJECT-IKE term T2 then accept
set interfaces lo0 unit 0 family inet filter input REJECT-IKE
VPN2 SRX Configuration (10.0.0.3):
Configure the Routed VLAN Interface:
set interfaces vlan unit 0 description "*** CORE ***"
set interfaces vlan unit 0 family inet address 10.0.0.3/24
set vlans CORE vlan-id 3
set vlans CORE l3-interface vlan.0
Configure the interfaces that will connect to the core switch:
set interfaces ge-0/0/13 unit 0 family ethernet-switching vlan members CORE
set interfaces ge-0/0/14 unit 0 family ethernet-switching vlan members CORE
Configure the interface that will act as the WAN interface for our MPLS connection:
set interfaces ge-0/0/15 unit 0 description "*** MPLS CONNECTION TO REMOTE SRX ***"
set interfaces ge-0/0/15 unit 0 family inet address 2.2.2.1/30
Configure the interface that will be used for the VPN:
set interfaces st0 unit 1 description "*** CONNECTION TO REMOTE SITE ***"
set interfaces st0 unit 1 family inet address 200.200.200.1/30
Configure a default route:
set routing-options static route 0.0.0.0/0 next-hop 10.0.0.1
Configure a route to the remote site:
set routing-options static route 10.0.99.0/24 next-hop 200.200.200.2
Configure the route-based VPN:
set security ike policy IKE-POLICY-REMOTE-SITE mode main
set security ike policy IKE-POLICY-REMOTE-SITE proposal-set standard
set security ike policy IKE-POLICY-REMOTE-SITE pre-shared-key ascii-text testing123
set security ike gateway GW-REMOTE-SITE ike-policy IKE-POLICY-REMOTE-SITE
set security ike gateway GW-REMOTE-SITE address 2.2.2.2
set security ike gateway GW-REMOTE-SITE external-interface ge-0/0/15.0
set security ipsec policy IPSEC-POLICY-REMOTE-SITE perfect-forward-secrecy keys group2
set security ipsec policy IPSEC-POLICY-REMOTE-SITE proposal-set standard
set security ipsec vpn VPN-REMOTE-SITE bind-interface st0.1
set security ipsec vpn VPN-REMOTE-SITE ike gateway GW-REMOTE-SITE
set security ipsec vpn VPN-REMOTE-SITE ike ipsec-policy VPN-REMOTE-SITE
set security ipsec vpn VPN-REMOTE-SITE establish-tunnels immediately
set security flow tcp-mss ipsec-vpn mss 1350
NOTE: There is no need to enable VPN monitoring on this device, as we only need to be monitoring our primary path to/from the remote site. Also, I configured the security zones and policies exactly like VPN1 SRX.
Configure a firewall filter so that VPN1 SRX only accepts IKE packets from specific devices:
set firewall family inet filter REJECT-IKE term T1 from source-address 0.0.0.0/0
set firewall family inet filter REJECT-IKE term T1 from source-address 2.2.2.2/32 except
set firewall family inet filter REJECT-IKE term T1 from protocol udp
set firewall family inet filter REJECT-IKE term T1 from destination-port 500
set firewall family inet filter REJECT-IKE term T1 then reject
set firewall family inet filter REJECT-IKE term T2 then accept
set interfaces lo0 unit 0 family inet filter input REJECT-IKE
Remote SRX Configuration (10.0.99.1):
set interfaces vlan unit 0 description "*** TRUST ***"
set interfaces vlan unit 0 family inet address 10.0.99.1/24
set vlans TRUST vlan-id 3
set vlans TRUST l3-interface vlan.0
Configure the interface that will act as the WAN interface for our MPLS connections:
set interfaces ge-0/0/0 unit 10 description "*** MPLS CONNECTION TO VPN1 SRX ***"
set interfaces ge-0/0/0 unit 10 family inet address 1.1.1.2/30
set interfaces ge-0/0/0 unit 20 description "*** MPLS CONNECTION TO VPN2 SRX ***"
set interfaces ge-0/0/0 unit 20 family inet address 2.2.2.2/30
Configure the interfaces that will be used for the VPN:
set interfaces st0 unit 1 description "*** CONNECTION TO VPN1 SRX ***"
set interfaces st0 unit 1 family inet address 100.100.100.2/30
set interfaces st0 unit 2 description "*** CONNECTION TO VPN2 SRX ***"
set interfaces st0 unit 2 family inet address 200.200.200.2/30
Configure a preferred route to the core via 10.0.0.2, and then a backup route via 10.0.0.3:
set routing-options static route 0.0.0.0/0 next-hop st0.1
set routing-options static route 0.0.0.0/0 qualified-next-hop st0.2 preference 6
Configure the route-based VPNs:
set security ike policy IKE-POLICY-TO-VPN1 mode main
set security ike policy IKE-POLICY-TO-VPN1 proposal-set standard
set security ike policy IKE-POLICY-TO-VPN1 pre-shared-key ascii-text testing123
set security ike policy IKE-POLICY-TO-VPN2 mode main
set security ike policy IKE-POLICY-TO-VPN2 proposal-set standard
set security ike policy IKE-POLICY-TO-VPN2 pre-shared-key ascii-text testing123
set security ike gateway GW-TO-VPN1 ike-policy IKE-POLICY-TO-VPN1
set security ike gateway GW-TO-VPN1 address 1.1.1.1
set security ike gateway GW-TO-VPN1 external-interface ge-0/0/0.10
set security ike gateway GW-TO-VPN2 ike-policy IKE-POLICY-TO-VPN2
set security ike gateway GW-TO-VPN2 address 2.2.2.1
set security ike gateway GW-TO-VPN2 external-interface ge-0/0/0.20
set security ipsec policy IPSEC-POLICY-TO-VPN1 perfect-forward-secrecy keys group2
set security ipsec policy IPSEC-POLICY-TO-VPN1 proposal-set standard
set security ipsec policy IPSEC-POLICY-TO-VPN2 perfect-forward-secrecy keys group2
set security ipsec policy IPSEC-POLICY-TO-VPN2 proposal-set standard
set security ipsec vpn VPN-TO-VPN1 bind-interface st0.1
set security ipsec vpn VPN-TO-VPN1 ike gateway GW-TO-VPN1
set security ipsec vpn VPN-TO-VPN1 ike ipsec-policy IPSEC-POLICY-TO-VPN1
set security ipsec vpn VPN-TO-VPN1 establish-tunnels immediately
set security ipsec vpn VPN-TO-VPN2 bind-interface st0.2
set security ipsec vpn VPN-TO-VPN2 ike gateway GW-TO-VPN2
set security ipsec vpn VPN-TO-VPN2 ike ipsec-policy IPSEC-POLICY-TO-VPN2
set security ipsec vpn VPN-TO-VPN2 establish-tunnels immediately
set security flow tcp-mss ipsec-vpn mss 1350
Configure VPN monitoring for the VPN to VPN1 SRX:
set security ipsec vpn-monitor-options interval 2
set security ipsec vpn-monitor-options threshold 10
set security ipsec vpn VPN-TO-VPN1 vpn-monitor optimized
set security ipsec vpn VPN-TO-VPN1 vpn-monitor source-interface ge-0/0/0.10
set security ipsec vpn VPN-TO-VPN1 vpn-monitor destination-ip 1.1.1.1
Create security zones and policies:
set security zones security-zone TRUST host-inbound-traffic system-services all
set security zones security-zone TRUST host-inbound-traffic protocols all
set security zones security-zone TRUST interfaces vlan.0
set security zones security-zone MPLS host-inbound-traffic system-services all
set security zones security-zone MPLS host-inbound-traffic protocols all
set security zones security-zone MPLS interfaces ge-0/0/0.10
set security zones security-zone MPLS interfaces ge-0/0/0.20
set security zones security-zone VPN host-inbound-traffic system-services all
set security zones security-zone VPN host-inbound-traffic protocols all
set security zones security-zone VPN interfaces st0.1
set security zones security-zone VPN interfaces st0.2
[edit security policies]
set from-zone TRUST to-zone VPN policy TRUST-TO-VPN match destination-address any
set from-zone TRUST to-zone VPN policy TRUST-TO-VPN match application any
set from-zone TRUST to-zone VPN policy TRUST-TO-VPN then permit
set from-zone VPN to-zone TRUST policy VPN-TO-TRUST match source-address any
set from-zone VPN to-zone TRUST policy VPN-TO-TRUST match destination-address any
set from-zone VPN to-zone TRUST policy VPN-TO-TRUST match application any
set from-zone VPN to-zone TRUST policy VPN-TO-TRUST then permit
set from-zone VPN to-zone VPN policy VPN-TO-VPN match source-address any
set from-zone VPN to-zone VPN policy VPN-TO-VPN match destination-address any
set from-zone VPN to-zone VPN policy VPN-TO-VPN match application any
set from-zone VPN to-zone VPN policy VPN-TO-VPN then permit
set from-zone TRUST to-zone MPLS policy TRUST-TO-MPLS match source-address any
set from-zone TRUST to-zone MPLS policy TRUST-TO-MPLS match destination-address any
set from-zone TRUST to-zone MPLS policy TRUST-TO-MPLS match application any
set from-zone TRUST to-zone MPLS policy TRUST-TO-MPLS then permit
set from-zone MPLS to-zone TRUST policy MPLS-TO-TRUST match source-address any
set from-zone MPLS to-zone TRUST policy MPLS-TO-TRUST match destination-address any
set from-zone MPLS to-zone TRUST policy MPLS-TO-TRUST match application any
set from-zone MPLS to-zone TRUST policy MPLS-TO-TRUST then permit
set from-zone VPN to-zone MPLS policy VPN-TO-MPLS match source-address any
set from-zone VPN to-zone MPLS policy VPN-TO-MPLS match destination-address any
set from-zone VPN to-zone MPLS policy VPN-TO-MPLS match application any
set from-zone VPN to-zone MPLS policy VPN-TO-MPLS then permit
set from-zone MPLS to-zone VPN policy MPLS-TO-VPN match source-address any
set from-zone MPLS to-zone VPN policy MPLS-TO-VPN match destination-address any
set from-zone MPLS to-zone VPN policy MPLS-TO-VPN match application any
set from-zone MPLS to-zone VPN policy MPLS-TO-VPN then permit
set from-zone TRUST to-zone TRUST policy TRUST-TO-TRUST match source-address any
set from-zone TRUST to-zone TRUST policy TRUST-TO-TRUST match destination-address any
set from-zone TRUST to-zone TRUST policy TRUST-TO-TRUST match application any
set from-zone TRUST to-zone TRUST policy TRUST-TO-TRUST then permit
Testing and Validation:
We can test and validate our design by issuing the following command at the remote site:
set interfaces ge-0/0/0.10 disable
commit confirmed 5
By issuing commit confirmed 5, we are committing the configuration for a total of 5 minutes, and then allowing it to roll back to how it was before we disabled the interface. Make sure to disable the interface and NOT deactivate the interface. Deactivating is the equivalent of removing it from the configuration and should not be used to test failover.
Traceroute prior to disabling the interface:
traceroute to 10.0.99.1 (10.0.99.1), 64 hops max, 52 byte packets
1 10.0.0.1 (10.0.0.1) 4.093 ms 4.044 ms 5.812 ms
2 10.0.0.2 (10.0.0.2) 4.219 ms 4.543 ms 4.456 ms
3 10.0.99.1 (10.0.99.1) 7.260 ms 6.652 ms 6.949 ms
Continuous ping during failover:
PING 10.0.99.1 (10.0.99.1): 56 data bytes
64 bytes from 10.0.99.1: icmp_seq=0 ttl=61 time=9.161 ms
64 bytes from 10.0.99.1: icmp_seq=1 ttl=61 time=8.943 ms
64 bytes from 10.0.99.1: icmp_seq=2 ttl=61 time=8.779 ms
64 bytes from 10.0.99.1: icmp_seq=3 ttl=61 time=8.885 ms
64 bytes from 10.0.99.1: icmp_seq=4 ttl=61 time=8.730 ms
64 bytes from 10.0.99.1: icmp_seq=5 ttl=61 time=8.648 ms
64 bytes from 10.0.99.1: icmp_seq=6 ttl=61 time=6.584 ms
64 bytes from 10.0.99.1: icmp_seq=7 ttl=61 time=9.076 ms
64 bytes from 10.0.99.1: icmp_seq=8 ttl=61 time=6.396 ms
64 bytes from 10.0.99.1: icmp_seq=9 ttl=61 time=8.971 ms
64 bytes from 10.0.99.1: icmp_seq=10 ttl=61 time=9.061 ms
64 bytes from 10.0.99.1: icmp_seq=11 ttl=61 time=9.226 ms
64 bytes from 10.0.99.1: icmp_seq=12 ttl=61 time=9.581 ms
64 bytes from 10.0.99.1: icmp_seq=13 ttl=61 time=8.922 ms
64 bytes from 10.0.99.1: icmp_seq=14 ttl=61 time=8.447 ms
Request timeout for icmp_seq 15
Request timeout for icmp_seq 16
Request timeout for icmp_seq 17
Request timeout for icmp_seq 18
Request timeout for icmp_seq 19
Request timeout for icmp_seq 20
Request timeout for icmp_seq 21
Request timeout for icmp_seq 22
Request timeout for icmp_seq 23
Request timeout for icmp_seq 24
Request timeout for icmp_seq 25
Request timeout for icmp_seq 26
Request timeout for icmp_seq 27
Request timeout for icmp_seq 28
Request timeout for icmp_seq 29
Request timeout for icmp_seq 30
Request timeout for icmp_seq 31
Request timeout for icmp_seq 32
Request timeout for icmp_seq 33
Request timeout for icmp_seq 34
Request timeout for icmp_seq 35
Request timeout for icmp_seq 36
64 bytes from 10.0.99.1: icmp_seq=37 ttl=61 time=7.194 ms
64 bytes from 10.0.99.1: icmp_seq=38 ttl=61 time=9.577 ms
64 bytes from 10.0.99.1: icmp_seq=39 ttl=61 time=9.314 ms
64 bytes from 10.0.99.1: icmp_seq=40 ttl=61 time=9.300 ms
64 bytes from 10.0.99.1: icmp_seq=41 ttl=61 time=9.399 ms
64 bytes from 10.0.99.1: icmp_seq=42 ttl=61 time=9.414 ms
64 bytes from 10.0.99.1: icmp_seq=43 ttl=61 time=11.261 ms
64 bytes from 10.0.99.1: icmp_seq=44 ttl=61 time=9.284 ms
64 bytes from 10.0.99.1: icmp_seq=45 ttl=61 time=6.125 ms
Traceroute after failover:
traceroute to 10.0.99.1 (10.0.99.1), 64 hops max, 52 byte packets
1 10.0.0.1 (10.0.0.1) 4.055 ms 4.495 ms 4.185 ms
2 10.0.0.2 (10.0.0.2) 4.466 ms 4.745 ms 4.257 ms
3 10.0.0.3 (10.0.0.3) 4.732 ms 4.510 ms 4.490 ms
4 10.0.99.1 (10.0.99.1) 13.306 ms 6.384 ms 5.940 ms
As you can see, with failover in effect, the new route traverses our backup route to the remote site. Enjoy!