For the wormhole vlan only, it will appear as though the branch host is on the HQ network when in reality, the host is actually on a remote branch network.

Scenario

You have a branch office in Reykjavik and you need a WiFi Internet connection for IoT devices that egress the Internet connection of your HQ in San Antonio, Texas, so that when the IoT devices access the Internet, the Internet resource traces your GEO-IP location to San Antonio, TX instead of the actual location in Reykjavik. Other devices on the Reykjavik network should use the local Iceland Internet directly.

TLDR; For the wormhole vlan only, appear as though your branch host is in the same GEO-IP as the HQ network when in reality, the host is actually on the remote branch network.

Goals

1. Create an SSID for branch office direct Internet access that uses the branch office Internet gateway for general Internet traffic.

2. Create another SSID that for any device connected to that SSID where all traffic is forced across a VPN tunnel between the branch office and HQ. This SSID will use the Internet gateway from the remote/far side of the tunnel (The HQ Internet connection) for all Internet traffic. We will call this special SSID access a "wormhole" (Wormhole is a Sci-Fi reference for those that do no speak that dialect of nerd).

3. The branch office uses a consumer grade Internet connection with a dynamic IP address. So any site-to-site (S2S) VPN configuration will need to be able to respond dynamically to changing endpoint IP addresses without manual configuration.

Possible solutions & why we can't use them

• RAVPN solutions like AnyConnect could certainly accomplish this, however many devices do not support AnyConnect or any software SD-WAN/VPN client for that matter. (IoT being the poster child for this).

• A simple policy based or route based S2S VPN sounds attractive, however, the nature of a default route 0.0.0.0/0 to use the VPN would impact all traffic on both SSIDs. This would not meet the goal of #1 above.

• Policy Based Routing (PBR) would be an excellent fit here, however, at least in the case of the current feature set of the Cisco Secure Firewall Threat Defense (FTD) platform, vlan interfaces (interface "switch" mode) are not supported as PBR sources. Only routed interfaces (interface in "router" mode) can be used as PBR sources. We only want to use 1 Wireless Access Point (WAP), so the interface feeding the WAP must support both the direct Internet SSID and the wormhole SSID. The WAP interface MUST be "switchport mode trunk", and to be a trunk, the firewall interface needs to move from a "router" interface to a "switch" interface making it ineligible to use as a source in a PBR policy on FTD.

Working Solution Summary

1. Goal #1: Nothing special is needed. We will configure the usual needed outbound access-control policies and NATs needed for direct Internet access.

2. Goal #2: Cisco Secure Firewall Threat Defense supports virtual routing and forwarding (VRF). This will allow us to create one routing table for the dia SSID and another routing table for the wormhole SSID. This will allow us to have 2 different default gateways, which would otherwise be impossible (without PBR).

3. Goal #3: We will use SD-WAN; a dynamic VTI (dvti) hub and spoke S2S VPN configuration. HQ (FPR-1150) is the hub and the branch office (FPR-1010) as a spoke. This solves the branch "IP changing" problem.

Detailed Solution

My devices are managed by a Cisco Secure cloud-delivered Firewall Management Center (cdFMC) provided by Cisco Defense Orchestrator (CDO), which is an excellent SaaS management platform from Cisco. This solution would also work using a traditional "OnPrem" Firewall Management Center (FMC).

SD-WAN Configuration

• Detailed configuration of the SD-WAN tunnel is beyond the scope of this article, but it is a dvti hub-and-spoke VPN configuration. You can learn more here.

• Create a loopback interface on both FTDs and assign them a /32 IP address. We will use this address as the "borrowed" IP for the dvti tunnel endpoints and for routing purposes.

• Create a vti interface on both firewalls and associate a security-zone for both.

• Use the Site-to-Site VPN wizard, select "Hub and Spoke" and configure your SD-WAN tunnel between the branch (spoke) FTD and the HQ (hub) FTD.

• Configure BGP on both FTDs to exchange needed routes (Or use static routes) using the loopback addresses as the neighbor IPs. I only added the wormhole subnet on the branch side and the Internet egress subnet on the HQ side as BGP advertised networks. The loopback networks will be shared automatically thanks to IKEv2.

• Make sure the tunnel is up and passing traffic before moving on to the VRF configuration.

  

VRF Configuration

• On the branch FTD, create a new VRF. Add only the wormhole vlan interface, the loopback interface, and the vti interface to this new VRF.

  

On the branch FTD, create a default static route in the wormhole VRF. This will be a 0.0.0.0/0 route on interface vti_1 that points to the loopback interface IP of the HQ FTD.

HQ FTD

In addition to setting up the SD-WAN and BGP on the HQ firewall, there are a few more considerations to take into account.

Access-policy

• Source Network: wormhole vlan IP subnet

• Destination Network: any-ipv4 or whatever your outbound policy is...

• Source Zone: HQ vti interface security-zone

• Destination Zone: HQ outside interface security-zone (Internet)

• Ports/Applications/GeoIP: Per your usual policy

NAT

• You will want to policy NAT the wormhole vlan IP subnet (original source) to whatever your HQ outside interface uses (static NAT, PAT, etc).

Troubleshooting

• Make sure S2S VPN is up and passing traffic

• Check BGP/Routing config with "show route" and "show route vrf"

• Run packet-tracers on both firewalls and pay attention to

  • egress interface

  • NAT behavior

  • access-policy behavior

• Packet captures can help in determining where the packets are coming from, going to, and behaving