AWS Direct Connect Note

Routing Static vs Dynamic

• Static: add route explicitly

• Dynamic:

  • Routing routes get propagated automatically using BGP - TCP port 179

  • Using Path-vector protocol to exchange best path between AS (AS_PATH)

  • Routing decision:

    • Weight (within AS)

    • AS_PATH between AS

    • LOCAL_PREF (within AS)

    • MED (between AS)

Direct Connect Components

• Dedicated DX:

  • Allows 1, 10, and 100 Gbps

• Hosted DX:

  • Allows 50, 100, 200, 400, 500 Mbps and 1, 10 Gbps (by AWS DX partner)

  • 1 hosted connection = 1 VIF

• Hosted VIF:

  • Applicable for hosted DX and dedicated DX

  • Connection < 1Gpbs support only 1 VIF

  • Private VIF should associate VGW

Single-mode fiber 1000BASE-LX -> 1G, 10GBASE-LR -> 10G, 100GBASE-LR4 -> 100G and 801.1 Q VLAN must be supported.

DX Virtual Interfaces

Public VIF

Only Public VIF has prefixes to be advertised. Customer Router CIDR /30 if don't have AWS support /31. Each BGP allows max 1000 route prefixes from the customer router.

Public VIF Inbound

• Traffic must be destined to Amazon public prefixes

• AWS DX allows packet filtering

Public VIF Outbound

• Longest prefix match & AS_PATH can be used to influence the routing

• Advertise public prefixes with NO_EXPORT BGP community tag

Private VIF

Only Private VIF has a gateway type (VGW or Direct Connect Gateway). Jumos frame applicable only to Private - 9001 & Transit - 8500 MTU.

Must attach VPC <-> VGW <-> Private VIF. VIF and VGW must be in the same region.

Can announce 100 prefixes to AWS & the routes can be automatically propagated into subnet routable.

The propagated route takes precedence over the default route to IGW

Can't access inside VPC:

• VPC DNS resolver at Base + 2

• VPC gateway endpoints

Transit VIF

• Transit VIF <-> DX GW <-> TGW

• Allow attaching multiple DX GW to 1 TGW

Direct Connect Gateway

• FREE, charge only egress + port

• Allows access to multiple VPCs using a single private VIF (not public) using VGW. (multi-regions & multi-accounts)

• No transitive connection

• DX GW & VIF should be created in the same account

• 1 DX <-> 50 VIFs

• (1 - 30) VIF -> 1 DX GW

• 1 DX GW <-> 10 VGWs (VPCs)

\=> 50 X 1 X 10 = 500 VPCs

DX GW + TGW

• 1 Transit VIF <-> 1 DX

• 3 TGWs <-> 1 DX GW

• Transit VIF does not allow for hosted conn < 1Gpbs

DX SiteLink

• Enable for Private VIF or Transit VIF

• Support any combination of a dedicated or hosted DX with different port speeds

• The shortest path for traffic sent over AWS Global network

• Cost $0.5/hr + Data transfer cost

Routing Policies & BGP Communities

Public VIF

Public ASN:

• Active - Active: CGW advertises the same prefix

• Active - Passive: Same prefix and using AS_PATH and increasing local-pref

Private ASN:

• Active - Active: Not support

• Active - Passive: Using prefixes

BGP communities: control scope for the advertisement of prefixes (regional & global)

• Inbound: 7224:9100 (local), 7224:9200 (regions for continent), 7224:9300 (Global)

• Outbound: 7224:8100 (region), 7224:8200(continent),no_tag (Global), NO_EXPORT

Private VIF

Prefixes -> Physical distance -> local preference BGP -> AS_PATH

Local Preferences: 7224:7100 (low preference), 7224:7200 (medium preference), 7224:7300 (high preference)

Active - Passive: using local preference

Routing precedence:

• Local route

• Longest prefix match

• static route table over dynamic/propagated routes

• Dynamic routes:

  • DX BGP routes: shortest AS_PATH or load balance

  • VPN static routes

  • BGP from VPN: shorted AS_PATH

Link Aggregation Group (LAG)

Increase speed & failover by summing up multiple DX in a single logical conn using Link Aggregation Control Protocol (LACP) in Active/Active mode.

All the conns in LAG must have the same bandwidth and can have up to 4 aggregations. Allows to add new or use existing ones for setting up LAG.

Allow 4 aggregations for bandwidth 1 or 10 Gbps and 2 aggregations for 100Gpbs.

Num of operational conn: num LAG - num Oper = minimum up conn

Resilient DX Conn

• Single DX + VPN backup

• Dual DX + Dual devices

• Dual DX + Dual locations (High resiliency)

• Dual location + DX backup (Maximum resiliency)

DX Failover (Bi-directional forwarding detection)

Detection < 1s

The 90s for waiting for 3 keep-alive to fail

liveness detection 300ms and 3 -> failover under 1 second