Cloud Networking Architecture: VPC Topologies, Transit Gateways, BGP Peering, and Egress Optimization

Modern enterprise cloud architectures span multiple Virtual Private Clouds (VPCs), cloud accounts, geographic regions, and on-premises data centers. Poorly designed networking introduces routing loops, high cross-AZ transit costs, security perimeter vulnerabilities, and packet fragmentation.

This guide provides an architectural blueprint for enterprise cloud networking: Hub-and-Spoke Transit Gateways, BGP Autonomous Systems, PrivateLink Endpoints, and Jumbo Frame MTU Optimization.

1. Quick-Reference: Network Topology Models

+-----------------------------------------------------------------------------------------------------------------------+
|                                           CLOUD VPC TOPOLOGY COMPARISON                                               |
+-----------------------------------------------------------------------------------------------------------------------+
| Topology               | Routing Complexity                     | Peering Scaling            | Central Inspection     |
+------------------------+----------------------------------------+----------------------------+------------------------+
| Full-Mesh VPC Peering  | High ($O(N^2)$ direct peering links)   | Unmanageable beyond 10 VPCs| Impossible             |
| Hub-and-Spoke TGW      | Low ($O(N)$ links to central gateway)  | Scales to 5,000+ VPCs      | Native (Firewall Hub)  |
| Software Defined (SDN) | Centralized overlay controller (Andromeda)| Zero explicit peering      | Distributed Policy     |
+-----------------------------------------------------------------------------------------------------------------------+

2. Hub-and-Spoke Transit Gateway Topology

                  +-----------------------------------+
                  |   Central Inspection Hub VPC      |
                  |  (Palo Alto / AWS Network Firewall|
                  +-----------------+-----------------+
                                    |
                                    v
+-------------------------------------------------------------------+
|               Cloud Transit Gateway (TGW / NCC)                  |
+-------------+---------------------+-----------------+-------------+
              |                     |                 |
              v                     v                 v
     +-----------------+   +-----------------+   +-----------------+
     | Spoke 1: Prod   |   | Spoke 2: Staging|   | On-Premises via |
     | Workload VPC    |   | Workload VPC    |   | Direct Connect  |
     +-----------------+   +-----------------+   +-----------------+

Spoke Isolation: Spoke VPCs cannot communicate directly with each other without traversing the Central Inspection Hub.
Route Tables: Separate transit route tables enforce production and non-production network segmentation.

3. BGP Peering & Equal-Cost Multi-Path (ECMP)

Connecting on-premises enterprise data centers to hyperscalers requires Border Gateway Protocol (BGP, RFC 4271) over redundant dedicated interconnect circuits.

ECMP Load Balancing

By advertising identical IP prefix routes over multiple BGP sessions, routers utilize Equal-Cost Multi-Path (ECMP) hashing to distribute packet streams across multiple 100Gbps physical links, doubling aggregate bandwidth and providing sub-second failover upon fiber cuts.

References

Rekhter, Y., Li, T., & Hares, S. (2006). A Border Gateway Protocol 4 (BGP-4). RFC 4271.
Vahdat, A., et al. (2015). Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google's Datacenter Network. ACM SIGCOMM.
Amazon Web Services. (2024). AWS Transit Gateway Architecture and Best Practices Guide.