Possible incorrect statement for pods internet access in external-snat docs

[hendryanw]

Hi team,

The external snat documentation mentioned the following: (Quoted 1 point for brevity)

Due to this behavior:
- Your Pods can communicate with internet resources only if the node that they're running on has a public or elastic IP address assigned to it and is in a public subnet. A public subnet's associated route table has a route to an internet gateway. We recommend deploying nodes to private subnets, whenever possible.

However, in my experience this is not fully correct. I am running EKS 1.28 with VPC CNI plugin at v1.15.1. Correct me if I'm wrong:

Pod which is deployed on node running on private subnet (without public IP address) can still communicate with resources on the internet via NAT Gateway, even when AWS_VPC_K8S_CNI_EXTERNALSNAT is set to false (default). What happen is:

1. Pod private IP is translated to node primary private IP of primary ENI
2. The traffic is routed to NAT Gateway.
3. NAT Gateway will then translate the node primary private IP into NAT Gateway private IP, and then into NAT Gateway public IP routed via Internet Gateway

I believe, it is best to rephrase the statement in the doc into what AWS recommends in terms of AWS_VPC_K8S_CNI_EXTERNALSNAT configurations to enable outbound internet access for pods as per docs title.
For example:

1. AWS_VPC_K8S_CNI_EXTERNALSNAT is set to false by default to allows pods deployed on nodes running in public subnet to be able to access the internet. If the nodes are deployed on public subnets, the recommendations is to set AWS_VPC_K8S_CNI_EXTERNALSNAT to false.
2. If the nodes are deployed on private subnets with route to internet via NAT Gateway, the recommendations is to set AWS_VPC_K8S_CNI_EXTERNALSNAT to true.

Thank you.

[fincd-aws]

Hi, thanks!

Maybe I should split this page into two parts: SNAT for public subnets, SNAT for private subnets?

The main ideas of the page should be:

• "An internal IPV4 SNAT is on by default, so that pods on public nodes use the public IP of the node can reach the Internet"
• You can turn this off if you don't need it, b/c a NATGW or anything else is doing a SNAT for you.
• You should turn it off if you expect all pod traffic to leave through the pod's ENI, but... in public subnets that is gonna break, and is expected to.

[hendryanw]

Hi @fincd-aws , thank you! I agree with your suggestions.