Enabling IPv6 on Resources and VPCs

[yandy-r]

Hi,

I'm just starting out to play with the SDK, please forgive if I a missing something. First off, I'd like to probably point out how great this product is and I see a bright future for it. It's in the early stages and I understand things will be missing. This is not a complaint, just something I noticed while attempting to create an IPv6 enabled app.

It seems like the ability to map an amazonProvidedIpv6CidrBlock is missing within the @aws-cdk/aws-ec2 VpcNetwork construct as well as within the SubnetConfiguration interface.

Something along the lines of the below would be helpful.

import * as ec2 from "@aws-cdk/aws-ec2";

new ec2.VpcNetwork(this, name, {
// other attributes
  amazonProvidedIpv6CidrBlock: true,
// other attributes
}

Other values left out on purpose just to keep this concise.

From the subnetConfiguration[] it would be helpful to have.

import * as ec2 from "@aws-cdk/aws-ec2";

new ec2.VpcNetwork(this, name, {
// other attributes
  subnetConfiguration: [
    {
      name: "pub",
      cidrMask: 22,
      ipv6Cidr: true,
      mapIPv6OnLaunch: true,
      subnetType: ec2.SubnetType.Public
    }
  ]
// other attributes
}

The ipv6Cidr could be a boolean since all subnet cidrs must be /64 and then the obvious addition to map IPv6 address to instances within those subnet .

To get the amazonProvidedIpv6CidrBlock within the created VpcNetwork I had to do something like this, and in testing, it seems to work. Though going through the documentation, it's advised to use native constructs, which I'd agree completely.

this.vpc = new ec2.VpcNetwork(this, `${name}Vpc`, args);
new ec2.cloudformation.VPCCidrBlockResource(
  this,
  `CidrRes`,
  {
    vpcId: this.vpc.vpcId,
    amazonProvidedIpv6CidrBlock: true
  }
);

[rix0rrr]

We'll definitely take this under consideration. Thanks for the suggestion!

[FlorinAsavoaie]

Well, it is really complicated to have real IPv6 support for now with CDK. Even if you add the CfnVPCCidrBlock, you cannot configure Subnets properly. And even if you create an aspect to alter the Subnets somehow. you still need to write the logic for assigning cidrs to subnets. And even if you do that, it might not be compatible with how it will be implemented in CDK in the future so when it will be implemented all your code is gonna regenerate the ipv6 vpc stuff...

Are there any plans to implement this?

[AlexChesters]

Is there any plans to support this? As far as I can tell this means it's more or less impossible to have IPv6 support using the AWS CDK today.

[TrueBrain]

After a bit of fiddling, I only needed two "hacks" to get IPv6 working via CDK. If those alone could be address, that would be very helpful. What I did:

• Add a CfnVPCCidrBlock with amazon_provided_ipv6_cidr_block set
• Walk the node.children of the VPC, to find the CfnInternetGateway
• For every public subnet:
• • Create a new default route for "::/0" to the above found CfnInternetGateway
• • Walk the node.children of the subnet to find CfnSubnet, and assign a Ipv6CidrBlock to it

That alone enables IPv6 on the VPC. For the ALB, as that was my goal, I also had to add a connection for IPv6 on the ports the ALB is listening on.

Assigning Ipv6CidrBlock per subnet I managed to do with the Fn::Cidr.

I did not add IPv6 to the private subnets, as I have no need for that. But doing this is now trivial; it just requires an extra EgressOnly gateway and a similar solution as to the public subnets.

In general, it feels that adding IPv6 support is not that far away; I wish I was a bit more into TypeScripting to give it a go, but alas, I am not. But hopefully this gives a bit of inspiration to someone who is :)

The hacks I used

• The first "walk the children" is because the igw variable is not stored in a place that is otherwise still reachable.
  https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/aws-ec2/lib/vpc.ts#L1048
• The second "walk the children" is because Subnet doesn't allow setting Ipv6CidrBlock at any point.
  https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/aws-ec2/lib/vpc.ts#L1236

The code I used

        self._vpc = Vpc(self, "Vpc",
            max_azs=2,
            nat_gateway_provider=nat_provider,
        )

        # IPv6 is currently not supported by CDK.
        # This is done manually now, based on:
        # https://gist.github.com/milesjordan/d86942718f8d4dc20f9f331913e7367a

        ipv6_block = CfnVPCCidrBlock(self, "Ipv6",
            vpc_id=self._vpc.vpc_id,
            amazon_provided_ipv6_cidr_block=True,
        )

        # We need to sniff out the InternetGateway the VPC is using, as we
        # need to assign this for IPv6 routing too.
        for child in self._vpc.node.children:
            if isinstance(child, CfnInternetGateway):
                internet_gateway = child
                break
        else:
            raise Exception("Couldn't find the InternetGateway of the VPC")

        for index, subnet in enumerate(self._vpc.public_subnets):
            subnet.add_route("DefaultIpv6Route",
                router_id=internet_gateway.ref,
                router_type=RouterType.GATEWAY,
                destination_ipv6_cidr_block="::/0",
            )

            # This is of course not the best way to do this, but it seems CDK
            # currently allows no other way to set the IPv6 CIDR on subnets.
            assert isinstance(subnet.node.children[0], CfnSubnet)
            # As IPv6 are allocated on provisioning, we need to use "Fn::Cidr"
            # to get a subnet out of it.
            subnet.node.children[0].ipv6_cidr_block = Fn.select(
                index,
                Fn.cidr(
                    Fn.select(
                        0,
                        self._vpc.vpc_ipv6_cidr_blocks
                    ),
                    len(self._vpc.public_subnets),
                    "64"
                )
            )
            # Make sure the dependencies are correct, otherwise we might be
            # creating a subnet before IPv6 is added.
            subnet.node.add_dependency(ipv6_block)

And for the ALB:

        http_listener = ApplicationListener(self, "Listener-Http",
            load_balancer=alb,
            port=80,
            protocol=ApplicationProtocol.HTTP,
        )

        http_listener.connections.allow_default_port_from(
            other=Peer.any_ipv6(),
            description="Allow from anyone on port 80",
        )

[misterjoshua]

I followed @TrueBrain's example and got it to work in Typescript for myself. Here's my code:

// Create a VPC
const vpc = new ec2.Vpc(this, "Vpc", {
  cidr: "10.0.0.0/16",
  subnetConfiguration: [
    {
      cidrMask: 24,
      name: "public",
      subnetType: ec2.SubnetType.PUBLIC,
    },
  ],
});

// Add an Ipv6 workaround
new Ipv6Workaround(this, "Ipv6Workaround", {
  vpc: vpc,
});

// ipv6-workaround.ts
import * as cdk from "@aws-cdk/core";
import * as ec2 from "@aws-cdk/aws-ec2";

/**
 * Gets a value or throws an exception.
 *
 * @param value A value, possibly undefined
 * @param err The error to throw if `value` is undefined.
 */
const valueOrDie = <T, C extends T = T>(
  value: T | undefined,
  err: Error,
): C => {
  if (value === undefined) throw err;
  return value as C;
};

export interface Ipv6WorkaroundProps {
  vpc: ec2.Vpc;
}

/**
 * Adds IPv6 support to a VPC by modifying the CfnSubnets.
 *
 * For example:
 * ```
 * const vpc = new Vpc(this, "MyVpc", { ... });
 * new Ipv6Workaround(this, "Ipv6Workaround", {
 *   vpc: vpc,
 * });
 * ```
 */
export class Ipv6Workaround extends cdk.Construct {
  constructor(scope: cdk.Construct, id: string, props: Ipv6WorkaroundProps) {
    super(scope, id);

    const { vpc } = props;

    // Associate an IPv6 block with the VPC.
    // Note: You're may get an error like, "The network 'your vpc id' has met
    // its maximum number of allowed CIDRs" if you cause this
    // `AWS::EC2::VPCCidrBlock` ever to be recreated.
    const ipv6Cidr = new ec2.CfnVPCCidrBlock(this, "Ipv6Cidr", {
      vpcId: vpc.vpcId,
      amazonProvidedIpv6CidrBlock: true,
    });

    // Get the vpc's internet gateway so we can create default routes for the
    // public subnets.
    const internetGateway = valueOrDie<cdk.IConstruct, ec2.CfnInternetGateway>(
      vpc.node.children.find(c => c instanceof ec2.CfnInternetGateway),
      new Error("Couldn't find an internet gateway"),
    );

    // Modify each public subnet so that it has both a public route and an ipv6
    // CIDR.
    vpc.publicSubnets.forEach((subnet, idx) => {
      // Add a default ipv6 route to the subnet's route table.
      const unboxedSubnet = subnet as ec2.Subnet;
      unboxedSubnet.addRoute("IPv6Default", {
        routerId: internetGateway.ref,
        routerType: ec2.RouterType.GATEWAY,
        destinationIpv6CidrBlock: "::/0",
      });

      // Find a CfnSubnet (raw cloudformation resources) child to the public
      // subnet nodes.
      const cfnSubnet = valueOrDie<cdk.IConstruct, ec2.CfnSubnet>(
        subnet.node.children.find(c => c instanceof ec2.CfnSubnet),
        new Error("Couldn't find a CfnSubnet"),
      );

      // Use the intrinsic Fn::Cidr CloudFormation function on the VPC's
      // first IPv6 block to determine ipv6 /64 cidrs for each subnet as
      // a function of the public subnet's index.
      const vpcCidrBlock = cdk.Fn.select(0, vpc.vpcIpv6CidrBlocks);
      const ipv6Cidrs = cdk.Fn.cidr(
        vpcCidrBlock,
        vpc.publicSubnets.length,
        "64",
      );
      cfnSubnet.ipv6CidrBlock = cdk.Fn.select(idx, ipv6Cidrs);

      // The subnet depends on the ipv6 cidr being allocated.
      cfnSubnet.addDependsOn(ipv6Cidr);
    });
  }
}

[t0gre]

Following the hacks of @TrueBrain and @misterjoshua I got the vpc and subnets to have ipv6 setup. However, ec2s launched into these subnets still don't get ipv6 addresses. It looks like https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/aws-ec2/lib/vpc.ts#L1234-L1237 doesn't have any attribute that could be used to fix this. It looks like this could be fixed with a custom resource, as per https://stackoverflow.com/questions/42047071/auto-assign-ipv6-address-via-aws-and-cloudformation . I'm wondering if I'm missing something? What's the value of a subnet with ipv6 cidrs if ec2 launched into that subnet don't get ipv6 addresses? Auto-assignment should be included as part of the proposal?

[t0gre]

In the hope of helping others with the same conundrum, here's some python cdk to implement the auto-assign-ipv6-address to all public subnets

from itertools import starmap

       lambda_role = iam.Role(self, "ipv6-auto-lambda-role",
                        assumed_by = iam.ServicePrincipal("lambda.amazonaws.com"),
                        inline_policies = {
                            "ipv6-fix-logs" : iam.PolicyDocument(
                                statements = [iam.PolicyStatement(
                                    effect = iam.Effect.ALLOW,
                                    actions = [ "logs:CreateLogGroup",
                                                "logs:CreateLogStream",
                                                "logs:PutLogEvents"
                                            ],
                                    resources = ["arn:aws:logs:*:*:*"]
                                )]
                            ),
                            "ipv6-fix-modify" : iam.PolicyDocument(
                                statements = [iam.PolicyStatement(
                                    effect = iam.Effect.ALLOW,
                                    actions = [ "ec2:ModifySubnetAttribute"],
                                    resources = ["*"]
                                )]
                            )
                        }
        )

        lambda_function = aws_lambda.Function(self, "ipv6-auto-true-lambda",
                                handler = "index.handler",
                                code = aws_lambda.Code.from_inline(
                                    f"""
import cfnresponse
import boto3
import json

def handler(event, context):

    print(json.dumps(event))

    if event['RequestType'] is 'Delete':
        cfnresponse.send(event, context, cfnresponse.SUCCESS)
    else:
        try: 
            responseValue = event['ResourceProperties']['SubnetId']
            ec2 = boto3.client('ec2', region_name='{core.Stack.of(self).region}')
            ec2.modify_subnet_attribute(AssignIpv6AddressOnCreation={{
                                        'Value': True
                                    }},
                                    SubnetId=responseValue)
            responseData = {{}}
            responseData['SubnetId'] = responseValue
            cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData, "CustomResourcePhysicalID")
        except:
            cfnresponse.send(event, context, cfnresponse.FAILED, responseData = "An error occured")
                            """
                                ),
                                runtime = aws_lambda.Runtime.PYTHON_3_7,
                                role = lambda_role

        )


        def auto_assign_ipv6(index, subnet):
            subnet_id = subnet.subnet_id
            return cfn.CustomResource(  self, f"ipv6-auto-truer-{index}",
                                        provider = cfn.CustomResourceProvider.from_lambda(lambda_function),
                                        properties = {"SubnetId" : subnet_id}
        )

        list(starmap(auto_assign_ipv6, enumerate(vpc.public_subnets)))

[rix0rrr]

Should probably be tackled as part of #5927

[NukaCody]

Out of pure curiosity, I've got both the associated IPv6 addresses and enabling auto-assign IPv6 on each public subnet working with typescript thanks to the solutions above.

This is 100% "Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should." But none the less, an awesome exercise working with escape hatches, tokens, new typescript operators, and custom resources...

export class UtpoiaStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Create our 2-Tier Network Stack with IPv4
    const vpc = new ec2.Vpc(this, 'DualStackVPC', {
      cidr: '10.0.0.0/16',
      maxAzs: 2,
      enableDnsHostnames: true,
      enableDnsSupport: true,
      // subnetConfiguration: [],  // interface ec2.SubnetConfiguration is limited (Great for Ipv4 though)
    })

    // Associate an IPv6 CIDR block to our VPC
    const ipv6Block = new ec2.CfnVPCCidrBlock(this, 'IPv6Block', {
      amazonProvidedIpv6CidrBlock: true,
      vpcId: vpc.vpcId
    })

    // Using escape hatches to assign an Ipv6 address to every subnet as well as a custom resource that enables auto-assigned Ipv6 addresses
    vpc.publicSubnets.forEach((subnet: ec2.ISubnet, idx: number) => {
      const unboxedSubnet = subnet as ec2.Subnet
      unboxedSubnet.addRoute("IPv6Default", {
        routerId: (vpc.node.children.find(c => c instanceof ec2.CfnInternetGateway) as ec2.CfnInternetGateway)?.ref,
        routerType: ec2.RouterType.GATEWAY,
        destinationIpv6CidrBlock: "::/0"
      })

      const vpcCidrBlock = cdk.Fn.select(0, vpc.vpcIpv6CidrBlocks);
      const ipv6Cidrs = cdk.Fn.cidr(
        vpcCidrBlock,
        vpc.publicSubnets.length,
        "64"
      )
      let cfnSubnet = subnet.node.children.find(c => c instanceof ec2.CfnSubnet) as ec2.CfnSubnet ?? new Error("Why am I still doing this?");
      cfnSubnet.ipv6CidrBlock = cdk.Fn.select(idx, ipv6Cidrs)
      cfnSubnet.addDependsOn(ipv6Block)

      // Define a custom resource to auto-assign IPv6 addresses to all of our subnets
      const autoAssignCR = new cr.AwsCustomResource(this, `AutoAssignIPv6CustomResource${Math.random()*100}`, {
        policy: cr.AwsCustomResourcePolicy.fromSdkCalls({resources: cr.AwsCustomResourcePolicy.ANY_RESOURCE }),
        onCreate: {
          physicalResourceId: cr.PhysicalResourceId.of(`AutoAssignIPv6Create${Math.random()*100}`),
          service: 'EC2',
          action: 'modifySubnetAttribute',
          parameters: {
            AssignIpv6AddressOnCreation: { Value: true },
            SubnetId: subnet.subnetId
          }
        }
      })
    })

Again, this is more of a fun hack than a solution. If you need IPv6 addressing, I would use L1 constructs until this issue and #5927 are resolved.

[sssd-dev]

Just a small modification in @misterjoshua approach to include private subnets for IPv6 -

import * as ec2 from "@aws-cdk/aws-ec2";

/**
 * Gets a value or throws an exception.
 *
 * @param value A value, possibly undefined
 * @param err The error to throw if `value` is undefined.
 */
const valueOrDie = <T, C extends T = T>(
  value: T | undefined,
  err: Error,
): C => {
  if (value === undefined) throw err;
  return value as C;
};

export interface Ipv6WorkaroundProps {
  vpc: ec2.Vpc;
}

/**
 * Adds IPv6 support to a VPC by modifying the CfnSubnets.
 *
 * For example:
 * ```
 * const vpc = new Vpc(this, "MyVpc", { ... });
 * new Ipv6Workaround(this, "Ipv6Workaround", {
 *   vpc: vpc,
 * });
 * ```
 */
export class Ipv6Workaround extends cdk.Construct {
  constructor(scope: cdk.Construct, id: string, props: Ipv6WorkaroundProps) {
    super(scope, id);

    const { vpc } = props;

    // Associate an IPv6 block with the VPC.
    // Note: You're may get an error like, "The network 'your vpc id' has met
    // its maximum number of allowed CIDRs" if you cause this
    // `AWS::EC2::VPCCidrBlock` ever to be recreated.
    const ipv6Cidr = new ec2.CfnVPCCidrBlock(this, "Ipv6Cidr", {
      vpcId: vpc.vpcId,
      amazonProvidedIpv6CidrBlock: true,
    });

    // Get the vpc's internet gateway so we can create default routes for the
    // public subnets.
    const internetGateway = valueOrDie<cdk.IConstruct, ec2.CfnInternetGateway>(
      vpc.node.children.find(c => c instanceof ec2.CfnInternetGateway),
      new Error("Couldn't find an internet gateway"),
    );

    // Modify each public subnet so that it has both a public route and an ipv6
    // CIDR.
    vpc.publicSubnets.forEach((subnet, idx) => {
      // Add a default ipv6 route to the subnet's route table.
      const unboxedSubnet = subnet as ec2.Subnet;
      unboxedSubnet.addRoute("IPv6Default", {
        routerId: internetGateway.ref,
        routerType: ec2.RouterType.GATEWAY,
        destinationIpv6CidrBlock: "::/0",
      });

      // Find a CfnSubnet (raw cloudformation resources) child to the public
      // subnet nodes.
      const cfnSubnet = valueOrDie<cdk.IConstruct, ec2.CfnSubnet>(
        subnet.node.children.find(c => c instanceof ec2.CfnSubnet),
        new Error("Couldn't find a CfnSubnet"),
      );

      // Use the intrinsic Fn::Cidr CloudFormation function on the VPC's
      // first IPv6 block to determine ipv6 /64 cidrs for each subnet as
      // a function of the public subnet's index.
      const vpcCidrBlock = cdk.Fn.select(0, vpc.vpcIpv6CidrBlocks);
      const ipv6Cidrs = cdk.Fn.cidr(
        vpcCidrBlock,
        256,
        "64",
      );
      cfnSubnet.ipv6CidrBlock = cdk.Fn.select(idx, ipv6Cidrs);
      
      // The subnet depends on the ipv6 cidr being allocated.
      cfnSubnet.addDependsOn(ipv6Cidr);
    });

    // Modify each private subnet so that it has both a public route and an ipv6
    // CIDR.

    vpc.privateSubnets.forEach((subnet, idx) => {
      // Add a default ipv6 route to the subnet's route table.
      const unboxedSubnet = subnet as ec2.Subnet;
      unboxedSubnet.addRoute("IPv6Default", {
        routerId: internetGateway.ref,
        routerType: ec2.RouterType.GATEWAY,
        destinationIpv6CidrBlock: "::/0",
      });

      // Find a CfnSubnet (raw cloudformation resources) child to the public
      // subnet nodes.
      const cfnSubnet = valueOrDie<cdk.IConstruct, ec2.CfnSubnet>(
        subnet.node.children.find(c => c instanceof ec2.CfnSubnet),
        new Error("Couldn't find a CfnSubnet"),
      );

      // Use the intrinsic Fn::Cidr CloudFormation function on the VPC's
      // first IPv6 block to determine ipv6 /64 cidrs for each subnet as
      // a function of the private subnet's index.
      const vpcCidrBlock = cdk.Fn.select(0, vpc.vpcIpv6CidrBlocks);
      const ipv6Cidrs = cdk.Fn.cidr(
        vpcCidrBlock,
        256,
        "64",
      );
      cfnSubnet.ipv6CidrBlock = cdk.Fn.select(idx+3, ipv6Cidrs);

      // The subnet depends on the ipv6 cidr being allocated.
      cfnSubnet.addDependsOn(ipv6Cidr);
    });
  }
}```

[slavkv]

The same problem here. I will be great to have simple options in ec2.SubnetConfiguration to indicate if a subnet should have IPv6 prefix, and in ec2.Vpc to assign IPv6 prefix.

[CCludts]

Our team is also interested in IPv6 support, specifically for use with an ApplicationLoadBalancer.

To provide scope, Apple requires all new apps to support IPv6-only networks.

Starting June 1, 2016 all apps submitted to the App Store must support IPv6-only networking

https://developer.apple.com/news/?id=05042016a

[dbolotin]

This is pretty unexpected that IPv6 is just not supported in the CDKs core networking construct.

[ronnyaa]

Is this still an open issue?
Norway demand ipv6 on all services towards govermental customers by 1/1 2023.. and we would like a head start :)

[likeabbas]

If you are going to use the example that @sssd-dev provided to give your private subnet ipv6 addresses, you should probably set the routerType to ec2.RouterType.EGRESS_ONLY_INTERNET_GATEWAY to keep your private instances/tasks from having an exposable ipv6 address.

[rverma-dev]

Actually our whole usecase is to have exposable ipv6, to replace NAT with free of cost egress only internet gateway.

[nick-kang]

Will it be possible to prioritize this or get some transparency on implementation timelines given the announced pricing changes to IPv4 (https://aws.amazon.com/blogs/aws/new-aws-public-ipv4-address-charge-public-ip-insights/)?

We're looking at introducing IPv6 and see that there's a workaround with L1 constructs. But if this is going to be implemented in the next couple of months, we can hold off until L2 constructs are introduced.

[TrueBrain]

Although this ticket is more about dual-stack than IPv6-only, just as a word of warning (which I found out the hard way) for those that are thinking about IPv6-only deployments:

Be aware that deploying a full IPv6-only setup on AWS is still not possible. Only some of AWS' own endpoints support IPv6. For example, the EC2 endpoint is only IPv6-enabled in some regions, and for example the autoscaling endpoint is not available on IPv6 at all.

This makes the news that AWS will charge for IPv4 a bit ironic; they themselves don't even have their shit together to be IPv6-only, but they except their customers to embrace IPv6 more :) Not sure how that math adds up, but it is fine. Just be aware of these restrictions. This is especially important for this project, as some constructs have things like user-data which makes calls to these endpoints. If you use a VPC that only uses an egress-only gateway (and as such is IPv6-only), those constructs will not work successful. In other words: you still need a little bit of IPv4 somewhere (for example a NAT64 or something), if you want to use things like ASG lifecycle hooks, aws ec2 in scripts, etc etc.

[sgryphon]

The Vpc construct definitely needs better support for ### IPv6, especially now that AWS will be charging for public IPv4.

i.e. It should be as simple as (keeping the default IPv4 the same):

const vpc = new ec2.Vpc(this, 'TheVPC', {
  vpcProtocol: VpcProtocol.DUAL_STACK,
})

Here is some sample code I am using, with IPv6 support:
https://github.com/sgryphon/iot-demo-build/blob/main/aws/aws-landing/lib/network-layer.ts

Some key ideas:

• Add at least two IPv6 /56 blocks, one for public and one for private; reasoning is along the same lines as defaulting to 3 availability zones. Rather than fiddle with subnet mask sizes, trying to balance number of subnets vs number of addresses, IPv6 is much simpler -- each subnet is /64, and we can add as many /56 blocks as needed; and there will never be any address collision.
• Add DNS64 support and Egress-only gateways
• Also add DNS64 to dual stack networks by default, to support IPv6 only machines on those networks (it also makes IPv6 the preferred traffic)
• Public networks probably still need to be dual stack, for NAT64, with appropriate routing.
• Turn off automatic mapping of IPv4 addresses, on dual stack networks (especially as they will be charged for)

Private networks can probably be IPv6 only, for simpler subnet allocation, but for consistency the default should be dual stack. However creating a configuration with private IPv6 only should only need minimal config (with defaults based on the protocol), e.g.:

const vpc = new ec2.Vpc(this, 'VPC', {
  ipv6NumberOfBlocks: 3,
  vpcProtocol: VpcProtocol.DUAL_STACK,
  subnetConfiguration: [
    {
      cidrMask: 24,
      name: 'Public',
      subnetProtocol: ec2.SubnetProtocol.DUAL_STACK,
      subnetType: ec2.SubnetType.PUBLIC,
    },
    {
      name: 'Private',
      subnetProtocol: ec2.SubnetProtocol.IPV6,
      subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS,
    },
    {
      ipv6Block: 2,
      name: 'Isolated',
      subnetProtocol: ec2.SubnetProtocol.IPV6,
      subnetType: ec2.SubnetType.PRIVATE_ISOLATED,
    },
  ]
});

Reasons:

• The level 2 Vpc construct by default only has IPv4, without parameters for IPv6, and you need several modifications to the level 1 Cfn constructs for support.
• Modifications will vary between use, without guidance on best practice or standardisation.
• AWS will be charging for public IPv4, so having them enabled, without any easy default way to get IPv6, is not good.

Suggestion:

• Add a simple vpcProtocol parameter that can be used to create a network that includes reasonable IPv6 support with a single option.
• The default can remain at VpcProcotol.IPV4, so that it is backwards compatible (not change to a existing configuration)

Suggested VPC defaults:

• Add two IPv6 CIDR /56 blocks, defaulting to Amazon-provided. Adding two blocks allows one for public and one for private, and encourages good practice of adding more as needed. This is similar reasoning to adding three availability zones by default.
• Add an IPv6 Egress-only internet gateway for the VPC.
• Add tag 'aws-cdk:vpc-protocol' with the option value

Configuration properties to add to VpcProps:

• vpcProtocol: VpcProtocol, with an enum VpcProtocol DUAL_STACK and IPV4. Controls the settings on the VPC, as well as the default subnet settings.
• ipv6NumberOfBlocks: number -- default to 2 for dual stack; or 0 for IPv4; cannot conflict with protocol (i.e. muts be 0/default for IPv4)
• createIpv6EgressGateway: boolean -- default to false for IPv4 and true for dual stack.
• ipv6Addresses: IIpv6Addresses -- with VpcIpamv6Options, similar to IPv4, to supply specific IPv6 addresses from IPAM (instead of Amazon-provided ones)

The created subnets should default to the same protocol as the VPC, i.e. for IPv4 (default) there is no change.

Subnet configuration

Default for dual stack (per availablity zone) is basically:

const vpc = new ec2.Vpc(this, 'VPC', {
  vpcProtocol: VpcProtocol.DUAL_STACK,
  subnetConfiguration: [
    {
      ipv6Block: 0,
      mapPublicIpOnLaunch: false,
      name: 'Public',
      subnetProtocol: ec2.SubnetProtocol.DUAL_STACK,
      subnetType: ec2.SubnetType.PUBLIC,
    },
    {
      ipv6Block: 1,
      mapPublicIpOnLaunch: false,
      name: 'Private',
      subnetProtocol: ec2.SubnetProtocol.DUAL_STACK,
      subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS,
    },
  ]
});

SubnetProtocol can be either DUAL_STACK, IPV4, or IPV6.

For IPV4 and DUAL_STACK, the IPv4 CIDR is calculated as normal, e.g. split a /16 evenly above for the default 3 zones gives /19 subnets. IPV6 only subnets don't participate in the split.

For a Dual Stack network, configure the subnets with the following defaults:

• Assign a /64 subnet, based on the allocated ranges. Rather than a specific CIDR block, all subnets are /64 in size.
• For customisation, rather than setting the mask, when can select which block of IPv6 addresses to take from. For automatic creation, each block is just assigned sequentially.
• This means for the default 3 availability zones, the public subnets are index 0, 1, 2 in the first block and the private subnets are index 0, 1, 2 in the second block.
• Add a dependency from the subnet to the relevant IPv6 allocation (this is needed to change/destroy in the correct sequence)
• Auto-assign IPv6 on creation
• Enable private DNS names -- We can get the instance ID as an output from CloudFormation and use that to build the DNS name to reference the machine, without having to know the machine IP address.
• Enable DNS64 -- This allows IPv6 only machines to still have outbound NAT64 access to IPv4 only external addresses. It does mean dual stack machines will preference NAT64 (over NAT44).
  • For a flexible solution you will need configuration property to disable this, for rare situations where DNS64 breaks an application and the app/machine can't be configure as single stack IPv4 (e.g. multi-purpose machine).
• Add tag 'aws-cdk:subnet-protocol' with the option value

Also, change this setting (different from IPv4 only):

• Turn off map public IPv4 on launch -- Public IPv4 addresses are now charged, so we only want to add them (manually) when needed, rather than automatically.

Note that public networks that contain NAT gateways need to be dual stack, so that the gateway can be assigned an IPv4 address, even if it is only used for NAT64.

Public network route configuration:

• IPv6 default ::/0 needs to go to the internet gateway.
• NAT64 64:ff9b::/96 needs to route to a NAT gateway. If you have one gateway per public network (e.g. per availability zone), then use that; if the number of gateways is less, then they can be assigned round-robin (like for private subnets). Note that you may still want a NAT64 route, even if you don't have DNS64 enabled, as a server/device may have it's own DNS settings (e.g. us a public DNS64 server).
• IPv4 default 0.0.0.0 routed to the internet gateway (same as existing).

Private networks with egress route configuration:

• IPv6 default ::/0 needs to go to the egress only internet gateway.
• NAT64 64:ff9b::/96 needs to route to a NAT gateway..
• IPv4 default 0.0.0.0 is routed to a NAT gateway (same as existing). Both NAT64 and NAT44 can use the same gateway, either per availabilty zone or a round robin configuration.

Configuration properties to add/change to SubnetConfiguration:

• subnetProtocol: SubnetProtocol, with an enum SubnetProtocol DUAL_STACK, IPV6, and IPV4.
• ipv6Block: number - Which block to assign this configuration from. Default based on the type (public = 0, private/isolated = 1). Incompatible with IPv4 only.
• enableDns64: boolean - defaults to true for dual stack and IPv6 (false for IPv4 and setting true is incompatible).
• assignIpv6AddressOnCreation: boolean - default to true for dual stack and IPv6(false for IPv4 and setting true is incompatible).

The following properties are changed.

• mapPublicIpOnLaunch: default to false for dual stack and IPv6 (i.e. need to manually add when desired). Incompatible with IPv6 only.
• cidrMask: incompatible with IPv6 only.

Manually adding individual Subnet constructs:

The Subnet construct also need to be updated to support IPv6. Note that class Subnet is actually defined in the 'vpc.ts' file.

SubnetProps needs additional settings, similar to above:

• subnetProtocol: SubnetProtocol, with an enum SubnetProtocol DUAL_STACK, IPV6, and IPV4.
• ipv6Addresses: IIpv6Addresses - the manually assigned range.
• enableDns64: boolean - defaults to true for dual stack and IPv6 (false for IPv4 and setting true is incompatible).
• assignIpv6AddressOnCreation: boolean - default to true for dual stack and IPv6(false for IPv4 and setting true is incompatible).

Maybe some utility methods would be useful to extract out specific IPv6 subnets from specific blocks.

The addDefaultInternetRoute() needs to support IPv6 and add the correct routes based on subnetProtocol.

Similarly there should be an addDefaultNat64Route() method, as NAT64 may be added to public IPv6 networks as well (so that IPv6 only machines can access external IPv4 destinations).

An addIpv6EgressOnlyRoute() is also needed for the egress-only gateway (equivalent of NAT on an IPv4 private network).

[arianvp]

One /56 should be large enough to alllcate both the private and public subnets though?

Subnets are always /64 in size in Ipv6 VPC so the allocation strategy is just incrementing a counter.
Whether they are 'private' or 'public' is decided by what routes you attach.

Apart from that agree with everything you write!

If allocating /56s is cheap then it probably doesn't really matter though.

Also ideally subnets would be /56, vpcs /48 and each server would get a /64 . But I don't think amazon would ever do that :(

[urda]

Will it be possible to prioritize this or get some transparency on implementation timelines given the announced pricing changes to IPv4 (https://aws.amazon.com/blogs/aws/new-aws-public-ipv4-address-charge-public-ip-insights/)?

We're looking at introducing IPv6 and see that there's a workaround with L1 constructs. But if this is going to be implemented in the next couple of months, we can hold off until L2 constructs are introduced.

I echo this, looking to reduce costs when IPv4 address pricing comes into play. Can we get an update for support of this?

[mikhaelsantos]

Is this raising in priority?

[jbm00n]

This is getting critical due to IPv4 price change and the fact that we migrated to CDK after AWS recommendations. Is there any kind of estimate so that we can plan how to move forward?

[urda]

So I received another notice reminding me that this new charge is still on the way.

What's the status on this? Why is this not getting priority attention?

Amazonians and AWS employees: this is not a Customer Obsessed way to push this new cost without CDK controls. It is also disappointing to see a lack of Ownership and Bias for Action on this issue. Where's the Day 1 results? This is Day 2 behavior.

[evgenyka]

Hi, the issue is on the CDK team radar. The cost structure change announcement increases urgency. As you can see, we label it as a p1, but the effort estimate is large. Hence, we are seeking an internal re-prioritization of committed tasks to accommodate this.

[sgryphon]

One /56 should be large enough to alllcate both the private and public subnets though?

Depends on how many subnets you have... it allows 256 subnets, like restricting to 192.168.0.0/16 and having /24 subnets.

But no reason to constrain yourself like this. You can create /56s for the rest of your life and not run into a conflict (there are more /56s than the entire current internet addresses).

Like defaulting to 3x regions, I suggest defaulting to 2x /56 ranges, so people aren't afraid to use them (they do not need to be conserved like IPv4 addresses).

[sgryphon]

I have a blog entry up for the demo L1 code I posted above: https://sgryphon.gamertheory.net/2023/11/deploying-a-secure-lwm2m-ipv6-test-server-on-aws/

This shows creating a dual stack server.

One thing for the IPv6 public address is to create an independent network interface with a public address, and then attach to the machine (in addition to the base network interface).

This is like attaching a public IPv4 (EIP) and keeps the EC2 instance independent of the interface with the (static) IPv6.

It allows cloud formation to replace/rebuild the instance (which will get a new random IPv6) and then reattach the same static IPv6.

If you don't do this, you either get a constantly changing IPv6, or you get into a rebuild loop conflict where it tries to build the new machine with the same IPv6 address before destroying the old one and fails. (The separate interface allows the new machine to be built, with a random IPv6, the interface attachment MOVED, and then the old one destroyed).

It means that the EC2 instanced has multiple interfaces and multiple IPv6 addresses, but this is not uncommon for IPv6. With IPv4 you generally only have one address for a machine (even with a second public IPv4 it might be via NAT and not on the machine itself, and local link is only used if there is no other). In contrast, IPv6 will have multiple addresses - local link addresses, EUI64 address, addresses from DHCP, maybe with multiple prefixes (ULA and global), and for privacy will often use additional random addresses for outgoing connections.

e.g. my local machine has 1x IPv4 (private, which goes through NAT), and 7x IPv6.

Many IPv6s is normal.

[shorn]

For folks that are currently considering changing to IPv6 because of the new charges: be aware that few AWS services support IPv6.
Here's a list: https://awsipv6.neveragain.de/
As at comment time, this shows that 91% of AWS services do not support IPv6.

You will not be able to "just use IPv6" - if your machines need to talk to any of those services (and they very likely will) - you will need a NAT gateway to support NAT-64 functionality, or to run a dual-stack setup (in which case you'll still pay for the public IPv4 address).

Also be aware that many other non-AWS services do not yet support IPv6, e.g. Github, Docker (needs an IPv6 specific host in the image path). Also for those wanting to connect from Widows hosts, note that WSL2 does not support IPv6.

[sgryphon]

For folks that are currently considering changing to IPv6

I agree, it is never that simple :-)

For outgoing, you will definitely need the NAT gateway, enable DNS64, and configure a NAT64 route. The NAT gateway needs IPv4 enabled, but you don't need to buy the IPv4s). For most cases this will work for backwards compatibility access to IPv4 resources. e.g. github.com works fine via NAT64. (but there can be some broken things that will require dual stack).

For incoming, if you expose your services as IPv6 then a front end CDN/reverse proxy can do translation for you, e.g. I use CloudFlare to provide both IPv4 and IPv6 incoming addresses for my IPv6 only company website.

Things are also changing: originally Docker Hub needed NAT64, then there was a custom IPv6 host configuration, and very recently the IPv6 is now default: https://www.docker.com/blog/docker-hub-registry-ipv6-support-now-generally-available/

Similarly, WSL2 now has mirror networking with IPv6 in preview (and WSL1 already had IPv6 support):
https://devblogs.microsoft.com/commandline/windows-subsystem-for-linux-september-2023-update/

If you only have a small number of IPv4 addresses, it may not be worth the savings, but if you have a lot, then it makes sense to switch. In the real world IPv4 addresses are increasing in price, and moving to IPv6 can be big savings. The local Australian telco, Telstra, has moved their mobile business to IPv6-only, generating millions of dollars selling off the IPv4 addresses https://www.sidn.nl/en/news-and-blogs/australias-telstra-switches-mobile-users-to-ipv6-only

[juinquok]

Hi! Also wanted to raise an issue here with regards to modifying existing VPCs to support the IPV6 standard. We recently attempted to update the VPC in our staging environment to associate an IPV6 block and enable the required settings accordingly. However, it seems that CloudFormation is updating the resources within the VPC in the wrong order. It should associate the underlying IPV6 block before updating the DNS64 settings for the VPC.

Upon stack deployment, I received an error message:

Resource handler returned message: "Invalid value 'true' for assign-ipv6-address-on-creation. Cannot set assign-ipv6-address-on-creation to true unless the subnet (subnet-060f6163cef48ce32) has an IPv6 CIDR block associated with it. (Service: Ec2, Status Code: 400, Request ID: bd970613-e35d-46a8-9d21-7e31cedfa36d)" (RequestToken: 2f594433-5023-f6c2-563c-a3ed833e86d3, HandlerErrorCode: InvalidRequest)

This then led to a barrage of error on the Update Rollback operation where the system attempted to rollback all the changes made to the VPC which will of course result in an error since the IPV6 block is associated and there is no way to rollback without first setting the related settings to false.

Resource handler returned message: "The subnet IPv6 CIDR block with association ID subnet-cidr-assoc-0e5a1bf47f0125e74 may not be disassociated, because its subnet has enable-dns64 set to true. (Service: Ec2, Status Code: 400, Request ID: 1ea9ae7c-1407-4943-aeb0-1d18ef2a0c38)" (RequestToken: a982d7fc-6f4c-3bbf-b825-4c283edff516, HandlerErrorCode: InvalidRequest)

CloudFormation seems to be creating the resources in the wrong order and thus resulting in the failure to update the stacks in-place.

[scanlonp]

For some visibility into how we are planning to begin ipv6 configuration in vpc, see #28480. The description gives an overview of the default dual stack configurations.

Appreciate comments here or on the PR! We are following it closely.

Additionally, we will track asks that are not included in the first PR in related feature requests (for example among the countless excellent suggestions given by @sgryphon, we are currently opting to create a single IPv6 CIDR, but there is no reason creating separate CIDRs for private and public subnets could not be enabled in the future).

[juinquok]

Hi @scanlonp, was looking at #28480 is there also any plans on extending perhaps the subnet or ec2 construct to have the ability to add a static IPv6 address to an instance? Currently the workaround is to create an IPv6 IP address from the AWS provided VPC block and then attach this to a network interface and have the interface attach to the EC2 instance all done using L1 constructs.

[sgryphon]

ability to add a static IPv6 address to an instance

I don't think this is a subnet, or network, thing, but more an instance thing. There is a setting to auto assign IPv6, which is turned on, but those addresses will change all the time.

The approach you mention is exactly what I do. First I build the network stack, which has IPv6, including the auto-assign flag.

Then I build the EC instance, which simply references the network it is in. For internal communication between machines you can then output the instanceId, which you can use for internal DNS references (if privateDnsNameOptionsOnLaunch is enabled in the network).

For external access the auto-assigned public IP will change on rebuild (it's fine for outgoing access), and if you try to add a fixed address to the machine it causes dependency issues.

So, what I do is the same as you -- create a separate fixed address and assign it to a separate interface, then associate that interface to the instance. That way if the instance changes and needs rebuilding, the attachment will be removed, the instance recreated, and then attached again, without any changes needed to the separate interface by CloudFormation, so it keeps the same fixed address with no dependency sequence issues.

Example code at: https://github.com/sgryphon/iot-demo-build/blob/main/aws-leshan/aws-lwm2m-demo/lib/lwm2m-demo-server-stack.ts

Now, maybe this could be part of a layer 2 EC creation (is assigning an EIP (v4) part of that?), but not part of networking/VPCs.

But yeah, layer 2 ECS Instance needs to be updated with IPv6 support, including maybe the approach you outline (separate interface) for stable IPv6 addresses (that persist across a recreation of the instance).

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[evgenyka]

Closing this older issue in favor of #28480. If there are any remaining unresolved issues, please create a new one and use the implementation in #28480 as a reference point.