November 9, 2022

VPC separation in AWS with Terraform

Creating two separate VPC's that can communicate with each other.

VPC separation in AWS with Terraform

A foreword

It has been a while since my last post. And I'm glad to say that I'm back :) The last few years were a bit crazy. As everyone probably knows we had some weird time with a pandemic and such, social distancing, a war going on in Ukraine. During that time I myself had some struggles of my own. These things combined have had it's effects. but I'm happy to say that we're back on the right track and very glad that I'm typing again to share my experiences in the Dev(Ops) world with you again! That being said, let's get on with some IT stuff!


AWS

Everyone is probably somewhat familiair with the cloud by now. Be it Google, Amazon, Microsoft or other parties. We know we can get resources from these companies with just a click.

I have been diving into AWS the past few months and setting up an EKS cluster inside it's own VPC and services needed like database, elasticsearch and redis, container registry. The separation of these things in different VPC's comes from a security and maintainability point of view.
Though the subjects are at some parts new for me, I've gotta say I like AWS and I love the way I can do this with Terraform. And I think our setup here has a good start!

In this article I'll describe the setup for two VPC's and how to add communication for VPC1 to VPC2. Creating an EKS is outside of the scope of this article and I'll create a different post for that later on.

So let's see how we can create two VPC's, one with a NAT gateway and the other without. And then add communication between those two VPC's

VPC with NAT

Amazon's VPC (Virtual Private Cloud) is basically a virtual network with a fancy name in which you place your resources. You configure a CIDR ranges for public, private and intra subnets. You then decide wether or not you want internet access.

Our first network will be created to contain our EKS cluster or other services that will eventually require internet access. This network will have a NAT gateway and the network will have an IPv4 CIDR range or of 10.0.0.0/16.
We assign 3 availability zones eu-west-1a, eu-west-1b and eu-west-1c. Then we define the private, public and intra subnet ranges.
For private:

  • 10.0.1.0/24
  • 10.0.2.0/24
  • 10.0.3.0/24

Public:

  • 10.0.4.0/24
  • 10.0.5.0/24
  • 10.0.6.0/24

Intra:

  • 10.0.7.0/24
  • 10.0.8.0/24
  • 10.0.9.0/24

Now we've divided our subnets up into 9 CIDR ranges.

In terraform this will look something like this:

module "vpc-1" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 3.0"

  name = "VPC1"
  cidr = "10.0.0.0/16"

  azs = [
    "eu-west-1a",
    "eu-west-1b",
    "eu-west-1c"
  ]

  private_subnets = [
    "10.0.1.0/24",
    "10.0.2.0/24",
    "10.0.3.0/24"
  ]

  public_subnets = [
    "10.0.4.0/24",
    "10.0.5.0/24",
    "10.0.6.0/24"
  ]

  intra_subnets = [
    "10.0.7.0/28",
    "10.0.7.16/28",
    "10.0.7.32/28"
  ]

  enable_nat_gateway   = true
  single_nat_gateway   = true
  enable_dns_hostnames = true
}
VPC1

VPC 2 without NAT

The second VPC will only be for our services like a database and other private resources. So no one needs access to this from the outside.

We basically do the same, only we set the CIDR for the second VPC to 10.1.0.0/16 so the first and second VPC our outside of each others range. And we only define the private subnet ranges. These will be:

  • 10.1.1.0/24
  • 10.1.2.0/24
  • 10.1.3.0/24

And the terraform definition looks something like:

module "vpc-2" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 3.0"

  name = "VPC2"

  cidr = "10.1.0.0/16"

  enable_dns_hostnames = true

  azs = [
    "eu-west-1a",
    "eu-west-1b",
    "eu-west-1c"
  ]

  private_subnets = [
    "10.1.1.0/24",
    "10.1.2.0/24",
    "10.1.3.0/24"
  ]
}
VPC2

No communication between the two

So now we have created two virtual networks, great! However they cannot communicate with one another. Less great... Because when we put a server inside the first and the database in the second it would be nice if the application on the server could use the database.

What we need is to setup a peering connection between the two. So that when an application on the server in VPC1 tries to access a resource with an ip somewhere in the range of 10.1.0.0/16, it knows it needs to go to VPC2 and has access to it.

Peering connection and route tables

We're going to create a peering connection and add the routes on the route tables. If you have applied the above two modules in your amazon account, you can head over to the Amazon console online and see the two VPC's. And each VPC will have the created subnets. Each subnet will have a route table. Here you can see that it has defined it's own local communication. For the route tables on the subnets of the first VPC it will be:

Destination: 10.0.0.0/16
Target: local

For the second VPC subnet route tables

Destination: 10.1.0.0/16
Target: local

So you see the local communication routes are defined. When we add the peering connection in terraform we then add the route definitions from VPC1 to VPC2 for the private subnets. These are the only subnets that need to communicate with each other.

The peering connection in terraform:

resource "aws_vpc_peering_connection" "vpc1-to-vpc2" {
  vpc_id = module.vpc-1.vpc_id
  peer_vpc_id = module.vpc-2.vpc_id

  accepter {
    allow_remote_vpc_dns_resolution = true
  }

  requester {
    allow_remote_vpc_dns_resolution = true
  }

  auto_accept = true
}
Peering connection

The DNS resolution parts are there to allow hostname resolving for the application to the services.

Now you can see the peering connection in the AWS console once applied.

Next we add the route table definition for vpc1 to the peering connection.
We add them for the private subnet route tables. We loop over the the array with private route table id's. This is what the count item does, it gives us the route table id's and for every route table found, it adds this route definition.

We refer to the already create items with Terraform, so we don't need manually place all the ip ranges and id's of items. (Don't you just love IaC with Terraform?)

The first route resource:

resource "aws_route" "vpc1_private_to_peering_connection" {
  count = "${length(module.vpc-1.private_route_table_ids)}"
  
  route_table_id = "${module.vpc-1.private_route_table_ids[count.index]}"
  destination_cidr_block = module.vpc-2.vpc_cidr_block
  vpc_peering_connection_id = aws_vpc_peering_connection.vpc1-to-vpc2.id
}
Route definitions from vpc1 private subnets to peering connection

Next we create the second route resource. VPC2 -> VPC1

resource "aws_route" "vpc2_private_to_peering_connection" {
  count = "${length(module.vpc-2.private_route_table_ids)}"
  
  route_table_id = "${module.vpc-2.private_route_table_ids[count.index]}"
  destination_cidr_block = module.vpc-1.vpc_cidr_block
  vpc_peering_connection_id = aws_vpc_peering_connection.vpc1-to-vpc2.id
}
Route definitions from vpc2 private subnets to peering connection

Once applied check in the AWS console and you should see in the route tables of the private subnets the route added for the other network range to the peering connnection.

Now you have everything setup for your applications in VPC1 to communicate with the resources you need in VPC2. And the non-public resources will not be accessible from the outside.

Conclusion and afterthoughts

I really love the possibilities the cloud gives us. AWS makes things easy. However I don't like the AWS console online. It is a lot you'll see on your screen, too much imho. That is why I prefer things like Terraform. It can divide up my infrastructure in code files and keep things maintainable and unclutterd.

Networking is still a bit uncharted area for me. I understand the basic concepts and I'm pretty sure there are things to improve on what I've written here and how we've done things. When we improve upon above setup I'll add another post ;)

Thank you for taking the time to read this, and if you have thoughts or idea's you want to discuss you can always hit me up on github!  I've added the above terraform definitions in one gist:

Two separated AWS VPC’s and communication between the private subnets
Two separated AWS VPC’s and communication between the private subnets - vpc-and-peering.tf