Terraform Workspaces: Managing Multiple Environments Efficiently

We are going to learn about Terraform Workspaces and, most importantly, how to use them to manage multiple environments like Dev, QA, Staging, and Prod efficiently. Previously we have covered the topic - Terraform State File Management Tutorial

You can also Watch this tutorial demo on our YouTube Channel

The Problem: Traditional Environment Management

Usually, when we manage different environments, we will either create separate Git branches in a repository (like a Dev branch, QA, Prod, and so on) or we'll create separate directories like this.

So, the Dev environment will have all the Terraform resource files. The Prod environment will have similar Terraform resource files. Similarly, for other environments, I will copy the same resource file and keep it in a different directory. This is how we generally manage multiple environments, with each directory for each environment having these resource files.

For example, in this Dev directory, which is managing my Dev environment, if you see this main.tf, I have a resource section which is nothing but the creation of an EC2 server in the Dev environment. The same code, the same directory, I will copy to create another environment.

Let's say I want to have a different environment for Prod, then I'll copy this directory and rename the directory to prod, and I will make the changes accordingly. So here, in this Prod directory as well, I have a main.tf which also has the same code. Similarly, if you want to manage multiple environments, this is what we do.

Here is the actual problem: If I want to make a small change, like in the Dev environment I want my instance to be T2.micro, and I want all my environments' resource types to be the same. In this case, if I want to use T2.micro or T3.medium, I need to go and update each main.tf file in every directory, update the value, and then I need to apply it.

But, if you want to manage these environments smartly, where you can use the same resource file and apply it for different environments, this is exactly where Terraform Workspaces come in and make our life easier.

Introducing Terraform Workspaces

A Workspace in Terraform is like an isolated environment that will have its own state file, but it will share the same codebase. If you see this diagram, the Terraform codebase will be the same, but for each environment, we will create a workspace. Each workspace will have a dedicated state file for that particular workspace. Here, we can consider each workspace as an environment. This is the beauty of using Terraform Workspaces.

So, think of it like this: we want to create and manage multiple environments like Dev, Prod, QA. Instead of creating these kinds of different directories and copy-pasting the same code, I have created a single directory called workspace-demo, and I have copied the same code here only once.

Creating and Switching Workspaces

Now, we'll see how to create workspaces, how to switch between different workspaces, and how to apply configurations to them.

This is the directory I'm going to use. Let me first initialize Terraform. Even if you have not created any workspace, by default Terraform has created a workspace called default.

terraform workspace show

Here, by default, Terraform has created a default workspace for you. This is the one you have been using even if you don't know it.

Let's create a new workspace for our environments. How can we create it?

terraform workspace new dev

For example, dev. This is nothing but we are creating a workspace for the dev environment. Once this is created, this will be switched into the newly created workspace. How would you confirm it?

terraform workspace show

Now you are in dev.

Similarly, we'll create another workspace, prod.

terraform workspace new prod

Now you are switched into prod. Again, I will create one more, qa.

terraform workspace new qa

If I run terraform workspace show, this will show me the workspace I'm currently in.

If you want to get a list of workspaces available in your current directory, I can give:

terraform workspace list

So we have this many, and I'm in qa now.

Now, if I want to switch over between any workspaces, I can use terraform workspace select. I can switch to any workspace I want. For example, if I want to get into dev, I can use the workspace name:

terraform workspace select dev

Now I have switched into the dev workspace. You can use list or show to confirm this information.

So, this is how we switch between workspaces. Let's see how we apply the configuration for each workspace.

Applying Configuration Across Workspaces

Now, I have a similar Terraform configuration. I have a provider where I have mentioned my provider is AWS and the region is US East-1. Also, I have a Terraform resource configuration file main.tf. Here, I have defined my resource to create an EC2 instance. You can create multiple resources as you want. For this demo, I'll be using this one resource.

Let's say if you apply this configuration from dev, the server will be created with a name like "server one". This is what we generally do from the default workspace. But smartly, what we can do is dynamically update the value, just like how we use variables. Similarly, we can use the Terraform workspace dynamically. For example, here, in the prefix or suffix, you can define it like:

resource "aws_instance" "server1" {

  ami                         = "ami-0e86e20dae9224db8"

  instance_type               = "t2.micro"

  subnet_id                   = "subnet-0105b1aef1e7755cd"

  key_name                    = "demov2"

  ebs_block_device {

    device_name = "/dev/sda1"

    volume_size = 10

  }

  associate_public_ip_address = "true"

  vpc_security_group_ids      = ["sg-04dd813e22c5a0b2f"]

  tags = {

    Name = "server1-${terraform.workspace}"

  }

}

This is my code. So, my server name will be created like server-one-dev. If you want, you can put it in the prefix; it's up to you.

Let's apply this configuration. So right now, I am in dev.

terraform apply -auto-approve

The configuration is applied now, and we are in the dev workspace. So, this configuration is applied for the dev environment. Let me go to my EC2 console and refresh my page.

So, server-one-dev is created. We can think like these resources are dedicatedly created for the dev environment. Similarly, you can create multiple resources. As we have only one single resource, this is fine.

Now, if I want to apply the same code to multiple workspaces, I need to switch over into that particular workspace and then apply it. The point is, I'm using the same Terraform code for multiple environments.

Let's select qa now.

terraform workspace select qa

terraform apply -auto-approve

So the configuration is applied.

Again, I will switch over to the prod environment.

terraform workspace select prod

terraform apply -auto-auto-approve

Okay. Now, if you take this example, we used the same Terraform resource files. I have created multiple workspaces to manage my different environments. I switched over to each workspace and applied the same configuration. If you go to EC2 and refresh the page, now you see server-one-dev, server-one-qa, server-one-prod. So you have servers for each environment.

Understanding State File Management

The same code is used, but the resource gets created independently in each environment. How is it maintaining its state file for each workspace? Because we need a state file, but if you see here, we don't see a TF state file directly. Instead, Terraform has created a directory called .terraform/terraform.tfstate.d. If you expand this, you can see it created a directory for each workspace, and it has saved the state file within that directory. So, it is maintaining the TF state file dedicatedly for each workspace.

Dynamic Configuration with Workspaces

Okay, one more thing: we can dynamically update the values based on the environment's needs. For example, for a lower environment, I can use a small instance type. For a production environment, I can go with a little higher instance type. So, how can we achieve this, especially since we use a single resource configuration? We can use conditions. This topic is already covered in a separate video about conditions, so go through it if you need a refresher.

So, what I can do for the instance type:

resource "aws_instance" "server1" {

  ami                         = "ami-0e86e20dae9224db8"

  instance_type               = terraform.workspace == "prod" ? "t3.medium" : "t2.micro"

  subnet_id                   = "subnet-0105b1aef1e7755cd"

  key_name                    = "demov2"

  ebs_block_device {

    device_name = "/dev/sda1"

    volume_size = 10

  }

  associate_public_ip_address = "true"

  vpc_security_group_ids      = ["sg-04dd813e22c5a0b2f"]

  tags = {

    Name = "server1-${terraform.workspace}"

  }

}

This is how we generally define conditions. So, terraform.workspace == "prod" is the condition. If this condition is true (meaning the workspace is prod), then it will pick the first value (t3.medium). If it is false (meaning any other workspace like dev or qa), then it will pick up the second value (t2.micro) as the instance type.

Let me start from dev workspace.

terraform workspace select dev

terraform apply -auto-approve

Here, the condition is false because the workspace is dev. That is why T2.micro will be set. It is already set, so that is why it is refreshing.

Now I will switch over to qa and I will apply the configuration.

terraform workspace select qa

terraform apply -auto-approve

Okay. Now I will switch over to prod then apply the configuration.

terraform workspace select prod

terraform apply -auto-approve

This time, the condition will become true. terraform.workspace will be prod, so it will pick T3.medium as the type. Initially, the instance type was T2.micro (from a previous run), so now a change is detected, and it will apply the new change. That is why we see this "modifying" message.

The configuration is applied in all workspaces. So, let's go and refresh the EC2 page.

So here, if you see, for dev and qa, the instance type is T2.micro, but for production, it is T3.medium.

Deleting Workspaces

Terraform Workspaces help us manage multiple environments using the same set of code. Each workspace has its own isolated state file. We can dynamically use the workspace name to change tags, names, instance types, and other parameters.

Okay, let's say we want to delete a workspace. You can simply run:

terraform workspace delete [workspace_name]

But the condition here is: if you are already in the workspace, you cannot delete it because it is an active workspace.

So, let me switch over to qa.

terraform workspace select qa

From here, I will try to delete the prod workspace. This is what I want to show you: if you are planning to delete a workspace, the first thing you need to do is destroy all the resources within that workspace using terraform destroy. Only if you do a destroy and make it clean, then only you will be able to delete a workspace.

So first, I will destroy all the resources, then I will delete the workspace. As we have only one resource, it is deleting that one resource. If you have multiple resources, it will destroy them accordingly.

Now I will switch over to default.

terraform workspace select default

From here, I can delete all the workspaces. First, I need to switch over to another workspace. From here, I will destroy the resources. Then we'll use prod again. We'll destroy it. Now we'll switch over to dev. Destroy the resources.

terraform workspace list

From here, I will switch over to select dev. Sorry, I have to get into default. From here, I will delete the workspaces one by one. So this is how I need to delete it.

terraform workspace delete dev

terraform workspace delete qa

terraform workspace delete prod

Now we don't have any workspaces other than default. This is how we use terraform workspace delete command.

Hope this article was helpful for you to understand Terraform Workspaces in a simple and practical way. See you in the next topic. Till then, keep practicing and have fun!