Ably engineering

Introducing tfgen: configure your Terraform stacks using plain Go

HashiCorp sunset CDKTF with no replacement in sight. Here's how we built our own Go alternative, tfgen, and used AI to migrate 150 production Terraform workspaces in two weeks instead of months.

Introducing tfgen: configure your Terraform stacks using plain Go

Until recently, we extensively used HashiCorp's CDK for Terraform. Then they announced the end of its development. We didn't want to spend months migrating to a new ecosystem, but we needed a replacement.

In the Infrastructure Team at Ably, we like Go. I wondered if there was an opportunity for something simpler.

Provisioning infrastructure at Ably

We provision all our infrastructure for the core Pub/Sub platform where possible using Infrastructure as Code, initially using Terraform and then making the switch to OpenTofu (but for simplicity I will refer to "Terraform" throughout, even though we use tofu). Every production change to our infrastructure has to go through peer review and is deployed using Scalr.

This includes our networking stack, which is a set of Amazon VPCs provisioned across multiple regions for both our non-production and production environments, in accounts separate from our application workloads.

Since each VPC must be peered with every other VPC along with resources such as security group rules and NAT gateways, configuring this entirely in HCL became unwieldy very quickly.

It was in this context that we adopted the "terraform-cdk", also known as "CDKTF". This tool was a natural way to get out of HCL's limitations, where engineers were able to write code in a variety of languages which produced Terraform-compatible JSON. We adopted this using TypeScript.

We had a lot of success using this tool, and because of this adopted it for some of our functionality that had a lot of repeated logic, but required us to provision resources in separate workspaces, in separate regions, and in separate accounts. Using CDKTF made this extremely simple, and we provisioned new resources using human-friendly configuration files.

The resulting generated JSON was committed to our repository, and deployed using our standard processes. We could see who made what changes to a configuration file, and see it clearly expressed in the more complex JSON output.

We maintained roughly 150 production workspaces using this mechanism.

Goodbye CDKTF

Unfortunately, on December 10, 2025, HashiCorp decided to sunset CDKTF.

This put us in a bit of a bind: yes, the tool was working fine, but there will be no new features, and any open bugs will never be fixed.

We did some research on a potential replacement: we wanted to keep the process we have that works really well for us, and we didn't want to commit to a lengthy migration that touches all of our critical production resources, which increases risk and takes a lot of engineering effort with relatively low value.

We considered various other tools: Terragrunt, Pulumi, Jsonnet and Cue.

Terragrunt is widely used and allows you to manage many projects at scale, which would work for us, but to be able to fit into this model, we would have needed to change our workflows, which probably would have been the bulk of the migration.

Given we wanted to continue using our centralised configuration files, we would have needed to write something custom anyway to keep them, or migrate to something else. Basically, it would have been a huge amount of work.

Pulumi suffered the same problem: it's a great tool, but would have meant a migration to a completely new ecosystem.

The templating languages of Jsonnet and Cue were probably closer to what we needed, but it would have been a great deal of work to ensure we had byte-identical output to what we already had committed to the repository.

Essentially, all these tools suffered the same issue: a huge amount of work that we were not prepared to undertake. In the end we settled on writing something ourselves, using Go.

Writing a replacement

We already primarily use Go for managing our infrastructure, so it was the natural choice for a homespun replacement.

As well as writing a drop-in replacement, which used our existing configuration and wrote identical output to existing paths, I also wanted to ease the constraints that we had previously bumped into with CDKTF.

CDKTF required us to "get" specific provider versions, which had generated strong types. This made writing TypeScript a little easier with LSPs, but also involved a build step to fetch these definitions. In practice, I didn't think the stronger types were worth the cost of the extra build step. I wanted something more flexible, that produced code that was validated by Terraform tooling itself.

My idea was to produce a simple public interface that matched Terraform concepts, where each component was a type:

// Resource represents a resource block in a Terraform configuration.
type Resource struct {
	Type   string // resource type, e.g. "aws_s3_bucket"
	Name   string // local name, unique per type within the stack
	Config Config // the resource's arguments
}

The Config type here is what makes it flexible. Rather than having to generate a large amount of types for each different provider and handle the problems of how they change between versions, instead the user is responsible for setting the required values.

When the Terraform-compatible JSON is generated, then the user can use validate to ensure it's correct, bypassing the requirement for generated types.

This means that we can express ourselves naturally using Go, while giving us the experience of writing HCL.

Mimicking CDKTF's language, each component is then appended to a "stack":

stack := tfgen.NewStack("example")

bucket := &tfgen.Resource{
	Type: "aws_s3_bucket",
	Name: "example",
	Config: tfgen.Config{
		"bucket": "my-example-bucket",
	},
}

stack.AddResource(bucket)

You can then write this stack to wherever you want:

f, _ := os.Create("main.tf.json")
stack.Write(f)

The resulting JSON is fully Terraform compatible and can be applied with tofu apply:

{
  "resource": {
    "aws_s3_bucket": {
      "example": {
        "bucket": "my-example-bucket"
      }
    }
  }
}

Migrating using AI

Like most engineers in 2026, I am producing a lot of my code using AI.

I can find AI tooling hit and miss when designing something from scratch. In fact, I did bounce some ideas off AI for writing this package and it did a terrible job.

This is why I wrote the public interface myself upfront: I knew exactly how I would like to use the package, which put me in control of the outcome.

The key with AI is to have strong examples that it can work from, which reduces the number of design decisions it has to make.

Since we were writing to an existing path requiring byte-identical output, it meant that my AI tool (in this case, Claude Code) also had a clear feedback loop.

With a strongly typed public interface, a clear example of what I want the human-readable code to look like, and the exact output we were after, which could easily be checked using git diff, I was confident that AI would make short work of the migration.

I asked it to perform the migrations in stages, using a separate agent for each:

Research: read the existing CDKTF code, the existing output, and the format of the configuration files. This produced a summary of what logic we needed to copy for these particular workspaces and where they needed to be written.

Planning: this agent used the previous output to create a clear, ordered plan for implementation, including specific feedback loops and tests at each step, such as:

Running go build, go test and validation tools such as go vet

Using git diff after generating the JSON to ensure zero changes, or only non-impacting ones (CDKTF wrote a metadata field that was fine to drop)

Implementation: the final agent worked through the plan, and also performed validation.

We had several differing sets of workspaces, and I worked through each in turn. I completely migrated a set of workspaces in production before moving on to the next set, which increased confidence in the changes it was producing.

I think in the past if I had to write it by hand, it would have taken me months. It almost certainly would have been put to the bottom of our backlog, given the actual resulting value was zero changes, other than a change of tooling.

Fortunately, using AI meant that I actually completed the migration of all our workspaces in a couple of weeks. Not only that, I did it alongside my other day-to-day work, because I could let AI work in the background while I did the more urgent work I was assigned.

I was slightly bowled over by the success of this project, and it clearly proved to me the merits of using AI tools.

Giving back to the community

Given the lack of a clear replacement to CDKTF, I wondered if others in the community might find this tool useful, and for this reason we have decided to open source tfgen.

You can find it at github.com/ably/tfgen.

The entire core package is only around 600 lines of Go, and contains everything needed to generate complete Terraform configurations, as well as provider helpers. For example, we've provided a small set of resources for the Ably Terraform provider:

stack := tfgen.NewStack("ably")

block := &tfgen.TerraformBlock{RequiredVersion: ">= 1.9"}
block.AddRequiredProvider("ably", ably.ProviderVersion())
stack.SetTerraformBlock(block)

stack.AddProvider(ably.Provider())

app := ably.App("main", ably.AppOptions{TLSOnly: true})
stack.AddResource(app)

var buf bytes.Buffer
if err := stack.Write(&buf); err != nil {
	panic(err)
}

fmt.Print(buf.String())

Since there are no generated types, references between resources are made using the Ref method, which renders as a Terraform expression (such as ${ably_app.main.id}) in the generated JSON:

stack.AddResource(ably.Namespace("ai-transport", ably.NamespaceOptions{
	AppID:           app.Ref("id"),
	MutableMessages: true,
}))

It also has the concept of writing multiple stacks built-in, since that's how we use it at Ably:

stack1 := tfgen.NewStack("stack1")
stack2 := tfgen.NewStack("stack2")

mystacks := tfgen.Stacks{stack1, stack2}

mystacks.Write("my/stacks/directory")

Frequently Asked Questions

What is CDKTF and why was it sunset? CDKTF (terraform-cdk) let engineers write infrastructure code in general-purpose languages like TypeScript, Python and Go, which compiled to Terraform-compatible JSON. HashiCorp announced in December 2025 that it would stop developing the tool, with no new features and bug fixes.

Is tfgen compatible with OpenTofu as well as Terraform? Yes. tfgen produces standard Terraform-compatible JSON, so it works with both Terraform and OpenTofu without any changes to how you write your configuration.

How is tfgen different from CDKTF? tfgen doesn't generate provider-specific types. Instead of fetching and maintaining strongly typed provider definitions, you set configuration values directly and validate the output using Terraform's own tooling. This removes the extra build step CDKTF required.

Can I use tfgen for providers other than Ably's? Yes. The core package is provider-agnostic. Ably has included a small set of helpers for the Ably Terraform provider as a working example, but tfgen's Resource and Config types work with any Terraform provider.

Do I need to write Go code for every Terraform resource type? No. tfgen doesn't require generated types per resource, so there's nothing provider-specific to install or keep in sync. You express any resource using the generic Resource type and its Config map.


If you find it useful and have any feature requests or questions, please get in touch or raise a GitHub issue.