Why Turborepo is migrating from Go to Rust

Turborepo is a high-performance build system for JavaScript and TypeScript codebases. We're reimagining build systems, taking inspiration from tools like Buck and Bazel, to make them accessible for everyone. At the heart of Turborepo is a very simple idea: never do the same work twice. We accomplish this through incremental builds, parallel execution, and Remote Caching.

As usage has grown and product needs have shifted, we've decided to start an incremental migration from Go to Rust in the 1.7 version. In this article, you'll learn about our motivations for this migration and problems that we are finding Rust solves for our team.

Copy link to headingWhy are we migrating?

The original decision for Turborepo to use Go followed in the footsteps of esbuild. As a JavaScript bundler written in Go, esbuild is fast and avoids much of the initialization overhead of Node.js. Additionally, Go's developer experience is tailored for iteration, something we needed as we learned more and more about what developers wanted from turbo.

In the early days of Turborepo, these properties of Go gave us exactly what we needed for the project to be successful. However, as the Turborepo codebase has scaled and merged with Turbopack, Go has begun to underserve both our core team and users in the areas that matter most to Turbo.

Copy link to headingComparing Go and Rust

We've been working on several other migrations lately and have enjoyed the opportunity to refine our approach:

• Doing a mock migration for the BBC's open-source frontend from React to Next.js

• Dogfooding the Next.js 13 App Router for vercel.com

In any major technical migration, there's a lot to consider and the decision shouldn't be taken lightly. In particular, a language migration is quite demanding, asking you to weigh dimensions like the strengths, weaknesses, and community of a given language according to your specific business and technical context.

In our case, we needed to compare Go and Rust to figure out which language was going to serve us best.

Copy link to headingGo's language design priorities

Go's strength is network computing in data centers and it excels at this task, powering these workloads at the world's largest scales. The goroutine-per-request model, Context API, and the standard library inclusion of server infrastructure is testament to this community focus.

Additionally, Go favors simplicity over expressiveness. A side effect of that decision means more errors are caught at runtime where other languages might catch them at compilation. With a service running in a data center, you can roll back, fix, and roll forward at your convenience. But, when building software that users install, the cost of each mistake is higher.

For us, it's worth using tools that prioritize up-front correctness. We fully recognize the mismatch of Go's priorities and what we are prioritizing as a problem that we created for ourselves to solve.

Copy link to headingOur needs align with Rust

The Rust language and community has prioritized correctness over API abstraction—a tradeoff that we care a lot about when working with:

• Process management

• Filesystems

• Other low level OS concepts

• Shipping software to our users' machines

This means additional complexity is surfaced into our codebase, but it's necessary complexity for the problems we're trying to solve.

Rust's type system and safety features allow us to put guardrails in place in our codebase where we need them. The language's expressiveness allows our developers to encode constraints that catch errors at compile time rather than in GitHub issues.

Copy link to headingComparing by example: file permissions

Go's preference for simplicity at the filesystem was creating problems for us when it came to file permissions. Go lets users set a Unix-style file permission code: a short number that describes who can read, write, or execute a file.

1
> ls -l turbo.json
2
-rw-r--r--  1 anthonyshew  users  247 Jan 1 00:01 turbo.json

While this sounds convenient, this abstraction does not work across platforms; Windows actually doesn't have the precise concept of file permissions. Go ends up allowing us to set a file permission code on Windows, even when doing so will have no effect.

In contrast, Rust's explicitness in this area not only made things simpler for us but also more correct. If you want to set a file permission code in Rust, you have to explicitly annotate the code as Unix-only. If you don't, the code won't even compile on Windows. This surfacing of complexity helps us understand what our code is doing before we ever ship our software to users.

Copy link to headingRust's strong (and growing) ecosystem

Rust has a fantastic ecosystem of high-quality, open-source crates (Rust's equivalent to an npm package) that have clear focus on what we care about. An example of where we benefit from this alignment is when we have to interface with native libraries written in C or C++.

As we've built out Turborepo, we've started to rely more often on native C packages like zstd, a library that helps us compress our cache files. Interoperating with these native libraries in Go requires the use of CGO, which switches us from a pure Go toolchain to a much slower C toolchain. Moreover, this switch is a global process, meaning that if we use a single native library, we have to build our entire codebase with CGO.

In Rust, this interfacing with native C libraries is far more contained. Libraries such as bindgen or cxx create safe wrappers and don't require global changes to our builds. Even better, many libraries come with this wrapper already generated.

For example, we ported our git interface using the git2 crate. git2 interfaces with the C library libgit2 underneath the hood, but exposes a safe, idiomatic Rust API. This allows us to get the benefits of both the Rust and C ecosystems while still maintaining a great internal developer experience.

Copy link to headingThe Turbo ecosystem

Internally, we share a codebase and work closely with the Turbopack team.

For their work, Rust was a clear choice from day one. This meant that, as we both continued our work on build tooling for JavaScript and TypeScript codebases, we were solving the same problems twice: once in Go and once in Rust.

Getting aligned means both teams can ship faster by sharing development and maintenance of common utilities in our problem space. For instance, we're taking a lot of inspiration from the Turbopack team when it comes to file-watching so we can build a feature for smart hot-reloading across workspaces sooner.

Copy link to headingRust makes the core team happier

Another great perk: our team wants to write Rust. It's a language that solves what we care about and brings us joy. The fact that we enjoy writing Rust is valuable, by itself, in more ways than one.

• Happier developers deliver better software. Your brain is better at complex problem-solving when it's happy.

• If we're happier while we work, we're much less likely to burn out.

• Rust's efficiency means less energy consumption, letting us do our part in global sustainability.

Copy link to headingRust makes other developers happier

Looking at the past seven years of StackOverflow survey results, we're not alone.

We also made this choice with future developers of Turborepo in mind.

Web developers are strongly looking towards Rust as a second language to learn after JavaScript, making it more accessible for those coming from JS-tooling-in-JS land. This enables web-focused developers to be able to get involved, enabling the Turbo community to grow.

Copy link to headingContinuing the migration

We're migrating incrementally, so it's not a complete rewrite overnight.

Right now, we have what we call a "Rust-Go-Rust Sandwich." Rust is the entry point, allowing us to choose whether the implementation for a particular command is in Rust or Go. Our Go code is able to call Rust code, too, giving us paths to keep Go around but always be able to get to Rust. Check out the turborepo-ffi crate and ffi.go to learn more.

We're excited about what Rust has already unlocked for our team and can't wait to finish the oxidation and carcinization of our codebase.

If you're a high-performance engineering team building developer tooling or doing systems work and you're debating Rust or Go, we hope our experience can be a helpful reference for you.

Copy link to headingTry out Turborepo today

Turborepo 1.8 was recently released with more features written in Rust. Learn more about what's new to the Turboverse on the 1.8 release post.

If you're looking to create a distributed caching system for your team and CI in three minutes, you can check out our post here.