In this blog post, we will delve into the topic of Monorepo adoption at Wealthfront. We will start by discussing the technical challenges posed by the previous tech stack and how switching to a new tech stack enabled us to modernize our web application. We will then take a closer look at the benefits of a Monorepo setup and the various tooling options we explored in the process implementing one. Lastly, we will provide an in-depth review of the advantages we have experienced since adopting this approach, including how it has transformed how we build and deploy features to our clients.

Wealthfront’s Monolithic Struggles

The use of monolithic software applications has been a common practice in the tech industry for a long time. At Wealthfront, we also relied on a single, sizable, monolithic codebase for our web application. Our monolith was based on a Rails Server-Side Rendered React App. Initially, this approach worked well when our team was small and had limited project dependencies. However, as our codebase grew more complex and the number of engineers working on it increased, we started facing challenges with the monolithic structure of the application.

In 2021, we decided to modernize our web technology stack to introduce new ways of building and shipping features to clients. Although the Server-Side Rendered React App had been a reliable option for over a decade, using any of its infrastructure in other applications wasn’t easy. For instance, we built a lot of infrastructure directly into the app, such as our Sentry SDK settings for error tracking, our internal component library created by our design system team, and various internal utility helpers. However, these tools were only compatible with specific frameworks, resulting in significant code duplication, inconsistent tooling, and non-modular code.

One example of code duplication was when we had to build design system components in a new app. At Wealthfront, we aim to create exceptional and recognizable products for our clients across all Android, iOS, and Web platforms. To achieve this, we depend heavily on our Design System Component library. However, on the Web platform, we faced a challenge in making these components accessible to every app. The monolithic approach was one of the factors that stood in our way. We were forced to copy these components into each new environment without an alternative approach.

To illustrate the issue further, we can take the example of our Explore Pages. This page allows users to view and learn about popular exchange-traded funds, such as VTI, that are available to clients. Initially, these pages were part of a logged-in initiative by our investing team, so the code was heavily tied to our Rails/React app. However, we wanted our clients to be able to access these pages, whether logged in or not, so we built a similar page for logged-out users. Unfortunately, the logged-out pages were created using a different tech stack than the logged-in pages, and we could not reuse any of the code. What should have been a simple migration ended with us having to re-implement everything from scratch.

The Rise of Monorepos

As our web engineering team at Wealthfront embarked on a mission to improve the scalability and efficiency of our development processes, we realized that we needed a significant overhaul of our existing infrastructure. Our previous setup had been sufficient for earlier stages of growth, but as our projects became larger and more complex, we faced limitations that we could no longer ignore.

We set out to find practical solutions and noticed a significant trend in Monorepo tooling within the web development community. In the past, these tools were only accessible to tech giants such as Google and Microsoft, who had the resources to develop and maintain sophisticated internal tooling. However, we saw a transformation in the landscape with the emergence of newer, more accessible open-source alternatives.

Open-source Monorepo tools like Lerna, Turborepo, Rush, and Nx have democratized access to Monorepo tooling. This has created opportunities for companies of all sizes, including companies like Wealthfront, to leverage the benefits of Monorepos in enhancing collaboration, code-sharing, and overall development efficiency. At Wealthfront, we have been able to streamline our development processes, improve scalability, and optimize collaboration among our team thanks to this newfound accessibility.

Understanding the Monorepo Approach

In order to comprehend the Monorepo approach, it is crucial to differentiate it from the conventional Monolithic approach. These two approaches possess similar characteristics, thus causing confusion among engineers. Both approaches store code within a single repository, but in a Monolithic structure, a project’s code is kept in a single tightly coupled repository. In contrast, a Monorepo approach intends to divide the codebase into well-defined and isolated applications and libraries while maintaining them in a single repository, making them collocated and simple to use. This allows for better collaboration and code sharing among team members. 

Now that we understand Monorepos and their benefits over traditional Monolithic setups, let’s explore their advantages in more detail.

Code Sharing

Switching to a Monorepo setup has the significant benefit of sharing code across packages. This setup promotes the creation of highly modular packages. At Wealthfront, we treat package creation like we would any typical pure function, which results in small and clear API surfaces that are easy to understand, document, test, and maintain. Following this modular package approach, we separated our Design System component library into its own package, and we created a UI package for more general-purpose components like the ones we used on our explore pages. Such composable packages have helped us remove code duplication and given us confidence in our ability to scale our frontend platform. We no longer face high costs while setting up additional teams and projects.

Consistent Tooling

Another significant benefit of having a Monorepo setup is enforcing consistent tooling across projects. Rather than individual teams having their commands for tests, building, serving, linting, etc., we can define configuration packages that set up tooling automatically. Each team can then extend from these configuration options. This saves time for product teams, as they no longer need to waste time redefining tooling. It also makes it easier for teams to navigate through different areas of the codebase they may only interact with occasionally . This is especially important for us at Wealthfront, as we are a lean team expected to help in various areas of the codebase frequently.

Atomic Commits

When working with a monorepo, engineers benefit from the use of atomic commits. Atomic commits ensure that each commit represents a single logical change, which makes it easier to understand the changes made to the codebase over time. It also makes it easier to revert to a previous state if necessary. Additionally, atomic commits give engineers immediate feedback on breaking changes, which makes it easier to address changes to design system components. This is because a design system engineer can make updates and test the code immediately without going through several steps.

Creating new projects is easy

In a Monorepo setup, creating new projects is simple. All you need to do is build a new directory. To make things even better, we can provide consistent tooling to ensure that the setup of new projects is similar to that of existing ones. This means that creating new projects is quick and easy and ensures consistency across the board. On the other hand, in a traditional Monolithic application, creating new projects is much more complicated. You must bundle your code in your existing app or build a separate repository.

We tried the multi-repo approach before moving to the Monorepo approach we’re now using with some of our internal libraries but encountered issues. For example, Wealthfront places a high value on automation, so all our repositories go through a strict CI/CD pipeline before being deployed to production. However, implementing these pipelines can be challenging, and creating a new repository means constructing a new pipeline from scratch. This manual process requires additional engineering resources and costs more money. Additionally, bumping packages in multiple projects has proven to be error-prone. Some teams must remember to bump packages on time, and bugs become more challenging to track down. None of these things are an issue when working in a Monorepo.

Monorepo Tooling

After thoroughly understanding Monorepos and their advantages over a Monolithic approach, our next task was determining the most suitable tool that best manages our monorepo. This was a challenging process, as many options needed to be carefully evaluated and compared.

To name a few monorepo build tools:

1. Bazel: An open-source build and test tool similar to Make, Maven, and Gradle, Bazel supports various programming languages and platforms and is known for its speed, reproducibility, and scalability.
2. Gradle: A powerful build automation tool that supports multiple languages and platforms, Gradle is known for its performance and flexibility and is the official build tool for Android.
3. Lage: A build orchestrator for monorepos that optimizes JavaScript and TypeScript project builds and tests by only building what is necessary, leveraging caching and parallel execution.
4. Lerna: A popular tool for managing JavaScript projects with multiple packages, Lerna optimizes the workflow around managing multi-package repositories with git and npm.
5. Moon: A tool for managing and developing large codebases or monorepos, Moon aims to provide fast, reliable, reproducible builds.
6. Nx: An extensible dev tools and innovative build system, Nx provides advanced capabilities for monorepo development, including computation caching, dependency graphs, and more.
7. Pants: A fast, scalable build system for monorepos, Pants supports multiple languages and provides fine-grained build isolation by only rebuilding what is necessary.
8. Rush: A scalable build orchestrator for monorepos, Rush is designed to handle large repositories, providing rigorous policy enforcement and incremental builds.
9. Turborepo: An ultra-fast build system for JavaScript and TypeScript monorepos, Turborepo optimizes build performance with efficient caching and parallel execution.

To determine which tool to use for our Monorepo, we carefully weighed the advantages and disadvantages of several of these options. We aimed to establish criteria that constitute a reliable and effective Monorepo tool. While we acknowledge that these criteria are subjective and may not apply to every situation, we believe they provide a solid foundation for making informed decisions about your Monorepo setup.

Package Publishing Feature-set

Monorepo tooling should pave the way for quickly generating change logs, automatically publishing builds, and keeping versions in sync across packages. Rush and Nx have this built-in via CLI flags, but other tools like Turborepo have sound examples of how to use De Facto libraries like `changesets`.

Flexibility in build tooling

Build tooling should be flexible. It should not restrict you from using tools like Webpack, Vite, Rollup, or whichever you prefer.

Best exit strategy

We have found that specific tools, like Rush, require more commitment than others because they require particular directory structures and CLI tools instead of just sitting on top of existing tools like npm, yarn, and pnpm. Tools that wrap existing package managers can be more challenging and often require some training since most engineers are unfamiliar with the custom CLI commands. For this reason, a crucial factor for us was how well the tool worked with the existing tools we used, like pnpm.

Precise and efficient task orchestration

Monorepo tooling should be able to run tasks in the correct order and concurrently. It should avoid running the entire monorepo each time. Your build tool should be smart enough to identify which packages your changes affect and only run relevant tasks for those packages.

Local computation caching

Monorepo tooling should maintain the ability to store and replay files and output tasks to avoid unnecessary rebuilding or testing on the same machine.

Remote computation caching

Remote computation caching allows sharing cached artifacts across different environments, which means the entire web team, including CI agents, will not have to rebuild or retest the same thing multiple times.

Community

It’s essential for the community to be active and have recent resources available for newcomers to join quickly. We shouldn’t have to create our documentation for existing tools. Ideally, we can direct engineers to the tool’s documentation for help when needed.

After thoroughly analyzing available options, we were left with two potential tools – Turborepo and Nx. Both tools had their pros and cons, and it was clear that either choice would have been good. However, we decided to go with Turborepo because it offered all the features we were looking for and had a narrow but focused scope, making it easier to maintain in the long run.

Conclusion

The decision to migrate our Monolithic Application to a Monorepo has proved to be a game-changer for us as we look back over the past year with great satisfaction. The benefits of this approach have been manifold – we have established a uniform set of tools and processes across all packages, eliminated redundancy in code, fostered greater collaboration through better reuse patterns, and, above all, future-proofed our CI pipelines. The Monorepo tooling has been instrumental in driving these outcomes, and we are grateful for the seamless integration it allows us.

However, we are not ones to rest on our laurels. While we have made significant strides in isolating code into separate packages, we acknowledge that there are still ample areas for improvement, and we are excited to continue our commitment to the Monorepo approach in the years to come. 

If working with technologies like the ones mentioned in this post sound interesting to you, and you enjoy working with very talented engineers, please check out our careers page!

Disclosures

The information contained in this communication is provided for general informational purposes only, and should not be construed as investment or tax advice. Nothing in this communication should be construed as a solicitation or offer, or recommendation, to buy or sell any security. Any links provided to other server sites are offered as a matter of convenience and are not intended to imply that Wealthfront Advisers or its affiliates endorses, sponsors, promotes and/or is affiliated with the owners of or participants in those sites, or endorses any information contained on those sites, unless expressly stated otherwise.

All investing involves risk, including the possible loss of money you invest, and past performance does not guarantee future performance. Please see our Full Disclosure for important details.

Wealthfront offers a free software-based financial advice engine that delivers automated financial planning tools to help users achieve better outcomes. Investment management and advisory services are provided by Wealthfront Advisers LLC, an SEC registered investment adviser, and brokerage related products are provided by Wealthfront Brokerage LLC, a member of FINRA/SIPC.

Wealthfront, Wealthfront Advisers and Wealthfront Brokerage are wholly owned subsidiaries of Wealthfront Corporation.

© 2024 Wealthfront Corporation. All rights reserved.