At the end of 2016, external account linking played a relatively small role in the Wealthfront ecosystem. Our integration with linking vendors was for the purpose of Instant Account Verification (IAV), which enabled faster funding during the account creation process. However, when we began designing and implementing Path we realized that aggregating more data from these same linking vendors would allow us to provide our clients with a much more vivid picture of their finances.
Over the course of a few short months, the team built out what is now the Wealthfront Linking Platform, a full scale data aggregation and processing platform.
When we learned in 2019 that our primary linking vendor was beginning the process of shutting down their APIs, we were forced to take a step back and consider what the future of the linking platform might look like. We realized that our existing system was riddled with implicit and explicit dependencies on Quovo, and that we would need to rewrite huge portions of the code base during our migration to a new vendor. We decided that this time, we should focus on building the platform in a way that would make it easier to integrate with other vendors in the future. After all, “Fool me once, shame on you. Fool me twice…”
This post will detail some of the decisions we made on our years long journey towards a generic Wealthfront Linking Platform, focusing on changes we made to our data models, APIs, and data ingestion processes.
What does the Wealthfront Linking Platform do?
Before we can dive any deeper into the changes we’ve been making, it helps to first understand at a high level what the role of the Linking Platform is. The platform serves as a middle man between our 3rd party linking vendors (Quovo, Yodlee, Coinbase, etc.) and other products at Wealthfront that depend on external account data (Path, Investments, Money Movement, etc.). We handle authentication on behalf of our clients, ingestion of data provided by our vendors, and computation of intermediate results over that data. Finally, we serve this data to other Wealthfront teams via a well defined API. In essence, the Linking Platform provides a layer of abstraction between our vendors and the wider Wealthfront ecosystem.
Defining a Generic Data Model
Motivation
We quickly realized that our first order of business in moving towards a generic Linking Platform was to define a generic data model. Up until this point, we stored the data that we aggregated in vendor-specific tables, each with its own vendor-specific schema (usually determined by the vendor’s API schema for that data type). We did this even when the underlying data types were the same. Using transactions as an example, we stored all of the transactions that we received from Quovo in the quovo_transactions table, and all of the transactions we received from Plaid (our main data vendor before Quovo) in the plaid_transactions table. As we considered adding more vendors, we realized that this approach was not scalable. Not only would we need to worry about joining a whole bunch of tables to get a sufficient view of a user’s data, we would also have to wrestle with the fact that each table had a slightly different schema.
Our solution to this problem was to define a single, generic schema for each different data type, and store all of the data for a given type in a single table governed by that schema, regardless of the source of that data.
Defining these schemas, though, was not as simple as it might seem. Each vendor that we work with offers a slightly different set of fields for a given data type, and it was not obvious which fields should be included in our unified schema.
Methodology
The first solution we considered was to simply use the union of fields provided by our vendors as the schema, but we realized this was essentially no different than our existing approach (joining all of the vendor specific tables would produce a schema with the union of all of the tables). We also considered using the intersection of fields, which would guarantee that the fields in our schema would be defined for every vendor. With this approach, though, we run the risk of arbitrarily dropping data that could be useful or even necessary.
In the end, we used the following three questions to help us determine which fields to include:
What fields define a record of this data type?
In other words, which fields, if they were missing, would make this data type useless? The idea of a transaction without a transacted amount doesn’t really make sense. So by this rule, transacted amount must be a field in the transaction model.
What fields do our clients rely on for this data type?
We already had systems built out that relied on the existence of certain fields. Path, for example, relied on the expected rate of return of each account to be able to correctly project that account into the future. So by this rule, expected rate of return must be a field on the accounts model.
What fields do our collective common sense tell us to include?
There are some fields that don’t fall squarely into either of the first two categories, but our intuition told us that they should be included. A great example of this would be the cost basis for a given position. A position is substantive without a cost basis, and none of our existing systems rely on that data, but we still decided it would be a useful field to include.
Using this process, we defined generic schemas for all of the relevant data types on the Linking Platform.
Designing a Generic Mapper Interface
Our next step was to figure out how to transform the raw data provided by our vendors into our newly defined generic data type. Because each vendor has a different API and aggregation model, the code we use to pull data is necessarily specific to each vendor. However, we wanted to standardize the process by which we transform vendor specific data to our generic data model, a process we call the “mapping step”.
The interface for a vendor specific mapper is simple:
We decided to formalize this step because it made onboarding a new vendor a simple two step process:
- Integrate with a vendor’s API to aggregate relevant data
- Implement a vendor-specific mapper to transform that data
Because our downstream processes would already be equipped to process and serve generic data to clients we’re able to gain confidence that new vendors will fit nicely into our existing systems if we isolate all of the data transformation to this single mapping step.
Conclusion
After defining our generic data model and designing (and implementing) mappers for each of our vendors, the final step was to start serving generic data over our API. The completion of this project had enormous implications for our platform, the most prominent being that we were able to swap out our main linking vendor (Quovo) for a completely new vendor (Yodlee) with no visible impact to our clients. This work will also make it easier to add new data vendors to our platform with very little added complexity to our system, which in turn will allow us to react more quickly to changes in the industry and ultimately provide a better product.
If this project sounds interesting to you and you want to be at the forefront of new banking technologies, we’re hiring! Visit our Wealthfront careers page for current opportunities.
Disclosure
This communication has been prepared solely for informational purposes only. Nothing in this communication should be construed as an offer, recommendation, or solicitation to buy or sell any security or a financial product. Any links provided to other server sites are offered as a matter of convenience and are not intended to imply that Wealthfront or its affiliates endorses, sponsors, promotes and/or is affiliated with the owners of or participants in those sites, or endorses any information contained on those sites, unless expressly stated otherwise.
Wealthfront offers a free software-based financial advice engine that delivers automated financial planning tools to help users achieve better outcomes. Investment management and advisory services are provided by Wealthfront Advisers LLC, an SEC registered investment adviser, and brokerage related products are provided by Wealthfront Brokerage LLC, a member of FINRA/SIPC.
Wealthfront, Wealthfront Advisers and Wealthfront Brokerage are wholly owned subsidiaries of Wealthfront Corporation.
© 2021 Wealthfront Corporation. All rights reserved.