Keeping human connection at the heart of commerce
A marketplace where users can grow
Creating an authentic connection between people in an ever-increasing digital world is a goal many technology companies aspire to achieve. But the proper infrastructure and platforms need to be in place to secure great experiences that enable those connections.
Etsy, a global online marketplace where people come together to make, sell, buy and collect unique and handmade items, has been in the business of creating authentic connections since 2005. Their goal is simple: to keep human connection at the heart of commerce by providing access to handcrafted items made with the human touch. Over the past five years, the online marketplace has seen significant growth, dramatically increasing in overall popularity and the onset of online shopping during the unpredictable COVID-19 pandemic. However, Etsy had been busy proactively planning for the future so when growth came, they were ready.
Etsy had foresight into the scaling challenges they would face if there was a large increase in traffic. They were seeing early signs of potential scaling bottlenecks. Two distinct areas were discovered that needed to be addressed. First, since Etsy relied on two physical data centers located in New Jersey, which were costly to maintain and the long lead times reduced their agility — they knew they needed to migrate over to the cloud as seamlessly as possible. And secondly, the company identified the product delivery process as another potential scaling bottleneck since they expanded their talent pool. While autonomous, Etsy's product teams had many different ways of doing delivery, which made it difficult to onboard new developers and to change teams. Finally, Etsy had never previously engaged with a technology consultancy but knew they needed the right partner to continue their momentum.
Scaling to success
With similar ideologies in Agile software development, technology sustainability, employee care, diversity and open-source, Etsy chose Thoughtworks as their strategic partner to assist scaling efforts and migration to Google Cloud.
For product delivery optimization, Thoughtworks and Etsy formed a cross-functional Product Delivery Culture (PDC) team to analyze the current product delivery process for product delivery optimization. We looked across the entire software development value stream and did an analysis of the complaints of managers and team members. From there, we created a product delivery blueprint, where a set of product delivery principles and related delivery practice changes.
The PDC team looked at a key metric called "Time to Learning," which refers to the time it took for a product team to validate an idea with a customer and better understand its value. Etsy had a baseline of fifty days that they wanted to reduce, so we helped Etsy implement a Lean UX approach, meaning they could develop cheaper prototypes that incorporated more direct user research. Finally, we worked with the product development teams to implement Lean Portfolio Management. Instead of creating lengthy project specifications and plans, the Product Delivery Culture team coached product teams to develop outcome statements for each initiative.
It’s tough to isolate a process tweak on an organization of several hundred engineers. However, we looked at this after implementing it for over two years and thought that all indicators were very positive. We improved our velocity by doing this work. We ultimately produced more benefits for the buyers and sellers — more features, better search, better processes — the whole marketplace became better for buyers and sellers and that was the goal.
To accelerate their migration to Google Cloud, Etsy wanted to bring on outside expertise to help in the adoption of new tooling and technology, such as Terraform, Kubernetes and Prometheus. Thoughtworks infrastructure engineers and technical product managers joined search infrastructure, continuous deployment services, compute, observability and machine learning infrastructure teams. Unsurprisingly, after the cloud migration, the optimization for the cloud didn't stop. Each team continued to look for opportunities to utilize the cloud to its full extent. With the help of the architecture advisory group, they looked at things such as how to reduce the amount of custom code by moving to industry-standard tools, improving cost efficiency and improving feedback loops.
Results
More connections for today and tomorrow
With the cloud and product delivery initiatives well underway, Etsy has continued its meteoric growth. The cloud shift has given the company better observability while making it easier for teams to analyze and track their operational costs when compared to the previous data center approach. The move to Google Cloud also enabled Etsy to build a new machine learning platform based on managed services that reduce operational costs and improves the time from idea generation to production deployment. Testing with pilot teams indicated that the Time to Learning went from fifty days to five days — a significant improvement on the product delivery side.
Etsy continues to embrace measuring everything. Their most recent innovation stems from a collaboration with Thoughtworks and Google Cloud to monitor metrics around energy consumption, reflecting a key goal for Etsy in its pursuit of sustainability. Thoughtworks also created an open-source tool called Cloud Carbon Footprint, inspired by some of the initial research by Etsy and further developed by an internal Thoughtworks team. Thoughtworks and Etsy continue their commitment to sustainability and both organizations look forward to what the future holds.
To learn more details about this story, please visit our Bottlenecks of Scaleups series here.
