How Segment's Product Growth Quietly Became an Engineering Crisis

What Segment actually does

Segment is what's called a Customer Data Platform. In plain terms: when a company's app or website generates data about what a customer just did (signed up, viewed a product, abandoned a cart), that data usually needs to reach a dozen different tools, an analytics platform, a marketing tool, a data warehouse, each one wanting its own copy. Without something like Segment, an engineering team has to build and maintain a separate connection to every single one of those tools by hand.

Segment's product is that connection layer. A company sends customer data to Segment once, and Segment forwards it out to whichever tools that company actually uses. Each of those forwarding targets, an analytics tool, a marketing platform, whatever, is called a "destination." The more destinations Segment supports well, the more valuable the product is. Destinations weren't a technical detail. They were the product growing.

The story

Segment's own engineering team wrote about this in detail, in a post called Goodbye Microservices. Here's what happened.

Early on, Segment's engineers ran into a real bottleneck: a single queue handled all destinations together, which meant that when one destination slowed down or failed, its retries would flood the queue and delay delivery to every other destination. The technical term for this is head-of-line blocking: one slow thing holds up everything behind it. They fixed it by giving each destination its own isolated queue and service. Sensible move, and it worked.

Then, as the company added more destinations, roughly three a month, they kept using the same idea: one destination, one service. It made sense at first. But over time a second problem emerged that had nothing to do with queues. Across those 140+ services, Segment had accumulated over 120 shared dependencies, chunks of code used across many services, each running slightly different versions. Updating any one of them meant testing and deploying dozens of services separately. Engineers under pressure would skip updates and move on, and the versions would quietly diverge further. A single broken test in one service could fail the test run across all of them. The full test suite took around 60 minutes to complete. By the time the situation became critical, three full-time engineers were essentially dedicated to keeping the system alive rather than shipping anything new.

So they undid it. All the destinations went back into a single service they called Centrifuge, with a shared monorepo and one standardised version of every dependency.

What the consolidation revealed was interesting in itself: under all that complexity, the destinations were actually more alike than different. The separation had made that invisible. Each service had looked like a unique thing when most of the logic was shared. After the rebuild, that same test suite ran in milliseconds. Shared library updates went from 32 a year to 46 within the same period, because changing something no longer meant coordinating 140 separate deployments.

One engineer could ship the whole thing in minutes. By their own account, productivity "substantially improved."

What this teaches

It's tempting to read this as "microservices were a mistake," and a lot of people who retell this story do exactly that. That's not quite it. Splitting off the original bottleneck was the right call. What went wrong is that Segment kept applying the same pattern, one service per destination, long after it stopped making sense, without ever stopping to ask whether it still did.

Here's the part worth remembering: destinations were literally how Segment's product grew. So the cost of running their system, one more service to deploy and watch every time, was tied directly to the thing they wanted more of. That's the actual lesson underneath this story: not "avoid microservices," but check whether your architecture gets more expensive every time your product succeeds. If it does, that's worth catching early, regardless of how many services you're running.

The reason this kind of thing is easy to miss is that the original decision was correct. Splitting off the bottleneck worked. So the team kept reaching for the same pattern because it had solved a real problem once, not because they were being careless, but because that's how patterns get established. That's a different thing from a bad decision. It's a good decision that outgrew the context it was made in. Most architectural problems of this shape look exactly like that: a pattern that solved something real, kept going after the problem it was solving had quietly changed.

The question that tends to surface it is a simple one: if someone made this same decision today, for the first time, knowing what the system looks like now, would they still make it? If the answer is no, it's probably worth understanding why not. Not necessarily to change anything immediately, but to know where the pressure is building before it becomes the thing slowing everything down.

It's also worth noting what Segment gave up by consolidating. Without separate services, a bug in one destination could now crash the whole thing, with no fault isolation between them. Caches that had been efficient per-service spread thin across thousands of processes. Updating a dependency could require touching code across many destinations at once. Segment accepted these trade-offs explicitly. The point isn't that the monolith was the obviously correct answer; it's that the balance had shifted far enough that the costs of staying distributed outweighed the costs of consolidating. That balance is different for every system, and at a different size or stage Segment might have landed somewhere else entirely.

TechTek has written about this same pattern from the other direction, in Most Startups Adopt Microservices Too Early, for anyone who wants the fuller technical version of the argument.