Newshot

Traffic mix matters

Original text(via Lobsters)

Marc's Blog

About Me

Links

Traffic mix matters

Wanna listen to my mixtape?

The preliminary report on last week’s M365 outage contains one statement that really caught my eye:

Our normal load testing did not detect the latent issue because [of] the traffic mix and SKUs used in our labs.

There’s a lot going on in this report (as there is in any report for a complex system), but I wanted to zoom in on this one issue: traffic mix. It’s important, because its the one thing that makes the difference between a great load test and a complete waste of time.

The performance of computer systems is extremely sensitive to workload mix.

Let’s start with a simple simulator, looking at a system with a table of locks. Requests arrive with a key, hold the lock for their key for some period of time, and then free the lock and complete. This isn’t an unreasonable mental model of a relational database, where each transaction takes a lock during its execution.

Each request takes one of 16 worker threads, waits in a FIFO queue for the lock on its key, holds the lock for an exponentially distributed time (mean 1 ms), then releases it and frees its thread. Keys are drawn from a Zipf distribution over N keys; a coefficient of 0 is uniform.

As we may expect, the more skewed the distribution, the less throughput you get out of the system. A small number of keys are highly contended, leading to under utilization of the rest of the resources of the machine, driven by the global concurrency limit.

Now, let’s consider a different mental model of a relational database (at least a single-system one like Postgres). A fast, local, in-memory cache containing some of the accessed data backed by a slower store containing the rest of the data.

Keys (N): Cache size: % of keys Cache is × faster than storage

Each request takes one of the same 16 worker threads and reads its key. A hit is served from an LRU cache; a miss goes to storage (exponentially distributed, mean 1 ms) and the key is added to the cache. Keys are drawn from a Zipf distribution over N keys; a coefficient of 0 is uniform. Note the log scale on the throughput axis.

This curve goes the other way! The more skewed the distribution, the better the performance, over something of a broad range. This is what makes it so important that workload mix in load tests and benchmarks matches real production traffic: throughput can vary by multiple orders of magnitude, in both directions, depending on things as subtle as key skew in the benchmark workload. This is especially true in relational database-shaped things, but applies to systems of all kinds.

Aside: This is also a reason I like optimistic concurrency control, and architectures with a single cost/latency mode. Aurora DSQL reflects this thinking on how to make things less workload sensitive: all reads go to a single fast storage layer, and nobody is holding resources waiting on locks.

Similar Posts

  • 16 Nov 2026 » What does it mean to understand what we built?
  • 23 Oct 2026 » Animations for System Builders: Metastability and Lost Writes
  • 04 Oct 2026 » Strands Decider: Why Not an Encoder?

Marc Brooker The opinions on this site are my own. They do not necessarily represent those of my employer. [email protected]

The text above was extracted automatically from the source page and may miss images, tables or code blocks. The original is authoritative.

Published by Lobsters
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗