Everyone talks about latency numbers when discussing realtime systems. They're useful, but they're also pretty easy to obsess over.
More important is how fast the product feels. If users click something and the data is already there by the time they look for it, they don't care whether your p95 is 42ms or 71ms. Chasing that feeling tends to push you toward very different architecture decisions than chasing benchmark screenshots.
A common bottleneck
A lot of realtime dashboards end up looking more or less the same: a database, WebSockets, some background workers, and a handful of cron jobs quietly keeping everything alive. It works. Until it doesn't.
Example:
// Reading directly from the database on every WebSocket message
const rows = await db.query(
"SELECT * FROM positions WHERE account_id = $1",
[accountId],
);
broadcast(rows);
Nothing is technically wrong with this. The problem is that every update now means another database query. Once enough users show up, the database spends most of its time answering the exact same questions over and over again.
Designing for responsiveness
Instead of making the database do all the work, it's usually better to optimize around the way the UI actually reads data.
A few ideas that consistently help:
- Keep hot data in memory. Store it using the same access patterns your UI already uses instead of making the database rebuild the answer every time.
- Split up event fan-out. One slow client shouldn't be able to slow everyone else down.
- Treat freshness as part of the product. Not everything needs to be perfectly live. Showing users what's live, what's a few seconds old, and what's refreshing is often more valuable than pretending everything updates instantly.
Measuring what matters
Latency numbers still matter, just maybe not in the way people think.
Shaving 5ms off the median usually isn't noticeable. Getting rid of random multi-second spikes absolutely is. Consistency almost always beats impressive averages.
At the end of the day, the goal is building something fast enough that nobody ever thinks about its performance.