TL;DR
Low-latency engineering in trading is the discipline of making a system respond quickly and, more importantly, consistently. Teams set a latency budget across each stage of the path, measure it precisely, and remove sources of variability such as memory allocation, context switching, cache misses and network queueing.
The defining difference from general performance work is the focus on tail latency. A system that is fast on average but slow occasionally will be slow exactly when markets move, which is when the cost is highest.
Key points
- Measure percentiles, not averages; the tail is where money is lost.
- Most gains come from removing variability, not from clever micro-optimisation.
- Kernel bypass, CPU pinning and preallocation are standard techniques, not exotic ones.
- Every optimisation needs a measurement to justify it.
Where latency comes from
| Source | Typical contribution | Mitigation |
|---|---|---|
| Network path and propagation | Fixed by geography and cabling | Co-location, cross-connects |
| Network stack processing | Significant in software paths | Kernel bypass, efficient drivers |
| Memory allocation | Unpredictable spikes | Preallocated pools, no allocation on the hot path |
| Cache misses | Large relative to compute | Cache-friendly data layout, prefetching |
| Context switches and scheduling | Jitter | CPU pinning, isolation, priority tuning |
| Locking and contention | Jitter and stalls | Lock-free structures, single-writer designs |
| Logging and telemetry | Hidden cost on the path | Asynchronous, sampled, off-path |
| Garbage collection, where applicable | Severe pauses | Avoid managed languages on the hot path |
Latency budgeting
Teams assign a target to each stage: ingress, decode, book update, strategy, risk check, encode and egress. Each stage is measured with hardware timestamps where possible, and results are reported as distributions. When a stage exceeds budget, the fix is chosen by evidence rather than instinct.
A typical reporting format is median, 99th percentile and 99.9th percentile, plus maximum observed during the session. Averages hide exactly the behaviour that matters.
Techniques in common use
Preallocation and memory pools. Nothing is allocated on the hot path; buffers and objects are created at startup.
Cache-aware data layout. Structures of arrays rather than arrays of structures, hot fields packed together, padding to avoid false sharing.
Lock-free single-producer, single-consumer queues for handing data between threads without contention.
CPU pinning and isolation. Critical threads pinned to isolated cores, with interrupts moved elsewhere.
Kernel bypass networking. Sending and receiving packets in user space, avoiding much of the operating system network stack.
Hardware offload. Moving decoding, book maintenance or risk checks into FPGAs where the logic is stable. See the FPGA engineer guide.
Busy polling rather than interrupt-driven waiting, trading CPU for predictability.
Measurement discipline
- Timestamp at hardware level on ingress and egress where possible.
- Keep telemetry off the critical path, sampled and asynchronous.
- Replay recorded market data to reproduce performance conditions.
- Test with realistic burst rates, not steady-state traffic.
- Track latency over time, since regressions creep in through ordinary changes.
What good engineers do differently
They start with a measurement, form a hypothesis about the cause, change one thing, and measure again. They know which parts of the system are irrelevant to latency and leave them readable and simple. They also know when the answer is architectural, such as moving work off the hot path entirely, rather than micro-optimising code that should not be there.
Talent implications
Low-latency engineers are the largest specialist hiring need in most trading technology teams, and the pool with genuine production experience is small. Strong candidates often come from operating systems, networking, databases, embedded systems and games engineering, provided they can show measured work. See the low-latency C++ engineer guide and our trading technology practice.
Frequently asked questions
What is tail latency and why does it matter?
Tail latency is the slow end of the distribution, such as the 99.9th percentile. It matters because slow responses cluster during volatile periods, when trading opportunities and risks are largest.
What is kernel bypass?
Kernel bypass lets a trading application send and receive network packets without passing through the operating system's standard network stack, writing directly to and from the network card in user space. It removes system calls, context switches and kernel buffer copies from the path, which cuts latency by several microseconds and, more importantly, removes most of the jitter that scheduling introduces. Common implementations include Solarflare Onload, DPDK and vendor-specific APIs.
Do all trading systems need microsecond latency?
No, and over-specifying is a common and expensive error. Requirements range from single-digit microseconds in competitive high-frequency trading, through milliseconds for most systematic and execution systems, to seconds for longer-horizon strategies. Demanding microsecond latency when the strategy does not need it shrinks the candidate pool to a few hundred people globally, adds engineering complexity, and buys nothing. Define the latency budget from the strategy before writing the brief.
How do firms measure trading latency accurately?
Accurately measured trading latency requires three things. Hardware timestamping at the network card on both ingress and egress, because software timestamps include the jitter you are trying to measure. Precise clock synchronisation, usually PTP, so timestamps across machines are comparable. And percentile reporting rather than averages: the 99th and 99.9th percentile is what loses money, and a good mean can hide a terrible tail. Replayed sessions let you compare changes against identical input.