TL;DR
A feed handler is the software that receives an exchange's market data, decodes its binary protocol, applies updates in sequence and publishes a usable view to strategies. It is the first component in the trading path and the one that determines whether everything downstream sees reality accurately.
The hard parts are not decoding. They are handling bursts without falling behind, detecting and recovering from gaps, and normalising different venues into a consistent internal model without losing venue-specific meaning.
Key points
- Exchange data is usually binary and multicast, optimised for throughput not readability.
- Sequence numbers and gap recovery are the core correctness problem.
- Bursts during volatile periods, not average rates, size the system.
- Normalisation across venues is where subtle bugs and lost information hide.
How market data arrives
| Element | Typical design |
|---|---|
| Transport | UDP multicast for speed, with TCP recovery channels |
| Format | Compact binary messages, venue-specific |
| Content | Order-by-order updates, or aggregated depth, plus trades and status |
| Redundancy | Two feed lines, A and B, carrying the same data |
| Recovery | Snapshot feeds and retransmission services |
Order-by-order feeds allow a firm to reconstruct the full book, including queue positions, which is why latency-sensitive firms prefer them where available.
What a feed handler does
- Receive and timestamp packets as early as possible, ideally in hardware.
- Decode the binary messages with minimal copying.
- Sequence messages, detecting gaps or duplicates across redundant lines.
- Apply updates to internal book state.
- Publish to strategies without locking or blocking.
- Recover from gaps using snapshots or retransmission, without publishing wrong state.
- Report health, latency and message rates for monitoring.
Failure modes engineers must handle
| Problem | Consequence | Standard mitigation |
|---|---|---|
| Packet loss | Missing updates, wrong book | Redundant lines, gap detection, recovery |
| Burst overload | Handler falls behind, stale prices | Capacity sizing, efficient decoding, backpressure policy |
| Out-of-order delivery | Incorrect state | Sequence checks, buffering |
| Venue protocol quirks | Subtle state errors | Conformance testing against recorded data |
| Clock drift | Misleading latency measurement | Precision time synchronisation |
| Silent staleness | Trading on old data | Heartbeats, staleness alarms, kill switch integration |
Falling behind during a burst is the most dangerous case, because the system keeps working while trading on a picture of the market that no longer exists.
Normalisation across venues
Firms trading several venues need one internal model. Done well, normalisation keeps venue-specific semantics such as order types, auction states and priority rules. Done badly, it flattens differences and produces strategies that behave incorrectly on some venues. A common approach is a shared core model plus per-venue extensions, with explicit tests per venue.
Testing and replay
Feed handlers are tested against recorded market data, including days with extreme volatility, gaps and unusual venue states. Replay is also how strategies are validated, which makes recording infrastructure a first-class part of the stack.
Talent implications
Market data engineering is a distinct specialism from general C++ work. The skills that matter are binary protocol handling, packet-level debugging, capacity thinking and disciplined state management. Candidates typically come from trading firms, exchanges, brokers, market data vendors and networking companies. See the network and exchange connectivity engineer guide and trading technology recruitment in India.
Frequently asked questions
What is a feed handler?
Software that receives, decodes and sequences exchange market data, maintains internal state such as an order book, and publishes it to trading strategies.
Why is market data sent over multicast?
Multicast lets an exchange send one copy of each message to many recipients efficiently, which suits high-rate market data distribution.
How do firms recover from a gap in market data?
Through redundant feed lines, retransmission services and periodic snapshots, while ensuring no incorrect state is published during recovery.
Is market data engineering different from general backend engineering?
Yes. It centres on binary protocols, burst capacity, deterministic state handling and packet-level debugging rather than typical application concerns.