Design a Stock Brokerage Platform

TL;DR

Build a system like Robinhood -- a retail brokerage that lets users buy and sell stocks, NOT the exchange itself. This distinction matters enormously. A brokerage routes orders to exchanges and market makers via the FIX protocol; it does not run an order book or match buyers with sellers. Robinhood makes 77% of its revenue from PFOF (Payment for Order Flow), sending orders to Citadel Securities and Virtu Financial in exchange for rebates. The system design challenge is the real-time data pipeline: streaming OHLCV candlestick charts to 23 million users while throttling updates via conflation, running pre-trade risk checks in < 10ms, and handling the GameStop/DTCC incident where clearing requirements forced Robinhood to restrict buying. The brokerage is a middleman, and designing it means understanding every system the middleman connects to.

---

The System

Robinhood. A user opens the app, sees a live stock chart for Tesla updating every second, places a market order to buy 10 shares, and sees the execution confirmation within 2 seconds. Behind that simple UX, the system has routed the order through a pre-trade risk check, transmitted it via FIX to a market maker, received the fill, updated the user's portfolio, calculated margin, and pushed the trade to a clearing pipeline that settles in T+1 (one business day).

Why is this different from designing a general web application? Because financial systems have regulatory requirements that override engineering preferences. You must keep an audit trail of every order and fill. You must execute pre-trade risk checks before sending orders. You must calculate and enforce margin requirements in real time. You must report trades to FINRA and the SEC. You must handle corporate actions (stock splits, dividends) that retroactively change share counts. And you must do all of this while streaming live market data to millions of concurrent users without dropping frames or showing stale prices.

---

Requirements

Functional Requirements

Requirement	Details
Account management	User creates account with KYC verification. Links bank for deposits/withdrawals.
Market data streaming	Real-time stock prices, candlestick charts (1m, 5m, 1h, 1d intervals).
Order placement	Market, limit, stop-loss orders for equities and options.
Order execution	Route to exchange/market maker via FIX. Display fill price and confirmation.
Portfolio view	Current holdings, P&L, cost basis, portfolio value over time.
Margin/buying power	Calculate available buying power based on cash + margin.

Non-Functional Requirements

Requirement	Target
Market data latency	< 500 ms from exchange to user's screen
Order submission to fill	< 2 seconds for market orders during market hours
Pre-trade risk check latency	< 10 ms
Availability	99.95% during market hours (9:30 AM - 4 PM ET)
Scale	23M funded accounts, 5M DAU, 10K orders/sec peak, 100K market data updates/sec

Back-of-Envelope Math

Funded accounts:             23 million (Robinhood's reported number)
Daily active users:          5 million
Concurrent users at peak:    2 million (market open, major earnings)

Orders per day:              ~5 million (avg for Robinhood in 2023)
Orders per second (avg):     ~770 (6.5 market hours = 23,400 sec)
Orders per second (peak):    10,000 (GameStop day: 30x normal volume)

Market data:
  Stocks tracked:            ~8,000 (all US-listed equities)
  Updates per stock per sec: ~10 (consolidated quote/trade feed)
  Total updates/sec:         80,000 raw, conflated to ~20,000 for clients
  Update size:               ~100 bytes (symbol, price, volume, bid, ask)
  Bandwidth per user:        ~100 updates/sec * 100 bytes = 10 KB/sec
    (a user watching ~10 stocks receives ~10 updates/stock/sec = ~100 updates/sec total)
  Total bandwidth (2M users): 20 GB/sec (this is why you need conflation)

Portfolio storage:
  Avg positions per user:    5 stocks
  Position record:           ~200 bytes (symbol, qty, avg_cost, current_value)
  Total portfolio data:      23M * 5 * 200 = 23 GB (fits in memory)

Order storage:
  Order record:              ~500 bytes (all fields + timestamps + audit)
  Orders per year:           5M/day * 252 trading days = 1.26 billion
  Annual storage:            1.26B * 500 = 630 GB

The number to focus on: 20 GB/sec of market data bandwidth to stream to all concurrent users. You cannot fan-out raw exchange data to 2 million connections. Conflation (throttling update frequency) is the answer.

---

Naive Design

Single server, WebSocket connections, PostgreSQL.

Schema:

CREATE TABLE accounts (user_id, cash_balance, margin_limit, status);
CREATE TABLE positions (user_id, symbol, quantity, avg_cost_basis);
CREATE TABLE orders (
    order_id, user_id, symbol, side, -- BUY/SELL
    order_type, -- MARKET/LIMIT/STOP
    quantity, limit_price,
    status, -- PENDING, SUBMITTED, FILLED, PARTIALLY_FILLED, CANCELLED
    fill_price, filled_quantity,
    created_at, updated_at
);

Order flow:

1. User places order via REST API.
2. Server checks cash balance >= order value.
3. Server sends order to exchange via HTTP API.
4. Server polls exchange for fill status.
5. On fill, update positions and cash balance.

Market data:

1. Server subscribes to exchange WebSocket feed.
2. For each price update, broadcast to all connected clients watching that symbol.

This is what a hackathon team builds in 48 hours. It handles 10 users.

---

Where It Breaks

Problem 1: Market Data Fan-Out Kills Bandwidth

If 500,000 users are watching TSLA and the price updates 10 times per second, that is 5 million WebSocket messages per second just for one stock. At 100 bytes per message, that is 500 MB/sec for a single stock. With 8,000 stocks, you are looking at terabytes per second. No server fleet can sustain this.

Problem 2: HTTP Polling for Fill Status Is Suicide

Polling the exchange for order status means thousands of HTTP requests per second per order. A single market order fills in 10-100ms. If you poll every 100ms, you get 1 request for the fill and 0 wasted. But if you poll every 1 second, the user waits up to 1 second for confirmation. FIX protocol solves this with push-based execution reports.

Problem 3: Cash Balance Checks Race with Concurrent Orders

User has $1,000 in cash. They simultaneously place two market orders for $800 each. Both pass the balance check. Both fill. The user now has -$600 in cash. You have extended unsecured credit to a retail user. Regulators are not amused.

Problem 4: No Audit Trail

Financial regulations (SEC Rule 17a-4) require brokerages to maintain immutable records of all orders and executions for 6 years. A status column that gets overwritten loses the history. When did the order go from PENDING to SUBMITTED? When did the partial fill happen? The regulator will ask.

Problem 5: Market Data Has No History

Streaming live prices but not storing historical OHLCV data means users cannot view charts for "last 3 months." You need a time-series database for candle data.

---

Real Design

Architecture Overview

┌─────────────────────────────────────────────────────────┐
│                    Exchange/Market Maker                  │
│  (NYSE, NASDAQ, Citadel Securities, Virtu Financial)     │
└─────────────┬────────────────────────┬──────────────────┘
              │ FIX Protocol           │ Market Data Feed
              │ (orders + fills)       │ (quotes + trades)
┌─────────────┴──────────┐  ┌─────────┴──────────────────┐
│   Order Management     │  │   Market Data Service       │
│   System (OMS)         │  │   (normalize, conflate,     │
│   - FIX engine         │  │    compute OHLCV candles)   │
│   - Order state machine│  │                             │
│   - Fill processing    │  └─────────┬──────────────────┘
└─────────────┬──────────┘            │
              │                       │
┌─────────────┴──────────┐  ┌─────────┴──────────────────┐
│   Pre-Trade Risk       │  │   Streaming Gateway         │
│   Engine               │  │   (WebSocket, conflation    │
│   - Buying power       │  │    per-client, pub/sub)     │
│   - Position limits    │  │                             │
│   - Pattern day trader │  └────────────────────────────┘
└─────────────┬──────────┘
              │
┌─────────────┴──────────┐
│   Account/Portfolio    │
│   Service              │
│   - Positions          │
│   - Cash balance       │
│   - Margin             │
└────────────────────────┘

Component 1: FIX Protocol Engine

FIX (Financial Information eXchange) is the standard protocol for communicating with exchanges and market makers. Every brokerage uses it. It is not HTTP. It is not REST. It is a tagged field, delimited text protocol over persistent TCP connections (fields are separated by the ASCII SOH character, 0x01). Note: FIX/FAST and SBE are binary variants used for high-frequency market data; the standard FIX 4.x session/application layer is plain ASCII text.

FIX message example (New Order):

8=FIX.4.2|35=D|49=ROBINHOOD|56=CITADEL|
11=ORD12345|55=TSLA|54=1|38=10|40=2|44=250.00|

Translation: Message type D (New Order Single), from Robinhood to Citadel, order ID ORD12345, symbol TSLA, side=Buy, quantity=10, type=Limit (40=2), price=$250.

FIX session management:

1. Logon (35=A): Establish session with heartbeat interval.
2. Heartbeat (35=0): Every 30 seconds. If missed, the counterparty sends a Test Request.
3. Sequence numbers: Every message has a sequence number. If a gap is detected (message 5 followed by message 7), the receiver requests a resend (35=2) of the missing messages. This guarantees delivery.
4. Execution Report (35=8): The exchange/market maker pushes fill notifications. No polling needed.

Why FIX matters for system design: FIX is push-based. When your market order fills, you get an Execution Report immediately (typically within 10-100ms). You do not poll. This changes the architecture: the OMS is event-driven, not request-response.

Connection topology: Robinhood maintains persistent FIX connections to Citadel Securities, Virtu Financial, and a few other market makers. Each connection handles ~1,000 orders/sec. With 10K orders/sec peak, you need ~10 FIX sessions spread across multiple OMS nodes.

Correlating fills with internal orders: FIX tag 11 (ClOrdID) is a client-order-id field where you embed your internal orderId. The exchange echoes it back in every Execution Report. REST-based exchange APIs offer equivalent metadata fields. This eliminates the need for a separate mapping database to correlate exchange fills with internal orders -- a common over-engineering mistake in interviews. Your OMS receives the fill, extracts the embedded orderId and userId, and updates the correct account directly.

Component 2: Payment for Order Flow (PFOF)

This is how Robinhood makes money. Instead of charging commissions, Robinhood routes orders to market makers like Citadel Securities and receives a rebate per order.

How it works:

1. User places a market order to buy 10 TSLA at market price.
2. Robinhood's order router evaluates: which market maker will give the best execution price AND pay the highest rebate?
3. The order is routed to Citadel Securities via FIX.
4. Citadel fills the order at $250.01 (the NBBO -- National Best Bid and Offer -- was $250.00 ask, but Citadel provides $0.01 price improvement).
5. Citadel pays Robinhood $0.002 per share rebate. On 10 shares, that is $0.02.
6. At scale: 5M orders/day * avg 50 shares * $0.002 = $500K/day in PFOF revenue.

System design implication: The order router is not just sending orders to an exchange. It is making a routing decision based on:

• Best execution obligation (SEC regulation -- you must seek the best price for the customer)
• PFOF rebate rates (different market makers pay different rates)
• Fill rate (some market makers reject more orders)
• Market maker capacity (do not send more than they can handle)

This routing logic is a critical path component that must execute in < 1ms.

Component 3: Pre-Trade Risk Checks

Before any order reaches the FIX engine, it must pass risk checks. These run in < 10ms.

Checks:

1. Buying power: cash_balance + margin_available - open_order_value >= order_value. This must account for all pending (unfilled) orders, not just filled positions.

2. Position limits: Concentrated positions in a single stock exceed regulatory limits. Options have additional complexity (naked call selling requires higher margin).

3. Pattern Day Trader (PDT) rule: If the account has < $25,000 and has executed 4+ day trades in 5 business days, block the order. This is an SEC/FINRA regulation, not a business rule.

4. Order rate limiting: No user should place more than 100 orders per minute. This prevents runaway algorithms and fat-finger errors.

5. Price reasonableness: A limit order at $0.01 for a stock trading at $250 is probably an error. Flag and confirm with the user.

Implementation: Risk checks run against an in-memory cache of the user's account state (positions, cash, open orders). The cache is updated synchronously on every fill and order submission. It is NOT a database query -- 10ms budget does not allow a database round trip.

The buying power race condition (revisited): Two concurrent orders from the same user must be serialized through the risk check. Use a per-user lock (or better, a single-threaded event loop per user partition) to prevent two orders from both passing the buying power check when only one should succeed. This is the same single-writer pattern as the auction system.

Component 4: OHLCV Candlestick Computation

Candlestick charts show Open, High, Low, Close, Volume for each time interval (1 minute, 5 minutes, 1 hour, 1 day).

Computation pipeline:

1. Raw trade feed from exchange: (symbol, price, volume, timestamp) arriving at 80K events/sec.
2. For each symbol, maintain a running state per interval:

struct Candle {
    open: f64,      // first trade price in this interval
    high: f64,      // max price in this interval
    low: f64,       // min price in this interval
    close: f64,     // last trade price in this interval
    volume: u64,    // total shares traded in this interval
    interval_start: Timestamp,
}

1. On each trade event:
2. If trade.timestamp >= interval_start + interval_duration: finalize the current candle, emit it to storage, start a new candle with open = trade.price.

3. Otherwise: update high = max(high, trade.price), low = min(low, trade.price), close = trade.price, volume += trade.volume.

4. Finalized candles are written to a time-series database (TimescaleDB, InfluxDB, or just PostgreSQL with BRIN indexes on timestamp).

Scale: 8,000 symbols * 6 intervals (1m, 5m, 15m, 1h, 1d, 1w) = 48,000 active candle computations. Each is trivial (3 comparisons and 1 addition per trade). A single server handles this easily.

Historical data: 8,000 symbols * 252 trading days/year * 390 minutes/day * 6 intervals = ~4.7 billion candle records/year at ~50 bytes each = ~235 GB/year. TimescaleDB handles this comfortably with time-based partitioning.

Component 5: Market Data Conflation and Streaming

This is where the 20 GB/sec bandwidth problem is solved.

Conflation: Instead of pushing every tick to every client, the streaming gateway aggregates updates per symbol per client at a configurable rate.

Conflation algorithm per client per symbol:
  Maintain: latest_quote (bid, ask, last_price, volume)
  On new market data event:
    Update latest_quote (overwrite previous values)
  Every 250ms (or configurable per client):
    If latest_quote has changed since last push:
      Push latest_quote to client via WebSocket
      Mark as sent

Impact: A stock updating 100 times/sec is conflated to 4 updates/sec per client. Bandwidth drops from 10 KB/sec to 0.4 KB/sec per symbol per user. A user watching 10 symbols receives 4 KB/sec. 2 million users * 4 KB/sec = 8 GB/sec total -- still significant but manageable with a WebSocket gateway cluster.

Tier-based conflation:

• Premium/active trader: 250ms conflation (4 updates/sec)
• Standard user: 1 second conflation (1 update/sec)
• Background tab: 5 second conflation (0.2 updates/sec)

Implementation: The streaming gateway subscribes to a Redis Pub/Sub channel per symbol. Market data service publishes to these channels. Gateway servers buffer and conflate per-client before pushing to WebSocket. Clients specify which symbols they are watching via subscription messages.

Component 6: The GameStop/DTCC Incident

On January 28, 2021, Robinhood restricted buying of GameStop (GME) stock. This was not a system design failure -- it was a capital requirements crisis.

What happened:

1. GME volume spiked 1000x. Robinhood's daily orders went from 5M to 50M+.
2. The DTCC (Depository Trust & Clearing Corporation) calculates collateral requirements based on settlement risk.
3. Because T+2 settlement (now T+1) means Robinhood owes money for 2 days before receiving the shares, DTCC demanded $3.7 billion in additional collateral.
4. Robinhood did not have $3.7 billion in cash.
5. To reduce its DTCC collateral requirement, Robinhood restricted buying (reducing new settlement obligations).

System design lesson: The brokerage system must model and enforce DTCC collateral requirements in real time. A risk engine must compute:

total_unsettled_buy_value = sum of all filled buy orders not yet settled
dtcc_collateral_required = f(total_unsettled_buy_value, volatility, concentration)
available_collateral = cash_reserves + credit_lines

if dtcc_collateral_required > available_collateral * 0.8:
    ALERT: restrict high-volatility symbol buying
    REDUCE admission rate for new buy orders

This is a system-level risk check that operates above individual user accounts. It is the brokerage-level equivalent of a circuit breaker.

---

Deep Dives

Deep Dive 1: Order State Machine and Audit Trail

Every order goes through a state machine, and every transition is logged immutably.

States:

CREATED -> RISK_CHECK_PENDING -> RISK_CHECK_PASSED -> SUBMITTED ->
  -> ACKNOWLEDGED (by exchange) -> PARTIALLY_FILLED -> FILLED
  -> REJECTED (by exchange)
  -> CANCELLED (by user or system)
RISK_CHECK_PENDING -> RISK_CHECK_FAILED (insufficient buying power)

Audit trail: Each state transition is an immutable event:

{
  "order_id": "ORD12345",
  "event": "RISK_CHECK_PASSED",
  "timestamp": "2024-01-15T14:30:01.234Z",
  "details": {
    "buying_power_before": 10000.00,
    "order_value": 2500.00,
    "buying_power_after": 7500.00
  }
}

These events are written to an append-only log (Kafka topic with infinite retention, archived to S3). SEC Rule 17a-4 requires 6-year retention. The log is the source of truth; the orders table status column is a materialized view of the latest state.

Deep Dive 2: Margin Calculation and Regulation T

Margin lets users buy stocks with borrowed money. Regulation T (the Fed's rule) allows brokerages to lend up to 50% of a stock's purchase price.

Buying power calculation:

buying_power = cash_balance + (marginable_positions_value * margin_rate) - open_order_value

Example:
  Cash: $10,000
  Positions: $20,000 in AAPL (marginable at 50%)
  Margin available: $20,000 * 0.50 = $10,000
  Open orders: $5,000
  Buying power: $10,000 + $10,000 - $5,000 = $15,000

Maintenance margin: If the portfolio drops below the maintenance margin threshold (typically 25% of position value), the brokerage issues a margin call. The system must compute margin requirements continuously (not just at order time) and trigger margin calls automatically.

Implementation: A background job recalculates margin for all accounts every 5 seconds using the latest market prices. Accounts below maintenance threshold get an automatic "margin call" notification. If the user does not deposit funds or sell positions within the deadline (usually 2-5 business days), the system auto-liquidates positions to meet the margin requirement.

Deep Dive 3: Settlement, Clearing, and T+1

When a trade executes, the shares do not transfer immediately. Settlement takes T+1 (one business day, reduced from T+2 in May 2024).

Settlement flow:

1. T+0 (trade day): Order fills. Robinhood debits user's cash balance. Shares show in portfolio as "pending settlement."
2. T+0 (end of day): Robinhood sends trade details to its clearing firm (Robinhood Securities LLC, their in-house clearing arm).
3. T+0 (overnight): DTCC calculates net obligations. If Robinhood bought 1M shares of TSLA and sold 800K, the net obligation is 200K shares.
4. T+1 (next business day): Actual settlement. Cash moves from Robinhood's bank to the seller's bank. Shares move from the seller's custodian to Robinhood's custodian (DTC -- Depository Trust Company).

System design impact: Between T+0 and T+1, Robinhood has credit risk. The collateral requirement from DTCC covers this risk. The system must track:

• Unsettled buy obligations (cash owed)
• Unsettled sell obligations (shares owed)
• Net position per stock across all users
• DTCC collateral requirements (updated intraday)

This is a separate risk engine from the per-user pre-trade risk check. It operates at the firm level and is the system that triggered the GameStop crisis.

---

Alternative Designs

Approach	Pros	Cons	When to Use
FIX + in-memory risk engine (described above)	Sub-second fills. Real-time risk. Push-based execution reports.	FIX is complex. In-memory risk requires careful cache consistency.	Production brokerage. Robinhood, Schwab, Interactive Brokers.
REST API to exchange	Simpler protocol. Standard HTTP libraries.	Polling for fills. Higher latency. No sequence number guarantees.	Paper trading platforms. Prototypes.
Direct market access (DMA)	Lowest latency (sub-millisecond). Bypass broker routing logic.	Requires exchange membership. Expensive. Not retail-appropriate.	Hedge funds, HFT firms. Not relevant for a Robinhood-style platform.
Third-party execution API (Alpaca, DriveWealth)	Fully managed order execution. No FIX implementation.	Dependent on third party. Higher per-trade cost. Limited control.	Fintech startups building a brokerage without exchange connectivity.
Crypto exchange model	No clearing house. No T+1. Instant settlement on-chain.	Regulatory gray area. No SIPC protection. Different architecture entirely.	Crypto-only platforms. Not applicable to equities.

---

Scaling Math Verification

Market Data Pipeline

Raw exchange feed:           80,000 events/sec
Per-event processing:        ~10 us (update candle, publish to Redis)
CPU for raw processing:      80K * 10 us = 0.8 seconds of CPU/sec (one server)

Conflated output:
  Symbols actively watched:  3,000 (not all 8,000 are popular)
  Updates/sec per symbol:    4 (after 250ms conflation)
  Total outbound updates:    12,000/sec per gateway server
  Users per gateway server:  50,000
  Messages per server/sec:   50K * 10 symbols * 4/sec = 2M messages/sec
  Gateway servers needed:    2M users / 50K per server = 40 servers

Redis Pub/Sub channels:      3,000 (one per active symbol)
Redis throughput:            12,000 publishes/sec (well within single-node capacity)

Order Processing

Peak orders/sec:             10,000
Pre-trade risk check:        10 us per order (in-memory)
FIX message generation:      50 us per order
Network to market maker:     1 ms
Fill response:               10-100 ms
Total order processing:      10K/sec * 100 us = 1 second of CPU/sec (one server)

OMS servers:                 5 (for redundancy, not for capacity)
FIX sessions:                10 (spread across market makers)

Account State Cache

Active accounts:             5M DAU
Account state size:          ~1 KB (cash, margin, positions, open orders)
Total cache size:            5 GB (fits in memory on one server)
Cache updates/sec:           10K fills + 10K order submissions = 20K/sec
Each update:                 ~1 us (in-memory hash map update)

---

Failure Analysis

Failure	Impact	Mitigation
FIX connection to market maker drops	Cannot route orders to that market maker. Orders queue.	Maintain connections to 3+ market makers. Automatic failover to alternate routes. FIX sequence numbers ensure no lost messages on reconnect.
Market data feed goes down	Users see stale prices. Orders placed at wrong prices.	Multiple feed sources (direct exchange feed + consolidated tape). Stale data detection: if no update for 5 seconds, show "delayed" badge.
Pre-trade risk engine has stale cache	Buying power calculation is wrong. Users can place orders that exceed their balance.	Synchronous cache update on every fill. If cache and DB disagree, reject the order and refresh cache. Conservative approach: if uncertain, reject.
Order fills but confirmation lost	User does not see fill. Places duplicate order. Double position.	FIX sequence numbers guarantee delivery. If execution report is lost, resend request recovers it. Idempotency key on order submission prevents duplicates.
DTCC collateral spike	Brokerage must restrict trading to reduce settlement obligations. GameStop scenario.	Monitor collateral requirements in real time. Maintain $1B+ in reserve capital. Pre-negotiate credit lines with banks. Restrict trading as last resort.
Database goes down	Cannot persist orders or account updates.	Write-ahead log to Kafka. Database is recovered from Kafka event stream. Risk engine operates from in-memory cache during outage.
Fat finger order (user buys 100,000 shares instead of 100)	User loses money. Regulatory scrutiny.	Order size validation (warn if order > 5x typical size). Confirmation screen for large orders. Cancel/correct within 10 seconds.

---

Level Expectations

Level	What the Interviewer Expects
Mid (L4)	REST API for orders. Database for positions and orders. Basic buy/sell flow. Mentions market data streaming. Knows about buy/sell balance checking. Simple WebSocket for prices.
Senior (L5)	FIX protocol for exchange communication (push-based fills). Pre-trade risk checks with in-memory account state. OHLCV candle computation pipeline. Conflation for market data bandwidth. Order state machine with audit trail. Distinction between brokerage and exchange.
Staff+ (L6)	PFOF routing logic and its revenue implications. DTCC collateral requirements and the GameStop lesson. T+1 settlement and its system design impact. Regulation T margin calculations. Firm-level risk engine separate from per-user risk. Market data tiered conflation. Reference to actual FIX message types and sequence number recovery.

---

References from Our Courses

• Kafka Partitions and Ordering — event streaming for trade execution and market data
• Time-Series Databases — storing tick-level price history and OHLCV data
• Exactly-Once and Transactions — preventing duplicate trade executions

---

Red Team This Design

Ready to stress-test this architecture? The Attack companion tears apart every decision in this design — from hardware physics to security holes to what actually happens at 10x scale.

Attack: Design a Stock Brokerage Platform →