← Back to Learn
III AdvancedWeek 28 • Lesson 77Duration: 45 min

LIVE Real-Time Trading Systems

Live trading is a fundamentally different challenge — and it will humble you

Learning Objectives

  • Design event-driven trading systems
  • Handle real-time data feeds and order management
  • Understand latency requirements for different strategies

Explain Like I'm 5

A live trading system is fundamentally different from a backtest. In a backtest, you process historical data sequentially — nice and calm. In live? Data arrives unpredictably. New bars drop. Ticks stream in. Orders fill (or don't). Connections die. Your system must react to EVENTS as they happen, handle errors without crashing, and never miss a signal. The first time you run a live system and watch it make real decisions with real money, it's a qualitatively different experience.

Think of It This Way

A backtest is like watching a recorded game and calling plays from your couch. Easy — you can pause, rewind, take your time. Live trading is like actually playing the game in real-time with real consequences. You must make instant decisions, handle unexpected situations, and there's no pause button.

1Event-Driven Architecture

The core of a live trading system is an event loop. Here's what it does every cycle: 1. Wait for new bar close 2. Fetch latest OHLCV data 3. Compute features (your 30-40 input features) 4. Run model predictions per symbol 5. Apply decision gate for qualifying signals 6. Execute qualifying trades 7. Update exit management for existing positions 8. Check risk limits (DD zones, daily limits) 9. Log everything 10. Sleep until next bar For M15 timeframe, this cycle runs every 15 minutes. Not exactly "high frequency." But it still must be: - Reliable: handle connection drops, data errors, partial fills - Deterministic: same inputs -> same outputs (no random behavior in production) - Auditable: log every decision for post-trade analysis - Failsafe: if anything goes wrong, default to NO TRADE (safe state) The failsafe principle is the most important. A missed trade costs you one opportunity. A bad trade costs you real money.

2Error Handling — Things WILL Go Wrong

Production trading is essentially error handling with some trading logic sprinkled in. Plan for ALL of these: Connection loss: Broker connection drops. -> Retry with exponential backoff (1s, 2s, 4s, 8s...) -> Don't open new trades without confirmed connection -> Existing positions have SL/TP set server-side (persist even if you disconnect) -> This is why server-side stops are non-negotiable Data errors: Bad tick, missing bar, stale data. -> Validate all data before using -> Skip the bar if data is suspicious -> Missing one signal is better than trading on bad data Order rejection: Broker rejects your order (insufficient margin, invalid price, market closed). -> Log the rejection with reason -> Do NOT retry automatically (market conditions may have changed) -> Alert the operator -> Your backtest assumed every order fills — reality disagrees Model error: Prediction returns NaN or unexpected value. -> Log the error with full context (features, model version) -> Skip the trade -> Investigate offline — don't try to "fix" it in the live loop System crash: Unexpected exception in the main loop. -> try/except around the main loop (the loop MUST survive) -> Restart with state recovery -> Positions are safe (SL/TP are server-side)

3State Management — The Hidden Complexity

Here's something that makes live trading substantially harder than backtesting: STATE. In a backtest, you start from scratch every time. Clean slate. In live trading, your system has state that persists: What state do you need to track? - Current open positions (symbols, sizes, entry prices, SL/TP) - Today's PnL (for daily risk limits) - Current drawdown zone (for risk scaling) - Pending orders (waiting to fill) - Model predictions made today (for audit) - Connection status and reconnection attempts The hard part: What happens when your system restarts? - It must recover ALL state from persistent storage or the broker - Query broker for open positions -> reconcile with local state - Query trade history for today's PnL - Recalculate drawdown zone from current balance State corruption is a real risk: - Your local state says you have 3 positions - Broker says you have 4 (one filled while you were disconnected) - Which is correct? Always trust the broker State management is the reason "just restart it" doesn't work for trading systems like it does for web servers.

4The Shadow Trading Pattern

Before you go live with real money, there's a critical intermediate step: shadow trading (also called paper trading with your live system). How it works: 1. Your system connects to live data feeds (real market data) 2. It generates signals using your real models 3. It simulates trades (records what WOULD have happened) 4. It does NOT send actual orders to the broker 5. You compare shadow results to backtest expectations What shadow trading reveals: - Actual latency in your pipeline - Real slippage estimates - Data quality issues you missed in backtest - System stability over days/weeks of continuous operation - Memory leaks, connection drops, edge cases Minimum shadow period: 2-4 weeks. Anything less is irresponsible. What to look for: - Win rate within 5% of backtest? [PASS] - No system crashes? [PASS] - Handles weekends/holidays correctly? [PASS] - Recovers from connection drops? [PASS] - Memory usage stable over time? [PASS] If shadow results are significantly worse than backtest, you have a bug — not a bad strategy. Fix the bug before going live.

Shadow Trading Timeline: Confidence Building

5Graceful Shutdown

One of the most overlooked aspects of live trading: how do you STOP? Wrong way: Ctrl+C, kill the process, close the terminal. -> Open orders might not be cancelled -> State not saved -> Positions orphaned with only server-side SL/TP protection Right way: Graceful shutdown sequence: 1. Catch the shutdown signal (SIGTERM, Ctrl+C) 2. Stop accepting new signals 3. Cancel any pending (unfilled) orders 4. Verify all open positions have SL/TP set 5. Save current state to disk (for recovery on restart) 6. Log the shutdown event with full state summary 7. Close broker connection cleanly 8. Exit This should take 2-5 seconds, not be instantaneous. Also important: test your shutdown path. Actually test it. Start the system, let it run, then Ctrl+C and verify everything cleaned up properly. You'd be surprised how many systems have never had their shutdown path tested.

Hands-On Code

Event-Driven Trading Loop

python
import time
import logging

logger = logging.getLogger('v7_live')

class LiveTradingLoop:
    """Main live trading loop with error handling."""
    
    def __init__(self, engine, connector, bar_interval_sec=900):
        self.engine = engine
        self.connector = connector
        self.interval = bar_interval_sec
        self.running = False
    
    def start(self):
        self.running = True
        logger.info("V7 Live Trading STARTED")
        
        while self.running:
            try:
                cycle_start = time.time()
                
                # 1. Check connection
                if not self.connector.is_connected():
                    logger.warning("Connection lost, reconnecting...")
                    self.connector.connect()
                    continue
                
                # 2. Process signals
                signals = self.engine.scan_all_symbols()
                
                for signal in signals:
                    # 3. Risk check
                    if not self.engine.risk_check(signal):
                        logger.info(f"SKIP {signal['symbol']}: risk limit")
                        continue
                    
                    # 4. Execute
                    result = self.connector.place_order(
                        symbol=signal['symbol'],
                        direction=signal['direction'],
                        volume=signal['volume'],
                        sl=signal['sl'],
                        tp=signal['tp'],
                    )
                    logger.info(f"ORDER: {result}")
                
                # 5. Update existing positions
                self.engine.update_exits()
                
                # 6. Log state
                self.engine.log_state()
                
                # 7. Sleep until next bar
                elapsed = time.time() - cycle_start
                sleep_time = max(0, self.interval - elapsed)
                time.sleep(sleep_time)
                
            except KeyboardInterrupt:
                self.running = False
                logger.info("Manual stop requested")
            except Exception as e:
                logger.error(f"Error in trading loop: {e}", exc_info=True)
                time.sleep(60)  # wait before retrying

The live loop must handle EVERYTHING: connection issues, data errors, order rejections, and crashes. The golden rule: when in doubt, do nothing. Missing a trade is better than a bad trade. The best live trading systems are 80% error handling and 20% actual trading logic.

Knowledge Check

Q1.Your live trading system loses connection to the broker. What should happen to existing positions?

Assignment

Implement a basic event-driven trading loop with: connection monitoring, error handling, logging, and graceful shutdown. Test it by simulating connection drops and data errors.