III AdvancedWeek 16 • Lesson 48Duration: 50 min

RGM Regime Detection

Markets don't behave the same way all the time — learning to identify the current state

Learning Objectives

•Understand why regime detection is critical for adaptive trading
•Implement Hurst exponent for trend vs. mean-reversion identification
•Use clustering to identify distinct market regimes
•Evaluate and avoid common regime detection pitfalls

Explain Like I'm 5

Markets alternate between different modes — trending, choppy, volatile, calm. Regime detection tries to figure out which mode the market is in right now, so your strategy can adapt.

Think of It This Way

It's like a weather system. You don't use the same clothing strategy for summer and winter. Regime detection is your market thermometer — it tells you whether to prepare for sunshine or storms.

1Why Regime Detection is a Meta-Signal

Regime detection isn't a signal in the traditional sense — it's a meta-signal that modulates how you use your actual signals. The insight is simple but powerful: Most signals work in some regimes and fail in others. A momentum signal thrives in trending markets and gets destroyed in mean-reverting ones. A mean-reversion signal is the opposite. If you could identify the current regime before deploying capital, you'd route signals to the regimes where they work. This is why regime detection sits upstream of signal generation in a well-designed system. It's the routing layer that decides which models get activated and how much risk they're allocated. The three core regime dimensions: 1. Trend vs. Mean-Reversion — Is the market directional or oscillating? The Hurst exponent is the standard tool here. 2. Volatility Regime — High, medium, or low realized vol? This affects position sizing, stop placement, and signal thresholds. 3. Correlation Regime — Are assets moving together (risk-on/risk-off) or independently? This affects diversification benefits and portfolio construction. Most production systems track all three simultaneously, because they're semi-independent dimensions.

2The Hurst Exponent

The Hurst exponent (

H

) measures the degree of persistence in a time series: -

H &gt; 0.55

: Trending behavior (momentum signals should work) -

H \approx 0.50

: Random walk (no exploitable structure) -

H &lt; 0.45

: Mean-reverting behavior (mean-reversion signals should work) The classic estimation method is rescaled range (R/S) analysis:

H = \frac{\log(R/S)}{\log(n)}

Where

R/S

is the rescaled range of cumulative deviations from the mean, computed over windows of different sizes

n

3Regime Clustering

Rather than defining regimes manually, you can let the data tell you what distinct states exist. K-Means clustering on market features is a common approach. Typical features for regime clustering: - Realized volatility (20-day, 60-day) - Return autocorrelation (1-lag, 5-lag) - Hurst exponent - ADX (trend strength) - Cross-asset correlation - VIX level and term structure With K=3 or K=4 clusters, you typically discover regimes that map to intuitive market states: | Cluster | Characteristics | Common Label | |---------|----------------|--------------| | 1 | Low vol, positive autocorr, moderate ADX | Quiet Trend | | 2 | High vol, low autocorr, high ADX | Volatile Trend | | 3 | Low vol, negative autocorr, low ADX | Range-Bound | | 4 | High vol, high correlation, extreme readings | Crisis | The beauty of clustering is that it's unsupervised — you're not imposing your assumptions about what regimes should look like.

4ADX and Trend Strength

The Average Directional Index (ADX) is a simpler, more responsive measure of trend strength than Hurst: - ADX < 20: Weak or no trend (range-bound, mean-reversion territory) - ADX 20-30: Developing trend (transitional) - ADX 30-50: Strong trend (momentum signals should perform) - ADX > 50: Very strong trend (often near exhaustion) ADX has the advantage of being fast to compute and responsive to recent price action. Its disadvantage is that it measures trend strength but not direction. Combining Hurst and ADX gives you a richer picture: - High Hurst + High ADX = strong, persistent trend - High Hurst + Low ADX = early trend development - Low Hurst + Low ADX = range-bound - Low Hurst + High ADX = contradictory signal — reduce exposure

5Autoencoder Regime Detection

A more sophisticated approach uses autoencoders — neural networks that compress market features into a low-dimensional representation, then reconstruct them. The encoded representation captures the essential "state" of the market. Why autoencoders work for regime detection: 1. Nonlinear dimensionality reduction. Unlike PCA, autoencoders can capture nonlinear relationships between features. 2. Anomaly detection. When reconstruction error is high, the market is in an unusual state. This is itself a useful signal. 3. Smooth transitions. The encoded representation changes continuously, capturing gradual regime shifts that discrete clustering misses. Architecture for market regime autoencoder: - Input: 10-20 market features (vol, autocorrelation, spread ratios, ADX, etc.) - Encoder: 2-3 layers compressing to 3-5 latent dimensions - Decoder: mirror of encoder - Training: minimize reconstruction error on rolling 2-year windows - Clustering: apply K-Means to the latent space

6Regime Detection Pitfalls

Regime detection is powerful but treacherous. Five common failures: 1. Hindsight labeling. Looking at a chart and saying "obviously that was a trending regime" is trivial. Identifying it in real-time with enough lead time to act is the challenge. Your detector must use only backward-looking data. 2. Regime persistence illusion. Just because a regime has lasted 3 months doesn't mean it will last another month. Transitions can be sudden. 3. Too many regimes. More clusters doesn't mean better — it means overfitting. Start with 2-3 regimes and demonstrate they're stable before adding complexity. 4. Look-ahead in feature standardization. If you z-score your clustering features using the full sample mean and standard deviation, you've built a regime detector that knows the future. Use expanding-window or rolling-window standardization. 5. Ignoring transition costs. Every time your regime detector flips, you may need to change positions. If it flips frequently, the transaction costs and whipsaw losses can exceed any regime-conditional improvement. Hamilton (1989) introduced Markov-switching models for regime detection. Ang & Bekaert (2002) extended this to asset allocation. Nystrup et al. (2017) showed that regime-based allocation can improve risk-adjusted returns by 15-30% when done carefully.

Key Formulas

Hurst Exponent (R/S Method)

H > 0.5 indicates trending behavior, H < 0.5 indicates mean-reversion, H ~ 0.5 is a random walk

Hands-On Code

Hurst Exponent and Regime Detection

python

import numpy as np

def compute_hurst(prices, max_lag=100):
    """Compute Hurst exponent using R/S analysis."""
    returns = np.diff(np.log(prices))
    lags = range(10, max_lag)
    rs_values = []
    
    for lag in lags:
        rs_list = []
        for start in range(0, len(returns) - lag, lag):
            chunk = returns[start:start + lag]
            mean = np.mean(chunk)
            cumdev = np.cumsum(chunk - mean)
            R = np.max(cumdev) - np.min(cumdev)
            S = np.std(chunk, ddof=1)
            if S > 0:
                rs_list.append(R / S)
        if rs_list:
            rs_values.append((np.log(lag), np.log(np.mean(rs_list))))
    
    if len(rs_values) < 3:
        return 0.5
    
    rs_arr = np.array(rs_values)
    coeffs = np.polyfit(rs_arr[:, 0], rs_arr[:, 1], 1)
    return coeffs[0]

def detect_regime(prices, lookback=252):
    """Classify current market regime using multiple indicators."""
    returns = np.diff(np.log(prices[-lookback:]))
    hurst = compute_hurst(prices[-lookback:])
    vol = np.std(returns) * np.sqrt(252)
    autocorr = np.corrcoef(returns[:-1], returns[1:])[0, 1]
    
    if hurst > 0.55 and vol < 0.15:
        regime = 'quiet_trend'
    elif hurst > 0.55 and vol >= 0.15:
        regime = 'volatile_trend'
    elif hurst < 0.45:
        regime = 'mean_reverting'
    else:
        regime = 'random_walk'
    
    return {
        'regime': regime,
        'hurst': round(hurst, 3),
        'annualized_vol': round(vol, 4),
        'autocorrelation': round(autocorr, 4)
    }

Computes the Hurst exponent using rescaled range analysis and classifies the current market regime based on Hurst, volatility, and autocorrelation indicators.

Knowledge Check

Q1.A Hurst exponent of 0.38 suggests the market is:

Q2.What is the primary advantage of unsupervised clustering for regime detection?

Assignment

Compute the rolling 120-day Hurst exponent for three FX pairs (EUR/USD, GBP/USD, USD/JPY) over 3+ years. Identify periods where Hurst > 0.55 and Hurst < 0.45. Backtest a simple strategy: use momentum when Hurst > 0.55 and mean-reversion when Hurst < 0.45. Compare to a non-regime-conditional version.

← Previous Lesson Next Lesson →