AI and ML Integration

ML Pipeline & Oracle

Detailed analysis of the Gaussian Mixture Model (GMM) market regime classification, three-sensor feature engineering, live prediction with caching, and regime-aware trading exits.

โฑ๏ธ 7 min read๐Ÿ“Š Level: Advanced

The AI Oracle (bot_module/oracle.py, ~132 lines) is a machine learning engine that classifies the current market phase into distinct "regimes" (e.g., Calm Range, Volatile Trend, Extreme Shock). Strategies use this state to gate entries (oracle filter block), filter tradable symbols (DYNAMIC_ORACLE mode), and react to regime changes (breakeven / loss-cut exits).

Beyond the offline GMM, the platform ships River-based online learning (bot_module/model_pipeline.py, bot_module/ml_strategy.py), a Compass strategy that consumes Oracle regimes (bot_module/compass_strategy.py), and Celery jobs tasks.generate_dataset / tasks.train_model (tasks.py) for dataset generation and model training.


GMM Market Regime Classification

DepthSight uses an unsupervised Gaussian Mixture Model (GMM) from scikit-learn to cluster market states. Unlike raw price action, GMM analyzes the underlying statistical distributions of the market, identifying hidden structural patterns.

Rendering diagram...

Model Loading

The Oracle loads a pre-trained GMM model once at instantiation (lines 14โ€“23):

Sources:

The model is trained offline (the training script train_oracle.py is referenced in a comment at bot_module/oracle.py:28 but is not part of this repository) and distributed as a .joblib file. No online training occurs in the Oracle โ€” the model is purely for inference.


The Three Sensors

The GMM model does not look at prices directly. It is fed three custom features (called "sensors") computed by engineer_features() (lines 25โ€“81). These sensors capture different dimensions of market behavior:

Sensor 1: Memory (Market Memory / Forgetting Speed)

Measures the ratio of short-term to long-term volatility:

Sources:
ValueInterpretation
< 1.0Current volatility is below long-term average (calm)
~ 1.0Normal market conditions
> 1.0Volatility spike โ€” transition to high-panic regime

Purpose: Detects volatility regime changes. A sudden jump from 0.8 to 2.5 signals a market panic event (flash crash, news reaction).

Sensor 2: News Background (News Sentiment Asymmetry)

Averages sentiment signals over a 12-hour window (720 candles on 1m timeframe):

Sources:
ValueInterpretation
> 0.0Net positive sentiment (bullish bias)
~ 0.0Neutral / mixed sentiment
< 0.0Net negative sentiment (bearish bias)

Purpose: Captures macro-sentiment shifts that often precede sustained trends. A news-aware strategy can avoid buying during negative sentiment spikes.

If sentiment columns are not available (e.g., the data source doesn't provide them), the sensor falls back to 0.0 gracefully.

Sensor 3: Complexity (Complexity Drift / Normalized ATR)

Measures normalized ATR โ€” how much price drift or noise is occurring relative to the absolute price level:

Sources:
ValueInterpretation
Low (0.001โ€“0.005)Tight range, low noise (ideal for scalping)
Medium (0.005โ€“0.02)Normal market drift
High (> 0.02)High noise, erratic movements (tighten SL)

Purpose: Evaluates market "friction" โ€” high complexity regimes warrant wider stop-losses to avoid noise-triggered exits, while low complexity regimes allow tighter risk management.

Feature Engineering Output

The three sensors are combined into a feature DataFrame:

Sources:

The explicit .copy() detaches the feature frame from the source DataFrame, and infinite / NaN values are sanitized to ensure the GMM model always receives valid input.


Live Prediction & Caching

The Oracle evaluates regimes at the end of each closed candle via get_current_regime() (lines 83โ€“132):

Sources:

Caching Strategy

ComponentBehavior
Cache Key_last_kline_timestamp โ€” the index of the most recent candle
Cache HitReturns cached (regime, confidence) tuple immediately
Cache MissRe-runs feature engineering + GMM inference, updates cache
Empty DataReturns (-1, 0.0) โ€” signals "unknown regime"
NaN GuardIf features produce NaN after sanitization, returns (-1, 0.0)

This optimization ensures the CPU-intensive GMM inference runs at most once per candle, even when hundreds of strategies query the Oracle simultaneously.


Regime-Aware Strategy Execution

How Strategies Consume Oracle Output

The Oracle's output (regime_index, confidence_pct) is stored in StrategyConfig at the database level:

ColumnTypeDescription
oracle_regimeInteger (nullable)Current GMM regime index (0 to N-1)
oracle_confidenceFloat (nullable)Confidence percentage of the prediction

Strategies can reference the Oracle state through two mechanisms:

  1. Oracle Filter Block: A visual builder block that gates entry conditions โ€” trades are only allowed when oracle_regime equals the required regime and oracle_confidence meets the configured minimum (bot_module/strategy.py:4670-4702).
  2. Regime-Change Reaction: when the breakeven_on_regime_change option is enabled, Strategy.check_on_screener_update() (bot_module/strategy.py:3230-3314) compares the position's entry regime with the screener's current oracle_regime: winning positions get their stop moved to breakeven, and losing positions are closed by TradingController.

Regime Change Exit

When the breakeven_on_regime_change option is enabled, a regime transition triggers a per-position reaction:

Regime flip detected on screener update (breakeven_on_regime_change = true)
    โ†“
Winning trade  โ†’ stop moved to breakeven (MOVE_SL)
Losing trade   โ†’ TradingController.close_position(reason="REGIME_CHANGE_LOSS_CUT")

This is a per-position safeguard, not a blanket liquidation: when the regime flips under a losing trade, that position is cut; winning positions are protected by a breakeven stop.


Model File Management

AspectImplementation
TrainingOffline (training script not included in this repository; referenced from bot_module/oracle.py:28)
Serializationjoblib.dump() / joblib.load()
Storagedata/oracle_model.joblib (ORACLE_MODEL_PATH, bot_module/config.py:890)
VersioningFile-based โ€” replace the .joblib file and restart
Error HandlingFileNotFoundError raised if model path does not exist

The model is pre-loaded during the FastAPI lifespan to eliminate cold-start latency on the first prediction request.