ML Pipeline & Oracle

The AI Oracle (bot_module/oracle.py, ~132 lines) is a machine learning engine that classifies the current market phase into distinct "regimes" (e.g., Calm Range, Volatile Trend, Extreme Shock). Strategies use this state to change execution parameters dynamically — tightening stop-losses during high-volatility regimes or increasing position sizes during calm trending markets.

---

GMM Market Regime Classification

DepthSight uses an unsupervised Gaussian Mixture Model (GMM) from scikit-learn to cluster market states. Unlike raw price action, GMM analyzes the underlying statistical distributions of the market, identifying hidden structural patterns.

Model Loading

The Oracle loads a pre-trained GMM model once at instantiation (lines 14–23):

Sources: ml_pipeline.py

The model is trained offline (via train_oracle.py, referenced on line 28) and distributed as a .joblib file. No online training occurs — the model is purely for inference.

---

The Three Sensors

The GMM model does not look at prices directly. It is fed three custom features (called "sensors") computed by engineer_features() (lines 25–81). These sensors capture different dimensions of market behavior:

Sensor 1: Memory (Market Memory / Forgetting Speed)

Measures the ratio of short-term to long-term volatility:

Sources: ml_pipeline.py

Value	Interpretation
< 1.0	Current volatility is below long-term average (calm)
~ 1.0	Normal market conditions
> 1.0	Volatility spike — transition to high-panic regime

Purpose: Detects volatility regime changes. A sudden jump from 0.8 to 2.5 signals a market panic event (flash crash, news reaction).

Sensor 2: News Background (News Sentiment Asymmetry)

Averages sentiment signals over a 12-hour window (720 candles on 1m timeframe):

Sources: ml_pipeline.py

Value	Interpretation
> 0.0	Net positive sentiment (bullish bias)
~ 0.0	Neutral / mixed sentiment
< 0.0	Net negative sentiment (bearish bias)

Purpose: Captures macro-sentiment shifts that often precede sustained trends. A news-aware strategy can avoid buying during negative sentiment spikes.

If sentiment columns are not available (e.g., the data source doesn't provide them), the sensor falls back to 0.0 gracefully.

Sensor 3: Complexity (Complexity Drift / Normalized ATR)

Measures normalized ATR — how much price drift or noise is occurring relative to the absolute price level:

Sources: ml_pipeline.py

Value	Interpretation
Low (0.001–0.005)	Tight range, low noise (ideal for scalping)
Medium (0.005–0.02)	Normal market drift
High (> 0.02)	High noise, erratic movements (tighten SL)

Purpose: Evaluates market "friction" — high complexity regimes warrant wider stop-losses to avoid noise-triggered exits, while low complexity regimes allow tighter risk management.

Feature Engineering Output

The three sensors are combined into a feature DataFrame:

Sources: ml_pipeline.py

Infinite and NaN values are sanitized to ensure the GMM model always receives valid input.

---

Live Prediction & Caching

The Oracle evaluates regimes at the end of each closed candle via get_current_regime() (lines 83–132):

Sources: ml_pipeline.py

Caching Strategy

Component	Behavior
Cache Key	_last_kline_timestamp — the index of the most recent candle
Cache Hit	Returns cached (regime, confidence) tuple immediately
Cache Miss	Re-runs feature engineering + GMM inference, updates cache
Empty Data	Returns (-1, 0.0) — signals "unknown regime"
NaN Guard	If features produce NaN after sanitization, returns (-1, 0.0)

This optimization ensures the CPU-intensive GMM inference runs at most once per candle, even when hundreds of strategies query the Oracle simultaneously.

---

Regime-Aware Strategy Execution

How Strategies Consume Oracle Output

The Oracle's output (regime_index, confidence_pct) is stored in StrategyConfig at the database level:

Column	Type	Description
oracle_regime	Integer (nullable)	Current GMM regime index (0 to N-1)
oracle_confidence	Float (nullable)	Confidence percentage of the prediction

Strategies can reference the Oracle state through two mechanisms:

1. Oracle Filter Block: A visual builder block that gates entry conditions — only allow trades when oracle_regime in allowed_regimes.
2. Risk Manager Integration: The RiskManager can detect regime transitions and trigger a Regime Change Exit, liquidating positions if the market moves to a prohibited regime (e.g., from "Normal" to "Extreme Shock").

Regime Change Exit

When the RiskManager detects a GMM regime transition to a prohibited state:

Normal (Regime 2) → Extreme Shock (Regime 4)
    ↓
RiskManager triggers: REGIME_CHANGE_EXIT
    ↓
TradingController._close_position(position, "REGIME_CHANGE_EXIT")
    ↓
All active positions liquidated at market

This serves as a catastrophic risk safeguard — if the ML model detects that market structure has fundamentally changed, all exposure is removed before losses compound.

---

Model File Management

Aspect	Implementation
Training	Offline via train_oracle.py (separate script)
Serialization	joblib.dump() / joblib.load()
Storage	data/models/oracle_model.joblib
Versioning	File-based — replace the .joblib file and restart
Error Handling	FileNotFoundError raised if model path does not exist

The model is pre-loaded during the FastAPI lifespan to eliminate cold-start latency on the first prediction request.