AI Co-Pilot Assistant
Technical architecture of the generative AI strategy assistant — prompt hydration with live market context, multimodal image ingestion, multi-provider abstraction (Google Gemini, OpenAI, Qwen, OpenRouter, Vertex Agent Builder), response validation, and security filtering.
The AIAssistant (api/ai_assistant.py, ~3,760 lines) allows users to generate strategy configurations from plain-text descriptions or chart screenshots. It acts as a bridge between the visual editor frontend and Large Language Models (Google Gemini / OpenAI / Qwen / OpenRouter / Vertex Agent Builder), transforming natural language into executable strategy JSON.
AI Ingestion Architecture
The Co-Pilot service accepts text and media assets, enriches them with real-time market data, and requests structured JSON outputs from the LLM.
Provider Selection
Sources:The provider is chosen by the AI_PROVIDER environment variable (or by admin settings persisted in the database) and validated against the supported set — Google's direct GenAI SDK, OpenAI, Qwen, OpenRouter, or Vertex Agent Builder (_get_active_ai_provider(), lines 748–770).
Real-Time Context Hydration
To prevent the LLM from generating obsolete or irrelevant strategy recommendations, DepthSight "hydrates" the user's prompt with real-time telemetry before sending the query.
enrich_market_context_for_ai() (lines 566–634)
The function:
- Parses the user prompt for ticker symbols matching
\b[A-Z0-9]{2,10}USDT\b. - Filters out stop-words (
LONG,SHORT,GRID,DCA,AND,OR). - Queries the Screener API (
http://localhost:8050/api/v1/metrics/{symbol}) for up to 2 detected symbols:
The resulting context block is injected into the LLM prompt, providing the AI with current market conditions to base its strategy recommendations on.
Context Content Injected
| Data Point | Source | Purpose |
|---|---|---|
| Last Price | Screener API | Entry/exit level calibration |
| NATR (Volatility) | Screener API | Stop-loss distance suggestion |
| Macro Trend | Screener API | Directional bias alignment |
| Oracle Regime | Screener API | Risk mode adjustment |
| 24h Volume | Screener API | Liquidity assessment |
Image-to-Strategy Ingestion
Gemini and modern multimodal models can read image pixels directly. When a user uploads a screenshot of a trading setup:
1. Image Normalization (lines 1339–1356)
Sources:The image is stripped of any data: URI prefix and paired with a normalized MIME type (defaulting to image/jpeg); the result is a (data, mime_type) tuple.
2. Provider-Specific Attachment
Google Gemini (lines 1373–1381): Converts base64 to bytes via base64.b64decode() and attaches as types.Part.from_bytes(data=..., mime_type=...).
OpenRouter (lines 1503–1511): Embeds as a {"type": "image_url"} content part whose URL is a data:{mime};base64,{data} URI.
3. Visual Parsing
The system prompt instructs the model to:
- Extract technical indicators visible on the screen (Bollinger Bands, RSI, MACD).
- Identify chart structures (ascending triangle, breakout levels, moving average crosses).
- Map visual patterns to mathematical block JSON nodes:
- Ascending triangle →
local_level+level_touch_analyzer+price_action_analyzer. - RSI divergence →
rsi_conditionwith specific threshold + trend filter.
- Ascending triangle →
4. JSON Mapping
The model generates block JSON nodes mapped to the visual builder canvas schema, which the frontend renders as draggable logic blocks.
Response Validation
LLMs can suffer from hallucinations, producing invalid JSON or referencing nonexistent logic blocks. DepthSight implements multi-layer validation:
Layer 1 — Python Code Detection (lines 523–562)
A critical security filter scans for Python code injection:
Sources:In Advisor mode, if Python code is detected in the text response, it is blocked and a safe message is returned (api/ai_assistant.py:2455-2465).
Layer 2 — JSON Extraction (lines 1942–1948)
If the AI wraps JSON in markdown or explanatory text, the first {...} block is extracted via str.find("{") / str.rfind("}"):
Layer 3 — Block Schema Validation
Generated strategies are checked against the user's plan restrictions (pro_only / kline_only blocks are rejected for ineligible users, api/ai_assistant.py:440-451); structurally unknown blocks are repaired by default-parameter injection rather than rejected.
Layer 4 — Parameter Default Injection (lines 244–301)
Missing parameters are merged with defaults via _ensure_default_params():
The recursive walk also assigns a missing block id (lines 254–257) and migrates legacy block types (lines 259–269). This prevents bot engine crashes from missing configuration fields.
Layer 5 — Pydantic Validation
The final JSON is validated against schemas.StrategyV2ConfigData.model_validate() (lines 2179–2181), ensuring type correctness.
Full Query Flow
Generator Mode (get_chat_response(), line 2002)
Used for strategy generation from text/screenshots:
- Achievement Grant (line 2014): Unlocks "used_ai_assistant" achievement.
- Quota Check (lines 2029–2034):
QuotaManager.check_and_consume("use_ai_assistant"). - Chat History Retrieval: Loads full session history; the modification path scans it for the previous advisor recommendation, and the scratch-generation path feeds the last 6 messages to the model.
- Modification vs Generation: Detects if the user is asking to modify an existing strategy or create a new one.
- RAG Injection: Calls
enrich_market_context_for_ai(). - Tier Context: Informs the AI about the user's plan restrictions.
- Backtest Context: If a
backtest_idis provided, appends backtest analytics to the prompt. - LLM Call:
_generate_json_response()with structured JSON output. - Validation Pipeline: JSON extraction → schema validation → default injection → Pydantic validation.
- Return:
schemas.AIChatResponsewithstrategy_json.
Advisor Mode
Used for strategy analysis and questions:
- Chat history retrieval.
- RAG injection with current market context.
- Tier context.
- Backtest Analytics: Parses decision traces into combination/individual foundation stats, best/worst trades.
- Real Trade Analytics: If available, includes live trade performance.
- LLM Call:
_generate_text_response()for natural language output. - Python Code Security Check: Scans for code injection.
- Return:
AIChatResponsewithtext_response.
WebSocket Real-Time Events
Technical details of the ASGI WebSocket server — connection lifecycle, JWT authentication, channel authorization via regex patterns, Redis Pub/Sub stream forwarding, and graceful cleanup.
ML Pipeline & Oracle
Detailed analysis of the Gaussian Mixture Model (GMM) market regime classification, three-sensor feature engineering, live prediction with caching, and regime-aware trading exits.