Trading Controller Lifecycle
Detailed analysis of the initialization, state management, position lifecycle, event processing loop, and graceful shutdown inside the core TradingController.
The TradingController (bot_module/controller.py, ~19,000 LOC) is the central nervous system of the DepthSight bot execution engine. It orchestrates incoming market data, runs strategy instances, validates every signal through the RiskManager, and dispatches orders to exchange executors — all within a single asynchronous event loop.
Critical Structural Components
When initialized, the constructor (init, lines 471–746) establishes these critical structures to coordinate asynchronous processing and ensure thread safety across hundreds of concurrently running strategies:
| Component | Type | Purpose |
|---|---|---|
| event_queue | asyncio.Queue(maxsize=1000) | Central bounded event bus for incoming market data and order updates |
| _event_handler_semaphore | asyncio.Semaphore | Limits concurrent event handlers (config.MAX_PARALLEL_EVENT_HANDLERS, default 64) to prevent thread exhaustion |
| _positions_dict_lock | asyncio.Lock | Protects structural integrity of the ActivePositionMap dictionary |
| _symbol_locks | Dict[str, asyncio.Lock] | Per-symbol locks ensuring sequential order processing on the same coin |
| _active_positions | ActivePositionMap | Thread-safe, market-aware runtime storage for active positions |
| running_strategy_instances | Dict[str, Tuple[BaseStrategy, dict]] | Maps running strategy config_id to its strategy instance and runtime config |
| executors | Dict[str, Executor] | References to "live" (CcxtExecutor) and "paper" (PaperTradingExecutor) |
| redis_client | redis.asyncio.Redis | Client for streaming state changes, runtime variables, and logs |
ActivePositionMap — Multi-Market Position Storage
The ActivePositionMap is a custom dict subclass that stores active positions using composite keys of the form market_type:SYMBOL (e.g., futures_usdtm:BTCUSDT). It preserves backward compatibility by allowing symbol-only queries when the lookup is unambiguous, and falls back to matching logic when multiple market types share the same ticker symbol. This design is critical for multi-market support (linear futures, spot, coin-margined) on the same asset.
Startup Orchestration ( start() )
The start() method (lines 1122–1386) executes a carefully ordered sequence of startup tasks that must complete before any trading activity can occur:
Step 1 — Configuration Loading (lines 1122–1174)
Fetches the user's AppConfig from PostgreSQL and applies runtime settings:
Sources:Key operations:
- Risk Limits: Applies risk-per-trade, stop-distance, drawdown, daily-loss, consecutive-loss, max-concurrent, and R:R settings via RiskManager.apply_user_settings (risk_manager.py:448–665).
- Notifications: Loads custom Telegram chat IDs for real-time trade and alert delivery.
- Blacklist: Syncs the static blacklist from the database into the controller's runtime memory.
Step 2 — Symbol Selection Config (lines 1002–1041, called at line 1179)
Loads the SymbolSelectionConfig which dictates the bot's market scope:
| Mode | Behavior | Use Case |
|---|---|---|
| STATIC | Trades only a fixed whitelist of symbols from strategy config | Conservative, manually curated portfolios |
| DYNAMIC_NATR | Selects symbols based on NATR (Normalized ATR) volatility ranking | Adaptive market making across top movers |
| DYNAMIC_ORACLE | AI-driven symbol selection using ML Oracle regime parameters | Experimental / high-alpha strategies |
Symbol selection is configuration-only at this stage; per-symbol data subscriptions are applied later by _update_monitored_symbols() (controller.py:8118), invoked from _check_and_update_symbols() during startup (line 1307).
Step 3 — Runtime State Recovery (_load_runtime_state, lines 4595–4826)
After a server restart or process migration, the controller must reconnect to its prior state without losing track of open trades:
Sources:After deserialization, positions are verified against the exchange via live_executor.get_open_positions() (line 4693) and inserted into the active map under _positions_dict_lock (lines 4774–4776).
Redis keys used for persistence:
depthsight:state:positions:{user_id}:{api_key_id}— ephemeral snapshot of active positionsdepthsight:state:portfolio:{user_id}:{api_key_id}— portfolio-level stats (daily PnL, balance)depthsight:state:strategies:{user_id}:{api_key_id}— running strategy instances and their paramsdepthsight:controller:runtime_state:{user_id}:{api_key_id}— durable runtime-state key written by _save_runtime_state() (line 4543), the source used for recovery
The depthsight:state:* snapshots carry a short TTL (REDIS_STATE_TTL_SECONDS, default 30 s, lines 5138–5152) so a dead controller's snapshot expires instead of lingering.
Step 4 — Exchange Reconciliation (_reconcile_positions_with_exchange, line 3855)
Automatically cross-references local state with actual exchange positions via the executor's get_open_positions(). This critical step handles:
- Orphaned Positions: If the exchange reports an open position that the controller has no record of (e.g., due to a prior crash), it imports the position with an adopted flag via _build_adopted_position (lines 3310–3500).
- Stale Entries: If the controller has a position in _active_positions but the exchange reports it as closed, it routes the trade through _handle_final_exit (line 10851) with reason CLOSED_ON_EXCHANGE and writes the closing record to the database.
- Quantity Discrepancies: If partial fills occurred during downtime, the controller adjusts internal quantity tracking to match exchange reality.
Step 5 — Data Consumer Registration (lines 1194–1268)
The controller starts the DataConsumer (line 1194) and registers exchange and Redis listeners. For each active symbol:
- Candle close events (type: "CANDLE_CLOSE", emitted by DataConsumer at data_consumer.py:4018) — triggers strategy evaluation on each new closed candle.
- Tick events (type: "TICK", emitted at data_consumer.py:4028) — used for intra-candle stop-loss monitoring and tape-reading strategies.
- Order updates — exchange user-data streams call _handle_order_update (line 1260, definition at 15654); private streams are lazy by default and only boot-opened when PRIVATE_WS_MODE=always.
In addition, the Redis command listener and HFT event listener tasks are spawned (lines 1262–1268) to accept external CLOSE_POSITION / EMERGENCY_STOP commands.
Event Processing Loop (_run_main_loop())
Once started, the controller enters its main asynchronous event loop (_run_main_loop, lines 5431–5459). This is the heart of all trading activity:
Event Types and Dispatch Logic
The main loop dequeues each event and spawns a handler task via _handle_event_with_context (lines 5420–5429), which throttles concurrency through the semaphore before calling _handle_event (line 5988).
| Event Type | Source | Action |
|---|---|---|
| CANDLE_CLOSE | DataConsumer — kline stream | Position management (line 6019) plus entry evaluation for on_candle_close triggers |
| TICK | DataConsumer — aggTrade stream | Spot virtual-TP checks (lines 6011–6016); entry evaluation for on_tick / on_condition_met triggers (lines 7517–7522) |
| SCALE_IN_RECALC | Controller (after DCA fill) | Forced position-management recalculation (line 6019) |
| ORDER_UPDATE | Exchange user data stream | _handle_order_update — fill confirmation, updates position state |
| SIGNAL | Strategy instance (internal) | _process_signal (line 8567) dispatches to RiskManager for assessment and execution |
| MANUAL_EXIT | Redis command bus / user command | close_position() via _redis_command_listener (line 1513) |
| REGIME_CHANGE | Screener / ML Oracle | check_on_screener_update (lines 5701–5705), close with reason REGIME_CHANGE_LOSS_CUT (line 5714) |
Position management runs through strategy_instance.manage_position() (line 6206); exit signals returned by the strategy are routed to close_position() (line 6226), and stop-loss modifications go through _replace_stop_loss (line 6362). Entry signals are checked last via _check_signals_for_symbol_on_event (lines 6511–6512).
Concurrency Model
The controller uses a triple-lock strategy to prevent race conditions:
- _positions_dict_lock (asyncio.Lock): Guards structural mutations to _active_positions dictionary (adding/removing positions).
- _symbol_locks[symbol] (asyncio.Lock): Per-symbol ordering guarantee. Ensures events for BTCUSDT are processed sequentially, preventing duplicate orders or conflicting state updates on the same asset.
- _event_handler_semaphore (Semaphore, default 64): Global throttle capping concurrent event handler tasks, preventing memory exhaustion during volatile market conditions when thousands of events arrive per second.
Position Lifecycle (Open — Manage — Close)
Each position traverses a well-defined lifecycle within the controller:
1. Position Entry (_process_signal → LivePosition, lines 8567–9456)
When a signal passes the RiskManager assessment (call at lines 9175–9182), the controller constructs the position object BEFORE placing the order to avoid races with early ORDER_UPDATE events:
Sources:The position is inserted into the active map under _positions_dict_lock (lines 9454–9456); exchange entry order placement and fill tracking are handled by _handle_entry_fill (line 9982).
2. Position Management (strategy.manage_position + controller actions)
On each CANDLE_CLOSE or SCALE_IN_RECALC event, the controller iterates active positions and calls the strategy's management layer:
- Stop Loss / Take Profit / Partial Targets: Checked inside the strategy's _execute_position_management (strategy.py:4417), which iterates configured management blocks — trailing stop (4455), move-to-breakeven (4458), conditional exit (4461).
- Trailing Stop: Supports ATR, percentage, and MA modes; when mode == "exchange" the controller places a native TRAILING_STOP_MARKET order (strategy.py:3895–3984).
- Breakeven: Activated by target_type / target_value with an optional offset_pips buffer (strategy.py:3995–4007).
- Controller-side actions: Management actions returned by the strategy are applied through _execute_management_actions (lines 2943–2991); stop modifications go through _replace_stop_loss.
3. Position Exit (close_position, lines 17692–17708)
Sources:The reliable closure procedure polls for exchange confirmation for up to 120 attempts at 0.5-second intervals (lines 17811–17812), then finalizes the trade record.
Order Submission Flow (Signal to Order)
The path from strategy signal to live order traverses four distinct layers:
RiskManager Gate
The assess_signal() call (bot_module/risk_manager.py, lines 1630–2216) performs twelve sequential stages of validation before any capital is committed — see Dynamic Risk Management for the canonical stage-by-stage breakdown and rejection codes. Highlights:
| Stage | Check | Rejection Code |
|---|---|---|
| 0 | Symbol blacklist | SYMBOL_BLACKLISTED |
| 1 | Balance fetch | ZERO_BALANCE / PAPER_ZERO_BALANCE |
| 3 | Global risk limits (daily loss, drawdown, max trades) | GLOBAL_RISK_LIMIT |
| 5 | Dynamic strategy/symbol multiplier | ZERO_RISK |
| 7 | Entry price & SL validation | INVALID_PRICE, SL_WRONG_SIDE, ZERO_SL_DISTANCE |
| 8 | Reward/Risk ratio | LOW_RR |
| 9 | Exchange lot filters (stepSize, minQty, minNotional) | MIN_QTY_VIOLATION, MIN_NOTIONAL_VIOLATION |
| 10 | Dollar-based R/R check | LOW_DOLLAR_RR |
Exchange Dispatch
Once approved, the controller selects the correct executor and calls place_order(). The CCXTExecutor.place_order() method (bot_module/exchanges/ccxt_executor.py, lines 773–1183) handles:
- Symbol normalization: BTCUSDT to BTC/USDT:USDT (futures) or BTC/USDT (spot).
- Order type mapping: Binance STOP_MARKET mapped to CCXT "market" with stopLossPrice / takeProfitPrice params (lines 801–843; mapping in _map_order_type_to_ccxt, lines 2509–2516).
- Exchange-specific parameters: OKX uses posSide (lines 877–896), Gate.io uses settle: "usdt" (lines 901–903), Bitget sets params["hedged"] = not is_unilateral (lines 917–919).
- Rate-limit compliance: Delegated to CCXT's built-in enableRateLimit token bucket.
State Persistence & Redis Publishing
The controller continuously publishes its runtime state to Redis (via _publish_state_to_redis_impl, lines 4847–5203), enabling:
- Real-Time Frontend Updates: Position, strategy, and portfolio snapshots are written every ~8 seconds (state_publish_interval, line 5473) to per-controller keys.
- Crash Recovery: A new controller instance reconstructs its active position map from the durable runtime-state key in _load_runtime_state (lines 4595–4826), then re-verifies against the exchange.
- Inspection: Operators and multi-account views can read the latest snapshot; ephemeral keys expire automatically if the controller dies.
Shutdown & Cleanup Sequence (stop())
When a stop command is received (via Redis command bus or API), the stop() method (lines 1388–1511) executes a graceful teardown:
Shutdown Steps
- _running = False: The main loop terminates on its next iteration.
- Task Cancellation: Cancels the main loop, config reloader, market-info updater, Redis listeners, dynamic symbol selection, and equity recorder tasks (lines 1400–1443), waiting up to 5 seconds for them to finish.
- Stream & Subscription Teardown: Clears DataConsumer subscriptions (line 1448), releases shared private WebSocket streams and stops executor user-data streams (lines 1453–1468), stops the consumer (line 1470), and closes executors (lines 1471–1492).
- TradeLogger Stop: Stops the trade logger (lines 1493–1495).
- State Finalization: Persists runtime state via _save_runtime_state() (line 1499) and closes the Redis client (lines 1501–1509).
Note that stop() does not liquidate open positions or cancel open exchange orders — positions remain on the exchange and are recovered on the next startup by _load_runtime_state + reconciliation. On-demand exits are issued via close_position() (e.g., CLOSE_POSITION / EMERGENCY_STOP Redis commands handled by _redis_command_listener, line 1513).
Error Handling & Recovery
The controller implements multiple layers of error resilience:
Exchange API Failures
Stop-loss placement failures are retried with fixed backoff inside _replace_stop_loss — up to 3 attempts with 2/4/8-second delays (lines 12624–12659), aborting early if the position is no longer OPEN:
Sources:Full position closure uses the reliable procedure in close_position (see Position Exit above) with its own bounded retry loop.
Position Reconciliation Timer
A background task (_run_periodic_tasks, line 5461) reconciles every 60 seconds (lines 5482–5485), with the first run staggered by a random delay to avoid thundering-herd REST calls after a mass restart. Reconciliation only runs while live strategy instances exist (lines 5543–5550) and:
- Fetches open positions from the exchange via the executor's get_open_positions().
- Cross-references with _active_positions.
- Repairs any discrepancies — adopting missing positions, closing stale entries, and correcting quantity mismatches (same paths as startup reconciliation).
Graceful Degradation
If a Redis state publish fails, the controller logs the error (lines 5199–5203) and continues trading on in-memory state — there is no separate degraded mode. Snapshots refresh on the next ~8-second interval once Redis is reachable again; if the controller dies, its ephemeral keys expire after their 30-second TTL.
Strategy and Signal System
Core technical engine for condition evaluations, visual strategy compiling, indicators calculation, and orderbook stop-loss adaptation inside DepthSight.
Dynamic Risk Management
Comprehensive analysis of the 12-stage signal assessment pipeline, automatic blacklisting, dynamic trade size scaling, and portfolio-level risk limits inside the DepthSight RiskManager.