Architecture Overview

DepthSight is a distributed, multi-tenant SaaS platform for algorithmic crypto trading — designed to be deployed as a self-hosted service or scaled horizontally as a commercial offering. Its architecture separates concerns across eight distinct service layers, unified by Redis as the central nervous system and PostgreSQL as the durable state store. This page maps the full system topology, traces the critical data and control flows, and identifies the design patterns that make DepthSight both modular and production-grade.

---

System Topology

Before diving into individual components, it's essential to understand how the services relate to each other at deployment time. DepthSight runs as a set of Docker containers orchestrated by Compose, each with its own Redis ACL identity and network boundary. The diagram below captures both the service dependency graph and the communication protocols between them.

The architecture follows a fan-out command pattern: the API never directly invokes trading logic. Instead, it publishes commands to Redis channels that the Bot Runner consumes asynchronously. This decoupling means the API can restart without disrupting active trading sessions, and multiple Bot Runner shards can scale independently behind the same command bus.

---

Component Catalog

Each row in the table below corresponds to a first-class service in the Compose stack. The "Redis ACL User" column reflects the principle of least privilege — every service authenticates to Redis with its own identity and password.

Service	Entry Point	Port	Redis ACL User	Primary Responsibility
API	api/depthsight_api.py	8000	api	REST endpoints, auth, strategy CRUD, task dispatch
WebSocket	api/websocket_server.py	8765	websocket	Real-time event streaming to frontend clients
Bot Runner	bot_runner.py	—	bot	Multi-user trading controller lifecycle & command listener
Market Data	market_data_service.py	—	market_data	Centralized exchange WS ingestion → Redis fan-out
Celery Worker	tasks.py	—	celery	Backtesting, genetic optimization, ML training, analytics
PostgreSQL	postgres:15-alpine	5432	—	Persistent state (users, strategies, trades, backtests)
Redis (System)	redis:7.2-alpine	6379	ACL per service	Celery broker, command bus, event bus, quota state
Redis (Market)	redis:7.2-alpine	—	bot, market_data	High-throughput HFT market data fan-out (no persistence)
Frontend	frontend/	5173	—	React SPA with visual strategy editor
PWA	pwa/	5174	—	Mobile-optimized React PWA client

[!IMPORTANT] The Redis Market instance is intentionally configured with --save "" --appendonly no — it operates as a pure in-memory pub/sub bus for low-latency market data. Never configure it for persistence; the write amplification from tick-level data would destroy performance.

---

Data and Control Flows

Understanding how data moves through DepthSight is the key to understanding the system. There are six distinct flow patterns, each serving a different architectural concern.

• Configuration Flow — The user creates or modifies a strategy through the frontend. The REST API validates the payload and persists it to PostgreSQL. No trading action occurs at this point; it's pure CRUD.
• Control Flow — When the user clicks "Start Strategy," the API publishes a structured command (e.g., START_STRATEGY) to a Redis channel. The Bot Runner, which maintains a persistent subscription to this command bus, deserializes the command and instantiates a TradingController for the relevant user and API key. This async command pattern ensures the API remains stateless with respect to trading runtime.
• Market Data Flow — The MarketDataService maintains a single WebSocket connection per exchange stream key, regardless of how many user bots need that data. It publishes normalized market payloads to the dedicated Redis Market instance. Each Bot Runner's DataConsumer subscribes to its specific channels in Redis mode, receiving pre-processed DataFrames without any direct exchange connectivity. This single-stream fan-out design is critical for scaling: 100 users trading BTCUSDT still consume only one exchange connection.
• Execution Flow — The TradingController receives DataFrames from DataConsumer, evaluates them through the active Strategy instance, and if a signal fires, routes it through RiskManager for position sizing and validation. Approved orders land at the ExchangeExecutor (via the CCXT-based factory), which translates them into exchange-specific REST API calls.
• Observation Flow — Every controller action (signal generated, order placed, position modified) is published to Redis Pub/Sub channels scoped to the user. The WebSocket server subscribes to these channels and pushes events to the connected frontend in real time, giving the user full visibility into the bot's decision-making.
• Analytics Flow — Completed trades are written to PostgreSQL. Celery tasks periodically compute aggregated analytics (win rate, drawdown, Sharpe ratio) and write results back, powering the dashboard's performance charts.

---

Core Engine: Trading Controller Lifecycle

The TradingController is the beating heart of DepthSight. Each running strategy gets its own controller instance, fully isolated per user and per API key. The bot_runner.py process manages a dictionary of these controllers — user_controllers[user_id][api_key_id] — and supports sharded deployment where multiple bot processes divide the API key space by modulo assignment.

The controller lifecycle follows a strict sequence: the Bot Runner queries the database for all live-eligible users, filters by plan permissions (checking _plan_allows_live_trading and _user_is_live_eligible), shards the API keys across available workers, and then instantiates a TradingController for each key. A persistent CommandListener task subscribes to Redis channels to handle runtime commands like START_STRATEGY, STOP_STRATEGY, and DEACTIVATE_API_KEY — allowing the API to control trading without direct process communication.

[!TIP] Sharding is keyed by api_key_id % num_workers. When scaling horizontally, adding a new worker shard will cause keys to be redistributed. Plan for a brief controller teardown and re-initialization during shard count changes.

---

Market Data Pipeline: Single-Stream Fan-Out

The MarketDataService solves a fundamental scaling problem: without it, each user bot would open its own WebSocket connection to the exchange, quickly hitting rate limits and wasting bandwidth. Instead, the service acts as a multiplexer — one connection in, many consumers out.

When a bot worker needs data for a new symbol, its DataConsumer sends a subscription command to the MARKET_DATA_REDIS_COMMAND_CHANNEL. The MarketDataService receives this command, checks if a DataConsumer already exists for that exchange, creates one if needed, and then registers the requesting worker as a subscriber. Market payloads are published both as Redis Streams (with warm snapshots for new subscribers) and as real-time Pub/Sub events to worker-specific channels. This dual delivery ensures new workers can reconstruct state from snapshots while receiving live updates through the stream.

The pipeline also maintains global shared caches — kline data, depth snapshots, and aggregated trade deques — stored in module-level dictionaries protected by asyncio locks. All DataConsumer instances within a single process read from these same caches, avoiding redundant computation for indicators like ATR, NATR, and volume percentiles.

---

Exchange Abstraction: Protocol-Based Factory

DepthSight uses a Protocol class (ExchangeExecutor) to define the runtime contract for exchange interaction, rather than inheritance-based abstraction. The create_exchange_executor factory function in bot_module/exchanges/factory.py normalizes exchange identifiers and market types, then returns a CcxtExecutor — a unified adapter built on the CCXT library that supports Binance, Bybit, OKX, Bitget, Gate.io, and BingX.

The factory handles several normalization concerns automatically: exchange aliases (e.g., bybit_linear → bybit), market type resolution (futures → futures_usdtm), and testnet detection from either the exchange ID suffix or the global ACTIVE_TRADING_ENVIRONMENT config variable. This means the calling code never needs to worry about exchange-specific naming conventions — it passes a logical identifier and receives a conformant executor.

Exchange	Stability	Market Types	Notes
Binance	✅ Stable	Futures USDT-M, Spot	Primary tested exchange
Bybit	✅ Stable	Futures USDT-M, Spot	Fully tested
OKX	🧪 Beta	Futures USDT-M, Spot	Active development
Bitget	🧪 Beta	Futures USDT-M, Spot	Active development
Gate.io	🧪 Beta	Futures USDT-M, Spot	Active development
BingX	🧪 Beta	Futures USDT-M, Spot	Active development

---

API Layer: Modular Route Architecture

The FastAPI backend organizes its REST endpoints into domain-specific route modules under api/routes/, each registered as an APIRouter on the main application. This modular structure keeps the 2200+ line main API file manageable while allowing each domain to evolve independently.

Route Module	Domain	Key Capabilities
auth.py	Authentication	JWT login, registration, password recovery, OAuth
strategies.py	Strategy CRUD	Create, update, delete, import/export strategies
backtests.py	Backtesting	Launch vector/candle backtests, retrieve results
portfolio.py	Portfolio	Portfolio-level backtesting and analytics
ai.py	AI Assistant	LLM-powered strategy generation and trade analysis
model_lab.py	ML Lab	Train and evaluate ML models, River/Sklearn/XGBoost
payments.py	Billing	BitCart crypto payment processing
account.py	Account	User settings, API key management
admin.py	Administration	User management, system diagnostics
discovery.py	Community Hub	Strategy presets, trading ideas, network map
support.py	Helpdesk	Ticket system with real-time chat
gamification.py	Gamification	XP, levels, achievements, leaderboard
affiliate.py	Affiliate	Referral tracking, payout management
diagnostics.py	Diagnostics	Health checks, data pipeline verification
config.py	Configuration	Bot configuration, plan management
notifications.py	Notifications	Push notification preferences
tasks.py	Task Status	Celery task polling and result retrieval
webhooks.py	Webhooks	External service integrations
public.py	Public	Unauthenticated endpoints (shared reports)
registry.py	Registry	Service registration and discovery
users.py	Users	User profile operations

The API dispatches long-running operations (backtests, genetic optimization, ML training) to Celery workers via the Redis broker, returning task IDs that the frontend polls for progress updates. This ensures that computationally intensive work never blocks the request-handling event loop.

---

Persistence: Async SQLAlchemy with Session Isolation

DepthSight uses SQLAlchemy with asyncpg for all database access, driven by environment variables assembled into a postgresql+asyncpg:// connection string. The api/database.py module provides three session strategies tailored to different access patterns:

1. get_db() — FastAPI dependency injection generator, yielding a session per-request with automatic cleanup.
2. get_session_for_worker() — Returns the shared AsyncSessionLocal factory for Celery tasks that need transactional consistency.
3. get_isolated_worker_session() — Creates a completely isolated engine and session for a single Celery task, guaranteeing that connection pools are disposed after use. This prevents connection leaks in long-running worker processes.

Database migrations are managed through Alembic, with versioned migration scripts in alembic/versions/. The schema models in api/models.py define the full multi-tenant data model: users, API keys, strategies, trades, backtest results, payments, achievements, and more.

---

Frontend Clients

DepthSight ships two React-based clients, both built with Vite, TypeScript, and shadcn/ui components:

• Web Dashboard (frontend/) — A full-featured SPA with 25+ pages covering strategy editing (drag-and-drop node graph), live position monitoring, backtest visualization, the Genetic Command Center, AI Co-Pilot, community hub, analytics, and admin panels. It connects to the API via REST for CRUD operations and to the WebSocket server for real-time log streaming and position updates.
• Mobile PWA (pwa/) — A mobile-optimized client with i18n support (English and Russian locales), covering core monitoring and control workflows. It shares the same backend API and WebSocket infrastructure but provides a touch-first interface for on-the-go trading management.

Both clients use a WebSocketProvider context that handles authentication, reconnection, and channel subscription management — abstracting the raw WebSocket protocol into reactive data stores.

---

Redis: The Central Nervous System

Redis is not merely a cache in DepthSight — it serves five distinct architectural roles, each leveraging a different Redis primitive:

Role	Redis Feature	Example Key/Channel	Used By
Celery Broker	List / Pub-Sub	celery namespace	API → Celery Workers
Command Bus	Pub-Sub	depthsight:commands:*	API → Bot Runner
Market Data Fan-out	Streams + Pub-Sub	depthsight:market_data:*	MarketDataService → Bots
Event Bus	Pub-Sub	depthsight:events:*	Bot Runner → WebSocket → Frontend
State & Quotas	Hash / String	depthsight:quotas:*, depthsight:state:*	API, Bot Runner

The dual-Redis topology (system + market) separates the high-throughput, low-latency market data path from the general-purpose system operations. The market Redis instance runs without persistence (--save "" --appendonly no) and is scoped to only bot and market_data ACL users, ensuring that tick-level data never competes with Celery task broker operations for memory or I/O.

---

Deployment Topology

DepthSight runs as a single Docker Compose stack with 9 service containers. The one-click deploy.sh script handles the full provisioning pipeline on any Ubuntu 22.04+ server: Docker installation, .env secret generation, Caddy reverse proxy with auto-SSL (via sslip.io for IP-only setups), firewall configuration, and service startup. Updates can be triggered either from the admin web UI (via a container-to-host trigger file mechanism) or by running update.sh on the host.

The minimum production requirement is 6 CPU cores and 16GB RAM, reflecting the concurrent demands of market data ingestion, multiple trading controller loops, Celery worker processes (8 prefork workers by default), and the PostgreSQL instance.

---

Where to Go Next

Now that you have the full architectural map, the logical next step is to dive into the subsystems that power the trading loop. The following pages trace the critical path from signal generation to order execution:

• Strategy and Signal System — Learn how market conditions are evaluated and weighted.
• Trading Controller Lifecycle — See how signals become executed trades.
• Centralized Market Data Service — Understand the real-time data pipeline.
• FastAPI REST Routes & Multi-Tenant Auth and Quotas — Inspect the SaaS management layers.