Genetic Strategy Optimization
Deep dive into the DEAP-based genetic evolution algorithm, parameter mutation, crossover operations, fitness functions, and Out-of-Sample validation inside DepthSight.
Manual parameter optimization (trying different EMA periods, RSI thresholds, or Stop Loss multipliers) can take weeks. DepthSight leverages genetic algorithms via the DEAP library (ot_module/genetic_strategy_finder.py, ~2,270 lines) to automate strategy optimization by simulating evolutionary processes.
Evolutionary Optimization Cycle
The GeneticStrategyFinder (line 906) maintains a "population" of strategy configurations and evolves them over multiple generations. Each "individual" is a complete strategy JSON (dict with ilters, entryConditions, initialization, positionManagement).
Evolutionary Loop (run(), line 1191)
Sources:Default Parameters:
| Parameter | Value | Description |
|---|---|---|
| population_size | 50 | Number of strategy individuals per generation |
| generations | 20 | Number of evolutionary iterations |
| crossover_probability | 0.7 | Likelihood of crossover between two parents |
| mutation_probability | 0.3 | Likelihood of mutation per individual |
| elite_count | 3 | Top strategies preserved unchanged each generation |
| hall_of_fame_size | 10 | Best-ever strategies tracked across all generations |
Gene Pool Initialization
Dynamic GENE_POOL from UI (uild_dynamic_gene_pool, line 287)
The UI configuration drives which indicators, filters, and risk parameters are evolvable. The gene pool defines the valid ranges for each parameter the algorithm is allowed to modify:
| Block Type | Evolvable Parameters | Range |
|---|---|---|
| rsi_condition | period, threshold | 5–50, 10–90 |
| ma_cross_condition | fast_period, slow_period | 5–50, 10–200 |
| bollinger_bands_condition | period, std_dev | 10–50, 1–4 |
| atr_stop_loss | atr_multiplier | 0.5–5.0 |
| trailing_stop | activation_pct, trail_pct | 0.1–5.0, 0.1–3.0 |
Seeded Population (_init_seeded_population, line 1074)
Instead of starting from complete randomness, the algorithm initializes a "seeded" population using the user's base strategy parameters:
Sources:The seed fills population_size // 2 slots with exact copies of user strategies, then fills the remaining slots with slightly mutated variations. This bootstraps the evolutionary curve, ensuring it builds upon viable logic rather than random noise.
Crossover Operations
The _crossover_individuals() method (line 1805) operates at two levels:
Level 1 — Initialization Parameter Swap
For each common parameter in the initialization sections (SL type, TP value, direction), there is a 50% chance of swapping:
Sources:Level 2 — Sub-Tree Exchange in Logic Sections
For ilters and entryConditions sections (70% probability each):
Sources:This allows entire logical branches (e.g., an RSI condition with all its children) to migrate between strategies, creating novel hybrid approaches.
Mutation Operations
The _mutate_individual() method (line 1870) supports four mutation types:
1. Parameter Mutation (per-param, ind_pb=0.1)
Randomly re-initializes individual parameters from the GENE_POOL allowed ranges:
Sources:2. Structural Node Replacement (20%)
Replaces a leaf node with a randomly generated node of a different type (e.g., replacing an RSI condition with a Bollinger Bands condition).
3. Logical Tree Mutation (15%)
Adds or removes a child from AND/OR nodes, effectively changing the complexity and logic of the condition tree.
4. Operator Flip
For AND/OR nodes, flips the logical operator, changing the relationship between child conditions.
Timeframe Mutation
Each node has a 30% chance of having its timeframe parameter randomly changed (e.g., from 5m to 15m), enabling cross-timeframe strategy evolution.
Fitness Function (_evaluate_fitness, line 997)
The success of each individual is assessed using a multi-asset, multi-metric fitness function:
Sources:Three Objective Modes
The optimizer supports three main fitness goals configurable by the user:
| Mode | Formula | Behavior |
|---|---|---|
| Maximize Net Profit | score = avg_pnl | Finds highest absolute profit, may accept high drawdowns |
| Maximize Sharpe | score = sharpe_ratio * 10 | Prioritizes risk-adjusted returns |
| Minimize Drawdown | score = 100 - avg_max_dd | Minimizes risk, seeks smooth equity curves |
Overfitting Prevention
A common issue in genetic search is overfitting — creating a strategy that memorized historical data perfectly but fails in live trading. DepthSight implements multiple safeguards:
In-Sample / Out-of-Sample Split (split_data_is_oos, line 863)
The historical data is chronologically split with oos_ratio=0.30 (30% held out):
[========= In-Sample (70%) =========][=== OOS (30%) ===]
Genetic search runs here Final validationWalk-Forward Windows (split_data_into_windows, line 818)
Supports N-window walk-forward optimization where the strategy is tested across multiple non-overlapping time periods:
Sources:OOS Validation in Final Results (lines 1506–1542)
After evolution completes, each Hall of Fame strategy is evaluated on the OOS window:
Sources:Multi-Asset Training
Fitness is averaged across multiple assets (configurable list), preventing single-asset overfitting. A strategy that works on both BTCUSDT and ETHUSDT is more likely to generalize.
Hall of Fame Diversity
The top 10 strategies across all generations are preserved, with diversity tracked by their JSON structure hash to prevent the population from converging on a single solution.
Checkpointing & Resume
The genetic optimizer supports JSON-safe checkpoints (lines 1400–1435) — no pickle serialization:
Sources:This allows resuming evolution from any checkpoint, useful for long-running optimizations that may need to survive server restarts.
Configuration Reference
| Parameter | Default | Description |
|---|---|---|
| population_size | 50 | Number of strategy individuals per generation |
| generations | 20 | Maximum evolutionary generations |
| crossover_probability | 0.7 | Likelihood of crossover between parents |
| mutation_probability | 0.3 | Likelihood of mutation per individual |
| tournament_size | 2 | Selection tournament participants |
| elite_count | 3 | Top strategies preserved unchanged |
| hall_of_fame_size | 10 | Best-ever strategies tracked |
| walk_forward_oos_ratio | 0.30 | Fraction of data held out for OOS |
| keep_structure | true | When true, only numeric params mutate |
| min_trades_kill_switch | 30 | Minimum trades to avoid elimination |
| max_trades_kill_switch | 1000 | Maximum trades before spam filter |
| max_drawdown_kill_switch | 25% | Maximum drawdown before elimination |
Dual Backtesting Engines
Technical analysis of the Fast Vector Backtester and Event-Driven DepthSight Backtester — their architectures, trade simulation strategies, KPI calculations, and when to use each.
Centralized Market Data Service
Technical breakdown of the centralized market_data_service.py daemon — WebSocket aggregation, subscription management, reference counting, snapshot persistence, and Redis publication.