Genetic Strategy Optimization
Deep dive into the DEAP-based genetic evolution algorithm, parameter mutation, crossover operations, fitness functions, and Out-of-Sample validation inside DepthSight.
Manual parameter optimization (trying different EMA periods, RSI thresholds, or Stop Loss multipliers) can take weeks. DepthSight leverages genetic algorithms via the DEAP library (bot_module/genetic_strategy_finder.py, ~2,270 lines) to automate strategy optimization by simulating evolutionary processes.
Evolutionary Optimization Cycle
The GeneticStrategyFinder (line 906) maintains a "population" of strategy configurations and evolves them over multiple generations. Each "individual" is a complete strategy JSON (dict with filters, entryConditions, initialization, positionManagement).
Evolutionary Loop (run(), line 1191)
Sources:Each generation ends with elitism: the top 3 Hall of Fame individuals are cloned over the worst offspring (lines 1385โ1398), and after the last generation run() returns final_results (line 1556).
Default Parameters:
| Parameter | Value | Description |
|---|---|---|
| population_size | 50 | Number of strategy individuals per generation |
| generations | 20 | Number of evolutionary iterations |
| crossover_probability | 0.7 | Likelihood of crossover between two parents |
| mutation_probability | 0.3 | Likelihood of mutation per individual |
| elite_count | 3 | Top strategies preserved unchanged each generation |
| hall_of_fame_size | 10 | Best-ever strategies tracked across all generations |
Gene Pool Initialization
Dynamic GENE_POOL from UI (build_dynamic_gene_pool, line 287)
The UI configuration drives which indicators, filters, and risk parameters are evolvable. The gene pool defines the valid ranges for each parameter the algorithm is allowed to modify. The defaults below come from the static GENE_POOL (lines 48โ252); build_dynamic_gene_pool (line 287) overrides indicator period ranges and risk ranges with the values configured in the UI:
| Block Type | Evolvable Parameters | Range |
|---|---|---|
| rsi_condition | period, value | 5โ14, 25โ75 |
| ma_cross_condition | fast_period, slow_period | 3โ20, 21โ50 |
| bb_condition | period, std_dev | 14โ20, 2.0โ2.5 |
| initialization (risk) | sl_value_atr, tp_value_rr, breakeven_trigger_rr, breakeven_buffer_atr | 1.5โ5.0, 2.0โ8.0, 0.5โ2.0, 0.02โ0.15 |
| initialization (exits) | max_hold_candles, max_partial_exits, partial_tp_value_rr, partial_size_pct | 100โ600, 1โ3, 1.0โ6.0, 20โ60 |
Seeded Population (_init_seeded_population, line 1074)
Instead of starting from complete randomness, the algorithm initializes a "seeded" population using the user's base strategy parameters:
Sources:The seed fills population_size // 2 slots with exact copies of user strategies, then fills the remaining slots with slightly mutated variations. This bootstraps the evolutionary curve, ensuring it builds upon viable logic rather than random noise.
Crossover Operations
The _crossover_individuals() method (line 1805) operates at two levels:
Level 1 โ Initialization Parameter Swap
For each common parameter in the initialization sections (SL type, TP value, direction), there is a 50% chance of swapping:
Sources:Level 2 โ Sub-Tree Exchange in Logic Sections
For filters and entryConditions sections (70% probability each):
Sources:This allows entire logical branches (e.g., an RSI condition with all its children) to migrate between strategies, creating novel hybrid approaches.
Mutation Operations
The _mutate_individual() method (line 1870) supports four mutation types:
1. Parameter Mutation (per-param, ind_pb=0.1)
Randomly re-initializes individual parameters from the GENE_POOL allowed ranges:
Sources:When keep_structure is enabled, the lighter _mutate_params_only (line 1145) runs instead: each numeric parameter has a 30% chance of being nudged by ยฑ20% while the block structure stays fixed (lines 1159โ1170).
2. Structural Node Replacement (20%)
Replaces a leaf node with a randomly generated node of a different type (e.g., replacing an RSI condition with a Bollinger Bands condition).
3. Logical Tree Mutation (15%)
Adds or removes a child from AND/OR nodes, effectively changing the complexity and logic of the condition tree.
4. Operator Flip
For AND/OR nodes, flips the logical operator, changing the relationship between child conditions.
Timeframe Mutation
Each node has a 30% chance of having its timeframe parameter randomly changed (e.g., from 5m to 15m), enabling cross-timeframe strategy evolution.
Fitness Function (_evaluate_fitness, line 997)
The success of each individual is assessed using a multi-asset fitness function: FastVectorBacktester is run per asset (lines 1003โ1010), and the average PnL% and average max drawdown% are aggregated across assets (lines 1018โ1023). Kill switches then eliminate bad candidates outright:
Sources:Only survivors are scored with a Calmar-style quality formula:
Sources:Single Calmar-Style Objective
There is exactly one objective โ no alternate modes. The score is (avg_pnl / max(1.0, avg_max_dd)) * 10.0, multiplied by 1.1 when the strategy produced more than 100 trades. Kill switches (hard-coded, not configurable) short-circuit scoring first:
| Kill Switch | Condition | Result |
|---|---|---|
| Spam filter | total_trades > 1000 | โ9999 |
| Undertrading | total_trades < 30 | โ5000 |
| Losing strategy | avg_pnl <= 0 | returns the negative avg_pnl as fitness |
| Excessive drawdown | avg_max_dd > 25% | โavg_max_dd ร 10 |
Overfitting Prevention
A common issue in genetic search is overfitting โ creating a strategy that memorized historical data perfectly but fails in live trading. DepthSight implements multiple safeguards:
In-Sample / Out-of-Sample Split (split_data_is_oos, line 863)
The historical data is chronologically split with oos_ratio=0.30 (30% held out):
[========= In-Sample (70%) =========][=== OOS (30%) ===]
Genetic search runs here Final validation
Walk-Forward Windows (split_data_into_windows, line 818)
Supports N-window walk-forward optimization where the strategy is tested across multiple equal, non-overlapping time periods (sliced on the 1m base timeframe):
Sources:OOS Validation in Final Results (lines 1506โ1542)
After evolution completes, each Hall of Fame strategy is evaluated on the OOS window and flagged invalid when OOS PnL is non-positive or OOS drawdown exceeds 25%:
Sources:Candidates that fail this check are dropped from the results (lines 1538โ1542); if every candidate fails, the rank-1 fallback is kept anyway (lines 1548โ1553).
Multi-Asset Training
Fitness is averaged across multiple assets (configurable list), preventing single-asset overfitting. A strategy that works on both BTCUSDT and ETHUSDT is more likely to generalize.
Hall of Fame Diversity
The top 10 strategies across all generations are preserved in a plain DEAP HallOfFame (line 969). The Hall of Fame is updated with the offspring of every generation (lines 1321โ1322) and keeps only the fittest unique individuals, preventing the population from converging on a single solution.
Checkpointing & Resume
The genetic optimizer supports JSON-safe checkpoints (lines 1400โ1436) โ no pickle serialization:
Sources:This allows resuming evolution from any checkpoint, useful for long-running optimizations that may need to survive server restarts.
Configuration Reference
| Parameter | Default | Description |
|---|---|---|
| population_size | 50 | Number of strategy individuals per generation |
| generations | 20 | Maximum evolutionary generations |
| crossover_probability | 0.7 | Likelihood of crossover between parents |
| mutation_probability | 0.3 | Likelihood of mutation per individual |
| tournament_size | 2 | Selection tournament participants |
| elite_count | 3 | Top strategies preserved unchanged |
| hall_of_fame_size | 10 | Best-ever strategies tracked |
| walk_forward_oos_ratio | 0.30 | Fraction of data held out for OOS |
| keep_structure | false | When true, only numeric params mutate |
| min_trades_kill_switch | 30 (hard-coded) | Minimum trades to avoid elimination (fixed in _evaluate_fitness) |
| max_trades_kill_switch | 1000 (hard-coded) | Maximum trades before spam filter (fixed in _evaluate_fitness) |
| max_drawdown_kill_switch | 25% (hard-coded) | Maximum drawdown before elimination (fixed in _evaluate_fitness) |
Dual Backtesting Engines
Technical analysis of the Fast Vector Backtester and Event-Driven DepthSight Backtester โ their architectures, trade simulation strategies, KPI calculations, and when to use each.
Centralized Market Data Service
Technical breakdown of the centralized market_data_service.py daemon โ WebSocket aggregation, subscription management, reference counting, snapshot persistence, and Redis publication.