Skip to main content

Backtesting Methodology

TradeAnon validates trading signals through systematic backtesting. This page documents our approach to strategy validation and how to interpret backtest results.

Philosophy​

Our backtesting follows several principles:

  1. Out-of-sample testing — Test on data not used to develop the strategy
  2. Realistic assumptions — Account for slippage, costs, and execution
  3. Multiple metrics — Evaluate performance from various angles
  4. Transparency — Document methodology and limitations
  5. Skepticism — Question results that seem too good

Backtest Framework​

Data Sources​

Backtests use the same data sources as the live platform:

  • FINRA short volume (dark pools)
  • CFTC COT reports (futures positioning)
  • Exchange price data (equities, futures)
  • FRED economic data (macro indicators)

Time Periods​

We use rolling windows to avoid overfitting:

  • Training: Parameter optimization (if any)
  • Validation: Strategy refinement
  • Testing: Final evaluation (not used for development)

Signal Generation​

Signals are generated using the same logic as the live platform:

  • No future information ("look-ahead bias")
  • Signals based on data available at decision time
  • Realistic data availability (T+1 for some sources)

Key Metrics​

Return Metrics​

Total Return: Cumulative return over the test period.

Annualized Return:

Annualized Return = (1 + Total Return)^(252/Days) - 1

Excess Return: Return above a benchmark (typically SPY or buy-and-hold).

Risk Metrics​

Volatility: Annualized standard deviation of returns.

Volatility = StdDev(Daily Returns) × √252

Maximum Drawdown: Largest peak-to-trough decline.

Max DD = min[(Equity / Peak Equity) - 1]

Calmar Ratio:

Calmar = Annualized Return / Max Drawdown

Risk-Adjusted Returns​

Sharpe Ratio:

Sharpe = (Return - Risk-Free Rate) / Volatility

Above 1.0 is good; above 2.0 is excellent.

Sortino Ratio: Like Sharpe but only penalizes downside volatility.

Sortino = (Return - Target) / Downside Deviation

Win/Loss Metrics​

Win Rate: Percentage of trades that were profitable.

Win Rate = Winning Trades / Total Trades

Average Win / Average Loss:

Profit Factor = (Avg Win × Win%) / (Avg Loss × Loss%)

Expectancy: Average expected return per trade.

Expectancy = (Win% × Avg Win) - (Loss% × Avg Loss)

Assumptions​

Entry/Exit​

Entry:

  • Signal generated at market close
  • Entry at next day's open
  • Limit orders when specified

Exit:

  • Signal generated at market close
  • Exit at next day's open
  • Stop-losses executed at trigger price

Costs​

Transaction Costs:

  • Round-trip: 0.1% of trade value (conservative)
  • Adjusted for instrument type

Slippage:

  • Included in entry/exit prices
  • Higher for less liquid instruments

Position Sizing​

Default:

  • Equal weight per position
  • No leverage unless specified

Risk-Based:

  • ATR-based sizing when appropriate
  • Maximum position limits applied

Validation Approaches​

Walk-Forward Analysis​

Test across rolling time periods:

  1. Train on period 1
  2. Test on period 2
  3. Roll forward and repeat
  4. Aggregate out-of-sample results

Advantage: More robust than single backtest.

Monte Carlo Simulation​

Randomize trade order to test robustness:

  1. Shuffle trade sequence
  2. Calculate metrics
  3. Repeat 1,000+ times
  4. Analyze distribution of outcomes

Advantage: Shows range of possible outcomes.

Parameter Sensitivity​

Test how results change with parameters:

  1. Vary key parameters ±20%
  2. Observe impact on performance
  3. Ensure results aren't sensitive to exact values

Advantage: Identifies overfitting.

Interpretation Guidelines​

What Good Results Look Like​

Robust Signal:

  • Positive returns across multiple periods
  • Consistent Sharpe ratio over time
  • Reasonable number of trades (not curve-fitted)
  • Results stable across parameter variations

Red Flags​

Likely Overfitting:

  • Exceptional returns (>50% annual)
  • Very few trades
  • Sharp performance degradation out-of-sample
  • Highly sensitive to parameters

Data Issues:

  • Look-ahead bias (using future data)
  • Survivorship bias (only using current securities)
  • Data errors affecting results

Realistic Expectations​

Historical backtests typically overstate live performance:

  • Expect 20-50% degradation from backtest to live
  • Factor in changing market conditions
  • Consider regime changes

Backtest Reports​

Standard Format​

Each backtest report includes:

  1. Strategy Description — What signal is being tested
  2. Data Period — Start/end dates, data sources
  3. Assumptions — Costs, sizing, execution
  4. Results — All key metrics
  5. Equity Curve — Visual performance chart
  6. Trade Distribution — Win/loss analysis
  7. Limitations — Known caveats

Comparing Strategies​

When comparing backtests:

  • Ensure same time period
  • Use same cost assumptions
  • Compare risk-adjusted returns
  • Consider capacity (how much capital can trade it)

Limitations​

General Limitations​

Past ≠ Future: Historical performance does not guarantee future results. Market conditions change.

Execution Differences: Backtest assumes perfect execution. Live trading involves slippage, partial fills, and timing differences.

Regime Changes: Strategies that worked historically may not work in new market regimes.

Specific to TradeAnon​

Data Availability: Some data sources have limited history, constraining backtest length.

Survivor Bias: Current universe may differ from historical universe.

Signal Changes: Strategy logic may have evolved over time.

Best Practices for Users​

Interpreting Results​

  1. Focus on risk-adjusted returns (Sharpe, Sortino)
  2. Check drawdown — Can you handle it emotionally?
  3. Verify sample size — Enough trades to be statistically meaningful
  4. Consider time period — Does it include various market conditions?

Using in Live Trading​

  1. Paper trade first — Validate understanding
  2. Start small — Size up gradually
  3. Track performance — Compare live vs expected
  4. Adapt as needed — Adjust if conditions change