Is a 15% drop from backtest to live normal?
Hey guys, I recently finished coding a mean-reversion strategy. The backtest on 99.9% tick data looked great, showing around a 65% win rate over the last two years.
I finally put it live last month. I route the orders through brondesbury global via API, so I know it is not a platform lag issue, but my live win rate has still dropped to about 52%.
I know the strategy tester does not perfectly simulate live market conditions, especially spread widening during the Asian session, but a 13% drop feels pretty massive.
For those of you running automated systems, what percentage drop do you usually expect when moving from a backtest to a live environment? Should I pull the plug and keep tweaking the logic, or is a 10-15% variance completely normal in forward testing?
There isn't a fixed 'normal' drop that makes this safe to dismiss. Going from 65% to 52% is 13 percentage points, and the number of live trades matters a lot.I'd freeze the rules and replay the exact live dates first. Match each expected entry with the actual order, fill and exit. API routing alone doesn't rule out different prices, rejected orders, missed signals or a bid/ask mismatch. For a mean-reversion setup, a small execution difference can change which trades hit a tight target.Then compare the same sessions and spreads. Check commission and slippage, and calculate expectancy from net winners and net losers, rather than win rate alone. Don't subtract costs twice if they're already in the trade results. A 99.9% data-quality label doesn't establish that the tester used your live costs and execution rules.I wouldn't retune the logic before separating those differences. If you share the non-sensitive entry and exit rules, I can help set up a repeatable comparison. How many closed trades are in that first live month?
Mmildap posted:Hey guys, I recently finished coding a mean-reversion strategy. The backtest on 99.9% tick data looked great, showing around a 65% win rate over the last two years.
I finally put it live last month. I route the orders through brondesbury global via API, so I know it is not a platform lag issue, but my live win rate has still dropped to about 52%.
I know the strategy tester does not perfectly simulate live market conditions, especially spread widening during the Asian session, but a 13% drop feels pretty massive.
For those of you running automated systems, what percentage drop do you usually expect when moving from a backtest to a live environment? Should I pull the plug and keep tweaking the logic, or is a 10-15% variance completely normal in forward testing?
I'm using brondesbury global via API as well, and I'd say it's worth looking beyond execution speed when comparing backtest and live results. Even with good tick data, spreads, slippage, and actual fill prices can make a noticeable difference. I'd compare your live fills with the backtest assumptions and break down the results by trading session, especially during periods of wider spreads. A drop from 65% to 52% doesn't automatically mean the strategy is broken, but I'd check live expectancy after all costs before deciding whether to keep it running or change the logic.
AdrisKK posted:Mmildap posted:Hey guys, I recently finished coding a mean-reversion strategy. The backtest on 99.9% tick data looked great, showing around a 65% win rate over the last two years.
I finally put it live last month. I route the orders through brondesbury global via API, so I know it is not a platform lag issue, but my live win rate has still dropped to about 52%.
I know the strategy tester does not perfectly simulate live market conditions, especially spread widening during the Asian session, but a 13% drop feels pretty massive.
For those of you running automated systems, what percentage drop do you usually expect when moving from a backtest to a live environment? Should I pull the plug and keep tweaking the logic, or is a 10-15% variance completely normal in forward testing?
I'm using brondesbury global via API as well, and I'd say it's worth looking beyond execution speed when comparing backtest and live results. Even with good tick data, spreads, slippage, and actual fill prices can make a noticeable difference. I'd compare your live fills with the backtest assumptions and break down the results by trading session, especially during periods of wider spreads. A drop from 65% to 52% doesn't automatically mean the strategy is broken, but I'd check live expectancy after all costs before deciding whether to keep it running or change the logic.
Appreciate the advice! Good to hear from someone on the same API setup. I'll pull the execution logs this weekend to compare live fills vs the backtest. You're probably right about the Asian session spreads chewing up the edge. I'll drop to minimum lot size for now and calculate the true expectancy before tweaking the logic.
Smart move on dropping the lot size. Mean-reversion systems are notoriously sensitive to live market conditions since the profit targets are usually so tight. To answer your original question: yeah, a 10-15% variance between 99.9% tick data and live forward testing is completely normal for this style of EA. Let it run on micros for a few weeks to build a true baseline before you tweak any code. Good luck with the forward test!