The difference between back-testing and live trading
... why a back-test allows only a weak prognosis of possible future developments
These are two inquiries our customers repeatedly make regarding back testing.
1. Why is it not possible to reproduce the Magic-Champ II live trades in a back test to 100%?
2. Why does ForexInnovation as well as other clients have different trades on their real accounts than I do? Shouldn't the trades be identical having the same Expert Advisor and sometimes even the same broker?
Question # 1: Why aren't the live trades identical with the trades in my back test?
The existing discrepancy between back-testing and live trading is often experienced as soon as an apparently well running system is activated for live trading for the first time. At that moment the systems’ successful back-test performance deteriorates into an unfavorable performance in live trading altering a winning into a losing system. Also our system developers have experienced this commonly and systems with 100% gain per month in the back- test disappeared in the drawer.
But what are the reasons?
1. MetaTrader does not provide real tick-data for back-testing.
The development of a system is generally based on historic data. Accordingly the systems’ decisions during paper-trading also base on this historic data. The crucial point is that the used historical data is NO real tick-data. It is often believed to have programmed a system grounded on actually occurred tick-data, which is simply wrong. Instead, it is based on hypothetical tick-data which MetaTrader calculates by using the 1min candle and its high/low/open/close as well as the volume correspondingly. Hence, using historical data for back-testing means you test your strategy only with ticks how they could have been, and NOT how they actually took place.
Particularly scalping systems with phenomenal back-test performances fail due to this barrier. Of course, also we develop our systems based on this historical data. However, our systems run through extensive live testing and are improved thoroughly if needed, or sometimes discarded.
2. All back-tests are based on the data downloaded from the Metaquotes server.
Invariably, and independent of your broker, the loaded data will always be based on the, from Metaquotes provided, 1min data. However, in the Forex market no ubiquitous correct data exist, instead every broker and dealing desk yields their own prices or mirrors the ones of their liquidity provider. Consequently we face the phenomenon of 3 different broker = 3 different prices in reality. Thus, with the same system, you will have a varying number of trades in live trading and back-testing dependent on the broker.
3. A fixed spread is the basis for the calculation and the trading of a back-test.
The real spread, in comparison to the used fix spread of a back-test, is dependent on your broker and might even fluctuate over time. This real spread is often decisive for the SL or TP of a particular trade. So if a real trade just misses the SL or TP by a few points, whereas the back test-trade reaches the SL or TP, the two trades might vary significantly. The reason is, similar to the pseudo-spread described above, that the fixed spread used in back-tests is not representative for the real spreads which actually occurred in the past.
Question #2: Why does my real account have different trades than the ones of ForexInnovation or other clients accessible via myfxbook?
Repeatedly we are asked why a specific Expert Advisor has different performances on different accounts. The following pictures and descriptions will exemplify the answer to this fact.
All images are taken from different real accounts, which run locally on one and the same laptop.
All accounts access the Internet through a common physical data line.
All depots are real accounts at the same broker, FxPro.
All depots are connected to the same FxPro server.
You see a nice doji marked with a small red candle body.
Here, you see a nice doji marked with a small green candle body.
Now imagine a trading system using the following conditions: Only trade after a candle with a red body! Accordingly only one of the depots will make a trade even if both are running with the same system. With this in mind it not hard to understand that varying trades are inevitably bound to occur as well as uninfluenceable.
The high of the candle is marked: 1.62959 !
The high of the candle is marked: 1.62956 !
Even though this difference seems very small, it can be decisive for the instruction whether to trade on one account and not on the other. Given this small deviation each indicator, being based on high-price, will compute a different value. One might be 30.05 and the other 29.25. Again, this difference might not appear big, but if the systems’ threshold value is 30, it will be sufficient to let one account trade while the other stays flat.
Therefore, these tiny differences can be crucial for the success or failure of the trade. For example, one depot might reach the TP whereas the other might miss it by only 0.3 points. As a matter of fact this happens in reality as displayed in the following trades publicized on myfxbook.
Conclusion
1. Sometimes an apparently perfect back-test is not worth the paper it is written on.
Much more important than a back-test are sufficient long and statistically valid live trading results with real money. Only the systems’ live trading performance will reveal in how far it is applicable for real market conditions. The back test can be seen as good indicator for how much future effort should be invested in a particular system, the actual performance and stability, however, can only be affirmed by live trading results.
The lesson is clear, whenever you want buy a system or subscribe to signals, you should ask for representative and current real money trading results before.
2. In the short run a systems‘ live trading performance will vary on different accounts.
As described above, minor deviations in the data result inevitably in different trades. Only after a statistically relevant number of trades, and identically provided signals, the results of the different accounts will converge. So don’t be surprised if your trades, and hence your results differ positively or negatively. Our experience shows that even using the same laptop and the same broker does not prevent to obtain different results. Sometimes there are series with 20 or 30 identically processed trades, but it also happens that one depot performs 3 trades while the other performs 6 trades at the same day.
On the behalf of a financial management company we are trading a modified version of Magic-Champ II. For that reason we reprogrammed the system in JForex for Dukascopy. The same system is also traded on various other accounts with different risk parameters. And here, too, we observe varying trade performances due to fractional price deviation although the entry only differs in milliseconds.
Magic-Champ II makes use of a mathematical algorithm producing trading signals based on available data. These are processed if the system is used fully automated. As explained, the slightest difference in the data provided results in different trading performances. This is unavoidable and uninfluencable.
