The Dangers Of Backtesting Your Trading System

Today we've invited Hugh of TradingHeros.com to talk a little bit about the dangers of backtesting.

Hugh loves to surf, drinks butter with his coffee, is almost the same height as Jeremy Lin and aspires to be a Professional Forex Trader.  He logs his trades, interviews professional traders, does trading software tutorials and analyzes his backtesting results on his Forex trading blog .

You have finally done it! After a 36 hour marathon backtesting session, 15 pots of coffee and some smelling salts, you finally have the vital stats on your new discretionary technical trading system called the GekkoSmasher 3. And the results look very promising...

According to your summary screen, you would have averaged 72.3% per year over the past 10 years. You can’t help but put your Realtor’s number back on speed dial. Time to move on up to the Hamptons.

Hoooold on there Tiger. Let’s take a step back and think about what those testing results really mean and what you can really expect in real life trading. Those numbers may be lying to you.

Of course this scenario is completely made up, but the pitfalls of taking your backtesting results at face value are very real. In this post, I’m going to briefly go through some considerations that have to be taken into account when backtesting a trading system and the booby traps that might be hidden in the numbers.

Are You Awake?

With backtesting, you can sit down and take every single trade that comes up because your are scrolling through the charts so rapidly. This is great because you can find out very quickly if your system is viable or not.

But in real life, you actually have to eat, run errands, exercise and do that annoying thing called sleep. So you may not be available during the times when your system makes the biggest profits.

The solution is to take your backtesting results and map out when most of your winning trades open. Are you available to trade during these times? If not, can you change your schedule to trade during these times?

Since Forex is a 24 hour market, the good news is that you can probably find a system that will work in a time frame that is convenient for you. In other markets, you might be able to narrow your optimal trading time to within an hour or two and concentrate on only trading during those times.

Cheater, Cheater, Pumpkin Eater

With a mechanical trading system, it is not possible to cheat during testing because everything is done automatically. But with a discretionary system, it is possible to back up the chart and “optimize” that last trade a little. “That is how it should have been traded,” you tell yourself.

Resist every urge you have to do that. Try to treat testing as you would live trading. This means not going back on a trade you have taken, imagine the money as real money and move the chart forward slowly when you are actually in a trade so you can experience the emotions that you would feel, had that been a real trade.

Mo’ Money, Mo’ Problems

It’s pretty easy to trade when there is nothing on the line. But if you throw some real money into the mix, your psychology is probably going to be very different. How do you get around this?

As I mentioned above, imagining that you are trading real money can help. However, that will never be a realistic substitute for actually risking your hard earned cash.

Therefore, if you want to start trading this system, start with a very small amount of money first. In Forex, you can trade “nano” lots in which you risk as little as a penny per tick/pip. In other markets, start with the smallest trade size possible.

After you get comfortable with that, you can start to move up to bigger lot sizes. This is the same if you are trading any other market. Start small and only trade bigger when you are confident that the system is trading as tested.

Another thing you can do is practice, practice, practice. Keep doing backtesting, even if you have done it before. This will keep your mind alert to what your setup looks like and you may notice nuances in the trade that you never noticed in previous sessions, which could improve your results.

Has The Market Gone Psycho?

Then there is the possibility that the market you are trading has changed in a fundamental way and will no longer function the same again. Is there anything you can do to change that? No.

But luckily, if you have backtested your strategy over a long period of time, you will know exactly how your system should behave. If your trading starts operating outside of those parameters, it may be time to review the system and make some changes.

Is Backtesting Even Worth It?

So with all these possible pitfalls in translating a backtested system to real life trading, is it even worthwhile to backtest a system?

Think about it this way...

What else are you going to do?

Trade your system untested with real money from day one? Wait several years until you have enough live trading results to declare your system viable? Psychic Friends Network?

Of course not. In my opinion, testing your system over as long a time period as possible is still the best way to find out if that system you learned from a guru or cooked up in your head is a hero or half-baked.

There will be losses in any system, but if you make your system as robust as possible, it should be able to handle most market conditions. But if it does stop working, you will know.

Do you believe that backtesting is worthwhile or totally worthless? I would love to hear your comments below.

To learn more about Hugh and his blog click here.

Trade Well,

Hugh