How to run a test
Testing with MetaTrader 4 or MetaTrader 5?
A frequently asked question is which version of MetaTrader should be used for strategy testing. This section will explain why MetaTrader 5 should be the primary choice for performing strategy tests and when it might still be feasible to use MetaTrader 4.
MetaTrader 4
The strategy tester view in MetaTrader 4 is simpler and easier to use than MetaTrader 5's strategy tester, but this comes with significant disadvantages. MetaTrader 4 does not use real tick data or an integrated mature tick data management tool. While third-party solutions that modify MetaTrader 4 to use real tick data are available, it begs the question: Why go through this process when MetaTrader 5 already fulfills every requirement to perform high-quality tests with precise tick data?
Even though MetaTrader 4's strategy tester offers the option Every tick as a selectable data model, it does not use real tick data. MetaTrader 4 was released at a time when high-frequency trading wasn't prevalent. In the early 2000s, simple long-term SMA and EMA strategies worked quite well in almost every timeframe. Therefore, there wasn't a real need for tick-precise data. Instead, MetaQuotes decided to emulate tick data to keep it simple. This outdated feature is explained in detail in this external article; please look at it.
For 2002, this could have been better but acceptable and simply was a product of its time. Tick data back then was not available nor practical to store. Twenty years of tick data in EURUSD require almost 30 GB of hard drive storage. In a time when 20GB drives were standard, this was not practical. Thus, MetaQuotes decided to emulate tick data. It is questionable why MetaQuotes still allows this data model or, more precisely, still calls it Every tick.
Countless unscrupulous expert advisor providers in the MetaTrader built-in market and on other websites provide advertisements with astronomical performance in the strategy tester to catch users without the knowledge of why this is as far from reality as possible. It is easy to trick the MetaTrader 4 strategy tester by performing trades based on this predictable interpolation.
From a contemporary perspective, the only use case for the MetaTrader 4 strategy tester is for the initial sketch phase with the visual mode enabled.
Never trust a performance diagram made with MetaTrader 4!
MetaTrader 5
In contrast, the 2008 released MetaTrader 5 and its strategy tester can perform strategy tests based on real tick data. The model is named Every tick based on real ticks. Only by using this data model combined with 10+ years of real tick data, you can adequately evaluate a trading strategy's performance and drawdown. There is just one problem: Where do you get tick data?
MetaTrader 5 has a built-in feature that provides users with tick data, but only for recent months. Since the tick data source is usually your broker, there is a high chance that this tick data does not represent an accurate historical chart because brokers like to polish the tick data they offer.
Nevertheless, this is an excellent start to performing high-quality strategy tests, but it could be better. The optimal solution would be to use independently collected tick data for 10+ years with a spread profile similar to or adopted to the actual broker you use. For the first, you can subscribe to one of the MT5 Tick Data plans to unlock access to a wide range of precise historical tick data for up to 20 years into the past! For the second, a service is available for adopting real tick data to the spread profile typical for your brokers' server.
Trust only the performance diagrams executed by MetaTrader 5 and performed with Every tick based on real ticks as the data model. Moreover, a proper strategy test should have at least 100+ simulated trades.
Defining of a reference test range
Trend check and trading timeframe
There are three primary trends in trading: short-term, intermediate-term, and long-term. A trend represents the general direction the market is heading over a specified period. Markets can trend upward, downward, or move sideways.
A fundamental rule in trading is to never trade against the trend!
Therefore, analyzing trends across multiple timeframes is crucial to ensure that every trade is aligned with this rule. The first step is to define the timeframe you intend to perform trading. The trading timeframe is primarily used to analyze the market for short-term opportunities and signal generation. Most of the decision-making process will occur within the trading timeframe, making it the primary source for generating signals. However, every effective trading strategy should include at least one filter to invalidate poor trading signals. For instance, a bad signal might indicate opening a long trade even though the long-term or intermediate trend is downward.
There are standard recommendations for which timeframe combinations work well together. The table below provides these recommendations.
| Trend type / Trading style | Scalping | Daytrading | Swing trading | Investing |
|---|---|---|---|---|
| Long term trend | M30 | H4 | D1 | MN |
| Intermediate trend | M15 | H1 | H4 | W1 |
| Short-term trend (Trading) | M1 | M15 | H1 | D1 |
If you want your automated trading systems to trade in the H1 timeframe, then H4 would be your intermediate trend, and D1 would be your long-term trend.
Reference test range
The reference test range is established by selecting a period within the long-term trend chart that encompasses all three market stages. This range must include an upward, downward, and sideways phase, and the cumulative change shall be around zero. To consider an automated trading system as long-term stable, it's essential to cover a sufficiently large period in the long-term trend. For instance, if the H1 trading timeframe is selected, the D1 chart should be utilized.
The chart below illustrates that this may require several years of tick data, even though the actual trading occurs in the H1 timeframe.
Backtesting and forward testing are vital methods in automated trading for validating the effectiveness of a trading strategy. The selected periods for both tests should not overlap. As depicted in the timeline above, the backtest range is at least twice the reference test range to ensure coverage of different trend phases.
The forward test range should be at least the same duration as the reference test range. This period is used to validate the trading strategy at the end of the development process. It's important to note that a strategy should never be optimized using the forward test period. The forward test period is exclusively reserved for the final performance test.
Test phases - Data model
Automated trading systems can contain countless input parameters, and the number of possible combinations can be enormous. Conducting an optimization test run that examines every combination would take an inordinate amount of time. Therefore, it's crucial to have a strategy in place for efficient iterative testing. The development of an automated trading system can generally be divided into four stages: planning, iterative optimization, performance evaluation, and forward testing.
Iterative testing in this context refers to testing the effect of each input parameter independently of the others. For instance, if you want to understand the impact of the trailing stop loss on a trading strategy, keep the signal generation static. This approach allows you to quantitatively assess the effect each trailing method has on the final performance chart.
Please note that the iterative phase may sometimes look differently than illustrated. There is a significant degree of flexibility depending on your specific trading strategy. Regardless, it's recommended to consider what sub-phases your automated trading system will have and how to verify each one correctly. The strategy test based on this illustration could serve as a data model for performing automated testing.
Test phases - Planning
The purpose of the planning phase is to conceptualize the trading idea. During this phase, it's essential to answer the most fundamental questions and address critical aspects of market analysis. Please note that this list needs to be completed and may need to be adapted based on your individual trading experience. Some questions may not be applicable, or additional questions may need to be added. Below is a list of questions that might be helpful to answer initially:
- What timeframes should be used for opening and closing trades?
- In which timeframe should the intermediate trend be identified, and what phase of the trend is required?
- In which timeframe should the long-term trend be determined, and what phase of the trend is necessary?
- What level of market volatility is needed in the trading timeframe?
- What level of market volatility is required in the intermediate timeframe?
- What level of market volatility is necessary in the long-term timeframe?
- What is the current momentum in the trading timeframe?
- Is the current spread sufficiently small for trading?
- Are there any economic news events that could significantly influence my trading idea?
- Are there any existing support and resistance levels that need to be considered?
- Where are the current Fibonacci retracements located?
- How risky should my trading strategy be?
- Do I want to avoid overnight swaps?
- And so on…
A recommended approach is creating a document describing the actual trading idea and answering the above questions. Once all these questions are answered, the idea can be roughly implemented using the visual mode of the MetaTrader strategy tester.
This is the only use case for the MetaTrader 4 strategy tester. Since neither the performance nor the accuracy of the test matters in this test phase, it is convenient to use MetaTrader 4 as tests start slightly faster. It doesn't matter which test period or data model you select; the only purpose is to configure the Expert Advisor Builder functional blocks to match your trading idea. For example, if your trading idea involves two indicators and two oscillators, you should check whether the modules are enabled adequately after setting all input parameters.
Test phases - Iterative optimization
The subsequent step involves the iterative optimization test phase. This phase aims to identify the optimal input parameters for each Expert Advisor Builder functional block. Given that the final trading decision is a cumulative result of all parameters, it's nearly impossible to instantly test all combinations or find the ideal input parameters. Each trading idea will have several local lows and highs, yielding acceptable but not perfect results. It's vital to circumvent the common optimization trap by limiting the number of parameters open to modification. A practical approach is to select 1-3 values for optimization while keeping all other values constant. Once a local performance high is identified, you can set additional parameters. Please note that once optimized, input parameters should not be modified again.
For iterative testing, it's recommended to use a combination of single test runs and optimization test runs. The results should be double-checked with a visual test. Testing with OHLC or Every tick based on real ticks is recommended as the data model and at least twice the reference test range. The MetaTrader 5 strategy tester should be utilized for this purpose.
Test phases - Performance
For this testing phase, it's recommended to use Every tick based on real ticks as the data model and to utilize all tick data, excluding those reserved for the forward test.
The primary objective of this phase is to assess the expected performance and drawdown of the developed trading system. A quick way to do this is by examining the performance chart. As mentioned, the cumulative price change within the reference test period should be approximately zero. This means that if you had invested traditionally and without fees at the beginning of the reference test range, there would be neither loss nor profit at the end of it. While this is a theoretical assumption, it's a fantastic basis for evaluating performance.
Based on this assumption, it's possible to categorize the automated trading system into three groups: overperforming, neutral performing, and underperforming strategies. Select significant data points in the price chart to compare the performance over the entire reference test period. Usually, the performance line looks similar to the actual price chart. Thus, you can compare whether the performance results are above or below the reference value.
Overperforming strategy
Remember, the price chart change within the tested reference period is 0%. An overperforming strategy is defined by most (more than 75%) reference checkpoints above the imaginary reference line. This line is illustrated in the image below.
As can be seen, the trading system in this example is a very successful approach because it significantly outperforms the reference market. This can be considered an automated trading system with long-term stability.
Neutral performing strategy
A neutral-performing automated trading system has checkpoints above and below the reference line. A trading system with this characteristic may be profitable over a long period, but it could also slowly but steadily result in a loss. If the performance line looks like this, revisiting the input parameters and returning to the iterative test phase is recommended.
An automated trading system with this performance line can often be adjusted to become profitable and stable. Therefore, there's no need to discard it immediately.
Underperforming strategy
If the performance line looks like the image below, there's a high chance that the automated trading system systematically trades with losses.
This strategy underperforms the reference market at any given time and will most likely continue to do so. If your strategy produces a performance line like this, it should not be applied to a live chart.
Test phases - Forward
The final step in developing an automated trading strategy with the Expert Advisor Builder is to conduct a forward test. This involves using a tick data period that has not previously been used for any tests. This aims to simulate live trading with the most recent tick data to verify the trading idea's profitability. If your trading idea outperforms the most recent market performance, there's a high chance you've found a long-term stable trading system ready to be applied to a live chart!