It is supposed that an Expert Advisor having inputs adjusted to the history will trade to a profit for the first (rather short) time. Indirect confirmations of this suggestion appeared after I had watched the Automated Trading Championship 2006. When the Championship started there were much more profitable Expert Advisors than later, when some of them turned to be noncompetitive. This is why I suppose the most of those Expert Advisors that had not come to the finish were adjusted to the history.
The idea to check this supposition in practice was born on the Russian forum of this website, in the section of Ideal Automated Trading System. The main idea is to start optimization of an EA automatically once a day and then analyze the obtained optimization results and record them in the EA's variables.To implement this idea, we decided to take the ready-made Expert Advisor, MACD Sample, from the MetaTrader 4 Client Terminal and insert our own function of automated optimization into it. A bit later, the code of that automated optimizer was ready and uploaded in the same forum, in the section of Automated Optimizer. After some more time, the first confirmations of the idea appeared in the branch of Automated Optimizer. Later on, the optimizer was transformed into an mqh-library for better usability.
Below is what must be done for performing this task:
For better usability, we will hereinafter name the original terminal "Terminal", whereas the copy will be named "Terminal-Tester". We will perform a checking test on EURUSD at timeframe H1 using the Expert Advisor initially provided by the terminal, but slightly changed - MACD Sample_1.mq4.
Terminal-Tester Setup
Please don't forget to compile MACD Sample_1.mq4 in the Terminal-Tester. Let's start the client terminal first, then strategy tester, and set it up as shown in the screenshot below.
Optimization will be performed for three days. This is quite enough to check the automated optimizer. We will select the optimization date "From" according to the following formula - the current date minus three days. During optimization, the necessary history for the selected symbol (in our case, it is EURUSD) must be downloaded.
Those who perform optimization for the first time can find the description of necessary procedures in the Help menu of the MetaTrader 4 Client Terminal: <Help - Help Topics F1 - Auto Trading - Expert Optimization - Setup>. Or they can read the article named Testing of Expert Advisors in the MetaTrader 4 Client Terminal: An Outward Glance.
Then let's check variables to be optimized as shown in the screenshot below.
datetime SetHour = 0; // Optimization starting hour; datetime SetMinute = 1; // Optimization starting minute.
- set the amount of days for optimization (it must be the same as that for pre-optimization):
int TestDay = 3;
- set the optimization end waiting time, in minutes, that we have calculated before, for example, 4 minutes:
int TimeOut = 4;
- type in the Expert Advisor name:
string NameMTS = "MACD Sample_1"; // EA's name
- type in the set-file name with the settings:
// Set-file name with the settings string NameFileSet = "MACD Sample_1.set";
- type in the path to the folder containing the installed Terminal-Tester, for instance:
// Path to the testerstring PuthTester = "D:\Program Files\Forex Best Trade Station";
- set the priority for filtering:
// Sorting by Maximal profit int Gross_Profit = 1;// Sorting by Maximal profit factor int Profit_Factor = 2;// Sorting by Maximal expected payoff int Expected_Payoff = 3;
- write variable names for optimization:
string Per1 = "FastEMA";string Per2 = "SlowEMA";string Per3 = "SignalSMA";string Per4 = "";
- copy attached file auto_optimization.mqh to the 'include' folder';
- include the library file in the Expert Advisor:
//--- Including the auto optimizer's library#include <auto_optimization.mqh>
- it is only remains to copy the code below to the beginning of the start() function of your Expert Advisor. MACD Sample_1.mq4 contains it already.
// Not to be launched at testing and optimizing if(!IsTesting() && !IsOptimization()){// Compare the current hour with that preset for launchingif(TimeHour(TimeLocal()) == SetHour){// Protection against restartingif(!StartTest){// Compare the minute range to the minute// preset for launchingif(TimeMinute(TimeLocal()) > SetMinute - 1){// the range is necessary, in case that for some reasons // no new tick is available for a long timeif(TimeMinute(TimeLocal()) < SetMinute + 1){// Flag of tester launchingStartTest = true;TimeStart = TimeLocal();Tester(TestDay, NameMTS, NameFileSet, PuthTester,TimeOut, Gross_Profit, Profit_Factor,Expected_Payoff, Per1, Per2, Per3, Per4);}}}}}FastEMA = GlobalVariableGet(Per1);SlowEMA = GlobalVariableGet(Per2);SignalSMA = GlobalVariableGet(Per3);TrailingStop = GlobalVariableGet(Per4);// If the tester launching is flagged if(StartTest){// If more time has elapsed the launching than it was set // to be the test waiting timeif(TimeLocal() - TimeStart > TimeOut*60){// Zeroize the flagStartTest = false;}}
That's all. After the automated optimizer has been recompiled, it can be launched, but it may be used for only the same symbol and timeframe, for which the pre-optimization was performed. In our case, it is EURUSD on H1. To check the auto optimizer, you can insert the code given below in the int init () function, then the auto optimizer will be launched at the Expert Advisor's start.
Tester(TestDay,NameMTS,NameFileSet,PuthTester,TimeOut, Gross_Profit,Profit_Factor,Expected_Payoff, Per1,Per2,Per3,Per4);
The automated optimizer works on the basis of using Terminal-Tester to optimize parameters of the Expert Advisor attached to the chart in Terminal. For this, the program sends to the Terminal-Tester a file containing optimization parameters (optimise.ini) and launches the Terminal Tester in the optimization mode. Then it copies the obtained results of "FileReport........htm" back to the Terminal and filters the best values from the obtained results.
// Path to the terminal string PuthTerminal = TerminalPath() + "\experts\files";// Name of the ini file for the testerstring FileOptim = "optimise.ini";string FileOptim1 = "\optimise.ini";// Calculation of the starting datedatetime DayStart = TimeLocal()-86400*TestDay;// Optimization starting datestring DateStart = TimeToStr(DayStart,TIME_DATE);// Optimization ending datestring DateStop = TimeToStr(TimeLocal(),TIME_DATE);// Tester report file namestring FileReport = "FileReport_" + Symbol() + "_" + DateStop + ".htm";string FileReport1 = "\FileReport_" + Symbol() + "_" + DateStop + ".htm";// Limitation for the minimum amount of trades per daydouble MinTr = TestDay - 2;// Limitation for maximal amount of trades per daydouble MaxTr = (60 / Period()*TestDay) + 2;// The amount of attempts to copy the report fileint KvoPptk = 10;// The amount of lines for sortingint StepRes = 12;
Then the parameters of the ini file are written in the string array:
// Prepare the ini file for optimizationArrayOpttim[0] = ";optimise strategy tester";// Enable/Disable Expert AdvisorsArrayOpttim[1] = "ExpertsEnable = false";// Name of the EA fileArrayOpttim[2] = "TestExpert=" + NameMTS;// Name of the file containing parametersArrayOpttim[3] = "TestExpertParameters=" + NameFileSet;// SymbolArrayOpttim[4] = "TestSymbol=" + Symbol();// TimeframeArrayOpttim[5] = "TestPeriod=" + Period();// Modeling modeArrayOpttim[6] = "TestModel=" + 0;// RecalculateArrayOpttim[7] = "TestRecalculate=false";// OptimizationArrayOpttim[8] = "TestOptimization=true";// Use dateArrayOpttim[9] = "TestDateEnable=true";// FromArrayOpttim[10] = "TestFromDate=" + DateStart;// ToArrayOpttim[11] = "TestToDate=" + DateStop;// Report file nameArrayOpttim[12] = "TestReport=" + FileReport;// Rewrite the report fileArrayOpttim[13] = "TestReplaceReport=true";// Shut down the terminal upon completionArrayOpttim[14] = "TestShutdownTerminal=true";
The optimization parameters are recorded into the ini file from the array. You can also read about how to create an ini file in the MetaTrader 4 Client Terminal Help, see <Help - Help Topics F1 - Tools - Configuration at Startup>.
// Write data into the ini file // Find out about the array sizeOptimArraySize = ArraySize(ArrayOpttim);// Open a file to writeopttim = FileOpen(FileOptim, FILE_CSV|FILE_WRITE, 0x7F);if(opttim > 0){for(int i = 0; i < OptimArraySize; i++){// from the array into the variableini = ArrayOpttim[i];// from the variable into the fileFileWrite(opttim, ini);}// close the fileFileClose(opttim);}else{Print("Failed writing data into the ini file. Error No ",GetLastError());return(0);}
After the parameters have been recorded in the ini file, the shell32.dll included in the standard Windows delivery is connected and function ShellExecuteA is launched.
#import "shell32.dll" //Connect a dll (provided with Windows) int ShellExecuteA(int hwnd,string Operation,string File,string Parameters,string Directory,int ShowCmd);#import
File containing the parameters will be sent to the Terminal Tester folder.
// copy the ini file into the tester folder copyini = ShellExecuteA(0,"Open","xcopy", "\"" + PuthTerminal +FileOptim1 + "\" \"" + PuthTester + "\" /y","", 3);// wait until the file is copiedSleep(1200);if(copyini < 0){Print("Failed copying ini file");return(0);}
Then the tester is launched and starts to optimize the predefined variables. The Expert Advisor is in the halted state during optimization.
// Start Tester start = ShellExecuteA(0, "Open", "terminal.exe", FileOptim,PuthTester, 3);if(start < 0){Print("Failed starting Tester");return(0);}Comment("Wait until optimization is complete");// wait until optimization is completeSleep(60000*TimeOut);
After optimization is complete, Tester will automatically record the results in the report file. That file is copied to the folder that contains the terminal.
for(Pptk = 0; Pptk < KvoPptk; Pptk++){//Start a cycle attempting to compy the resport fileComment("Attempt # " + Pptk + " to copy the report file");ShellExecuteA(0, "Open", "xcopy", "\"" + PuthTester + FileReport1 +"\" \"" + PuthTerminal + "\" /y", "", 3);// wait until the file is copiedSleep(1200);// Try to open the report filefile = FileOpen(FileReport, FILE_READ, 0x7F);if(file < 0){// if it fails to open, wait some more and try againSleep(60000);}elsebreak;}if(file < 0){Print("Failed copying the report file");return(0);}
Then the data from the report file will be placed in the string array for further processing.
// Read from file into the array// Cycle, until the file endswhile(FileIsEnding(file) == false){// Read a string from the report fileFileLine = FileReadString(file);// Find the necessary string and set the reference point thereindex = StringFind(FileLine, "title", 20);if(index > 0){// Increase the array in sizeArrayResize(ArrayStrg, NumStr + 1);// Record the strings from the file in the arrayArrayStrg[NumStr] = FileLine;NumStr++;}}// Close the fileFileClose(file);// Delete the file in order not to produce too many copiesFileDelete(FileReport);// Set the array size by the amount of data read from the fileArrayResize(ArrayData, NumStr); strings.
Then the necessary values are selected in the array:
for(text = 0; text < NumStr; text++){select = ArrayStrg[text];//-------------------------------------------------------------------------// Reporting text processing (These are apples and oranges) | //-------------------------------------------------------------------------// Position Pass ClStep=StringFind(select, "; \">",20)+4;// Find the end of positionClStepRazm = StringFind(select, "td>", ClStep);// Read the valueCycleStep = StringSubstr(select, ClStep, ClStepRazm - ClStep);// Position Profit // Find the beginning of the positionGrProf = StringFind(select, "", ClStepRazm);// Find the end of positionGrProfRazm = StringFind(select, "td>", GrProf);// Read valueGrossProfit = StringSubstr(select, GrProf+4, GrProfRazm - (GrProf + 4));// Position Total Trades// Find the beginning of positionTotTrad = StringFind(select, "", GrProfRazm);// Find the end of positionTotTradRazm = StringFind(select, "td>", TotTrad);// Read the valueTotalTrades = StringSubstr(select, TotTrad+4, TotTradRazm -(TotTrad + 4));// Position Profitability// Find the beginning of positionProfFact = StringFind(select, "", TotTradRazm);// Find the end of positionProfFactRazm = StringFind(select, "td>", ProfFact);// Read the valueProfitFactor = StringSubstr(select, ProfFact + 4, ProfFactRazm -(ProfFact + 4));// Position Expected Payoff // Find the beginning of positionExpPay = StringFind(select, "", ProfFactRazm);// Find the dn of positionExpPayRazm=StringFind(select, "td>", ExpPay);// Read the valueExpectedPayoff = StringSubstr(select, ExpPay+4, ExpPayRazm -(ExpPay + 4));// Variables' positions starting with the second one// Find the beginning of positionP1 = StringFind(select, Per1, 20);// Find the end of positionP1k = StringFind(select, ";", P1);// Read the VariablePerem1 = StringSubstr(select, P1 + StringLen(Per1) + 1, P1k -(P1 + 1 + StringLen(Per1)));// Find the beginning of positionP2 = StringFind(select, Per2, 20);// Find the end of positionP2k = StringFind(select, ";", P2);// Read the VariablePerem2 = StringSubstr(select, P2 + StringLen(Per2) + 1, P2k -(P2 + 1 + StringLen(Per2)));// Find the beginning of positionP3 = StringFind(select, Per3, 20);// Find the end of positionP3k = StringFind(select, ";", P3);// Read the VariablePerem3 = StringSubstr(select, P3 + StringLen(Per3) + 1, P3k -(P3 + 1 + StringLen(Per3)));// Find the beginning of positionP4 = StringFind(select, Per4, 20);// Find the end of positionP4k = StringFind(select, ";", P4);// Read the Variable Perem4 = StringSubstr(select, P4 + StringLen(Per4) + 1, P4k -(P4 + 1 + StringLen(Per4)));Comment("The obtained results are being analyzed");
After that, the obtained results, before they are transformed into number format, have been filtered by the minimal and the maximal amount of trades. Zero in the value of Profit_Factor is replaced with 1000 for correct sorting and subsequent sifting.
// Transform into number formatTotalTradesTransit = StrToDouble(TotalTrades);GrossProfitTransit = StrToDouble(GrossProfit);ExpectedPayoffTran = StrToDouble(ExpectedPayoff);nodubl = true;if(MinTr < TotalTradesTransit && MaxTr > TotalTradesTransit){// Filter by the amount of tradesPrFactDouble = StrToDouble(ProfitFactor);// Replace 0 in the Profit_Factor for proper analysisif(PrFactDouble == 0){PrFactDouble = 1000;}
Then the values are checked for duplications and filtered out.
// Filter data having identical valuesfor(Dubl = 0; Dubl <= ResizeArayNew; Dubl++){// Start the loop searching for identical valuesif(GrossProfitTransit == ArrayData[Dubl][1]){// check whether the results for maximal profit coincideif(TotalTradesTransit == ArrayData[Dubl][2]){// check whether the results for the amount of trades coincideif(PrFactDouble == ArrayData[Dubl][3]){// check whether the results for Profit Factor coincideif(ExpectedPayoffTran == ArrayData[Dubl][4]){// check whether the results for expected payoff coincidenodubl=false;// If everything coincides, flag it as coincided}}}}}
Then the values prepared for sorting are written in the array.
// Write the filtered data in the arrayif(nodubl){ArrayData[text][1] = GrossProfitTransit;ArrayData[text][2] = TotalTradesTransit;ArrayData[text][3] = PrFactDouble;ArrayData[text][4] = ExpectedPayoffTran;ArrayData[text][5] = StrToDouble(Perem1);ArrayData[text][6] = StrToDouble(Perem2);ArrayData[text][7] = StrToDouble(Perem3);ArrayData[text][8] = StrToDouble(Perem4);ResizeArayNew++;}
Then the data start to be analyzed in the preset priority order. The analysis is performed as follows:
// Analyzer// Analyzing principle is the sequential checking of maximal // values according to the predefined filtering priority ArrayResize(ArrayTrans, ResizeArayNew - 1);for(int PrioStep = 1; PrioStep < 4; PrioStep++){for(PrCycle = 0; PrCycle < ResizeArayNew; PrCycle++){Sort = ArrayData[PrCycle][0];Prior1 = ArrayData[PrCycle][1];transit = ArrayData[PrCycle][2];Prior2 = ArrayData[PrCycle][3];Prior3 = ArrayData[PrCycle][4];transit1 = ArrayData[PrCycle][5];transit2 = ArrayData[PrCycle][6];transit3 = ArrayData[PrCycle][7];transit4 = ArrayData[PrCycle][8];if(PrioStep == 1){//Prepare for the 1st sortingif(Gross_Profit ==1){SortTrans = Prior1;}if(Profit_Factor == 1){SortTrans = Prior2;}if(Expected_Payoff == 1){SortTrans = Prior3;}}if(PrioStep == 2){// Restoreif(Gross_Profit ==1){Prior1 = Sort;}if(Profit_Factor == 1){Prior2 = Sort;}if(Expected_Payoff == 1){Prior3 = Sort;}//Prepare for the 2nd sortingif(Gross_Profit == 2){SortTrans = Prior1;}if(Profit_Factor == 2){SortTrans = Prior2;}if(Expected_Payoff == 2){SortTrans = Prior3;}}if(PrioStep == 3){// Restoreif(Gross_Profit == 2){Prior1 = Sort;}if(Profit_Factor == 2){Prior2 = Sort;}if(Expected_Payoff == 2){Prior3 = Sort;}//Prepare for the 3rd sortingif(Gross_Profit ==3){SortTrans = Prior1;}if(Profit_Factor == 3){SortTrans = Prior2;}if(Expected_Payoff == 3){SortTrans = Prior3;}}ArrayTrans[PrCycle][0] = SortTrans;ArrayTrans[PrCycle][1] = Prior1;ArrayTrans[PrCycle][2] = transit;ArrayTrans[PrCycle][3] = Prior2;ArrayTrans[PrCycle][4] = Prior3;ArrayTrans[PrCycle][5] = transit1;ArrayTrans[PrCycle][6] = transit2;ArrayTrans[PrCycle][7] = transit3;ArrayTrans[PrCycle][8] = transit4;}ArraySort(ArrayTrans,StepRes, 0, MODE_DESCEND); // Sort the arrayArrayResize(ArrayTrans, StepRes); // Cut off the unnecessary thingsfor(int CopyAr = 0; CopyAr < StepRes; CopyAr++){ArrayData[CopyAr][0] = ArrayTrans[CopyAr][0];ArrayData[CopyAr][1] = ArrayTrans[CopyAr][1];ArrayData[CopyAr][2] = ArrayTrans[CopyAr][2];ArrayData[CopyAr][3] = ArrayTrans[CopyAr][3];ArrayData[CopyAr][4] = ArrayTrans[CopyAr][4];// Per1 Variable 1ArrayData[CopyAr][5] = ArrayTrans[CopyAr][5];// Per2 Variable 2ArrayData[CopyAr][6] = ArrayTrans[CopyAr][6];// Per3 Variable 3ArrayData[CopyAr][7] = ArrayTrans[CopyAr][7];// Per4 Variable 4ArrayData[CopyAr][8] = ArrayTrans[CopyAr][8];}StepRes = StepRes / 2;}
The values filtered in this manner are written in global variables. The values from global variables will be substituted in the EA.
// Write the obtained results in variablesdouble Peremen1 = ArrayTrans[0][5];double Peremen2 = ArrayTrans[0][6];double Peremen3 = ArrayTrans[0][7];double Peremen4 = ArrayTrans[0][8];// If the variable name is specified, write the result in // global variablesif(Per1 != ""){GlobalVariableSet(Per1, Peremen1);}if(Per2 != ""){GlobalVariableSet(Per2,Peremen2);}if(Per3 != ""){GlobalVariableSet(Per3,Peremen3);}if(Per4 != ""){GlobalVariableSet(Per4,Peremen4);}Comment(Per1, " ", Peremen1, " | ", Per2, " ", Peremen2, " | ", Per3," ", Peremen3, " | ", Per4, " ", Peremen4);Print(Per1, " ", Peremen1, " | ", Per2, " ", Peremen2, " | ", Per3," ", Peremen3," | ",Per4," ",Peremen4);} // Function ends. That's all, automated optimization is complete.
The operation results of the automated optimizer can be observed using messages that appear in the upper left corner of the chart as shown in the screenshot below:
// Limitation of minimal amount of trades per daydouble MinTr = TestDay - 2;// Limitation on maximal amount of trades per daydouble MaxTr = (60 / Period()*TestDay) + 2;// The amount of attempts to copy the report fileint KvoPptk = 10;// The amount of strings to be sortedint StepRes = 12;
This article is not aimed at teaching newbies elements of optimization, so it is highly recommended to learn normal optimization before setting up automated optimization of your Expert Advisor. It is better to use the automated optimizer after you have chosen the basic variables that will influence your Expert Advisor differently at different times. I.e., it is better to use this automated optimizer to fit parameters of the variables the changes in which influence the EA operation more than those in other variables, depending on the market volatility.
Besides, it is better not to set a very large automated optimization period. Suppose the Expert Advisor has been optimized for 6-12 hours every day. Then a question occurs: When will it trade? In other words, optimization is not necessary as itself. It is recommended to set optimization periodicity (the periodicity of optimizer launching is meant) considering the timeframe, on which the EA is supposed to trade. This means that it is necessary to consider that the historical data will be pumped when the Tester-Terminal is started and it is possible that the broker just does not possess the necessary historical data for the specified period of time. To verify the hypothesis described at the beginning of this article, you will need 24-hour and stable internet connection.
The automated optimization programs developed are located in the attached files: auto_optimization.mqh - the library itself, MACD Sample_1.mq4 - slightly changed Expert Advisor included in the MetaTrader 4 Client Terminal standard delivery set
联系客服