• Time Series Transformation Engine
• Data Screening Engine
• Data Visualization Tool
• Built-in Fitness Functions
• User Defined Fitness Functions
• Built-in Mathematical Functions
• Function Selection Tool
• Dynamic User Defined Functions
• Static User Defined Functions
• Learning Algorithms
• Monitoring the Modeling Process
• Analyzing Intermediate Models
• Simplifying a Model
• Comparing Actual and Predicted Values
• Essential Statistics
• Modeling from Seed Models
• Adding a Neutral Gene
• Complexity Increase Engine
• Visualizing Models as Parse Trees
• Generating Code Automatically Using Built-in Grammars
• Generating Code Automatically Using User Defined Grammars
• Making Predictions

Time Series Transformation Engine
Before modeling, the data in a time series must be transformed so that it can be used to create prediction models.

GeneXproTools 4.0 comes with a time series transforming engine that transforms the time series according to the Embedding Dimension and Delay Time, but it also transforms the time series depending on whether you wish to make predictions about the future or test the predictive power of the evolved models on known past behavior.

You can then observe the transformation on the Training Data window before proceeding with the new run.

The time series transforming engine of GeneXproTools 4.0 is operational every time you create a new run or every time you change either the Embedding Dimension or the Delay Time, the Prediction Mode or the number of Testing Predictions during a run in the General Settings Tab.

In practice, the embedding dimension corresponds to the number of independent variables (terminals) after your time series has been transformed. The delay time t determines how data are processed, that is, continuously if t = 1 or at t intervals.

These two parameters, together with the size of the time series and the prediction mode, will determine the final number of training samples after the transformation of the time series.

The data are then ready for evolving prediction models with them and can be visualized in the Data Panel before evolving a new model.

Data Screening Engine
Time series have a simple data format which consists of a column of time series observations.

GeneXproTools 4.0 data screening engine makes sure that the right format is fed into the time series transforming engine of GeneXproTools so that the time series is transformed correctly either to create a model for making predictions or testing the predictive potential of the model against known behavior.

Data Visualization Tool
The data visualization tool of GeneXproTools 4.0 enables you to analyze the distribution of all the inputs.

The visualization of the distribution of all data points is a valuable modeling tool as it can help you detect outliers in your data that can then be removed so that better models are created. Typographical or measurement errors generally cause outliers that can be detected by analyzing the time series graphically.

Built-in Fitness Functions
GeneXproTools 4.0 offers a total of 36 built-in fitness functions based on well-known statistical functions.

Fitness functions are fundamental modeling tools for they determine the nature of the search landscape. And different fitness functions open and explore different paths in the solution space. The fitness functions of GeneXproTools 4.0 are named as follows: Absolute with SR, Absolute/Hits, MSE, RMSE, MAE, RSE, RRSE, RAE, Relative with SR, Relative/Hits, rMSE, rRMSE, rMAE, rRSE, rRRSE, rRAE, R-square, and Correlation Coefficient. Furthermore, all these fitness functions have their own counterpart with parsimony pressure.

User Defined Fitness Functions
Despite the wide set of GeneXproTools built-in fitness functions, some users sometimes want to experiment with fitness functions of their own. GeneXproTools 4.0 gives the user the possibility of creating custom tailored fitness functions and evolve models with them.

The code for the custom fitness function must be in JavaScript and is written in the Custom Fitness Function window and can be tested before evolving a model with it.

Built-in Mathematical Functions
GeneXproTools 4.0 offers a total of 279 built-in mathematical functions. This wide set of mathematical functions allows the evolution of complex and rigorous models quickly built with the most appropriate functions.

From the simple arithmetic operators to complex mathematical functions available in most programming languages to more complex mathematical functions commonly used by engineers and scientists, the modeling algorithms of GeneXproTools 4.0 enable you the integration of the most appropriate functions in your models.

Function Selection Tool
For some problems, the simple arithmetic operators are more than enough to create extremely accurate and elegant models. But for other problems more complex functions are necessary to create good models and when inside knowledge of the problem at hand is not producing the expected results, it might be interesting to experiment with different combinations of functions. The Function Selection Tool of GeneXproTools 4.0 helps you to experiment with different function sets very quickly either by pressing the Random button in the Functions Panel or by selecting certain sub-sets of functions.

In addition, although you can always handcraft your function set so that it is well-balanced relatively to the embedding dimension in the problem at hand, in GeneXproTools 4.0 this is no longer essential as GeneXproTools balances automatically all function sets. This is particularly important for high-dimensional datasets.

Dynamic User Defined Functions (DDFs)
Despite the wide set of GeneXproTools built-in mathematical functions, some users sometimes want to model with different ones. GeneXproTools 4.0 gives the user the possibility of creating custom tailored functions and evolve models with them. A note of warning though: the use of dynamic UDFs slows considerably the evolutionary process and therefore should be used with moderation.

The code for the DDFs must be in JavaScript and is written in the Edit DDF window (in the Functions Panel, select the Functions Tab and then click Add in the Dynamic UDFs tool box).

Static User Defined Functions (UDFs)
Sometimes it is possible to spot relatively simple relationships between certain variables in your data, and GeneXproTools 4.0 allows you to use this information to build custom tailored functions (UDFs). These functions allow the discovery of more complex models composed of several simpler models.

The code for the UDFs must be in JavaScript and is written in the Edit UDF window (in the Functions Panel, select the Static UDFs Tab and then click Add in the Static UDFs tool box).

Learning Algorithms
GeneXproTools 4.0 is equipped with two different learning algorithms for evolving complex nonlinear prediction models.

The first is the fastest and simplest of the two. The second algorithm is very similar to the first, with the difference that it gives the user the possibility of choosing the range and type of numerical constants that will be used during the learning process. And because numerical constants are crucial to a wide set of problems, the second algorithm is the default in GeneXproTools 4.0. Keep in mind, however, that for problems where numerical constants are not necessary it pays to use the first algorithm as the models are not only much simpler but also the evolutionary process is much quicker.

In order to evolve models using the second algorithm, select Numerical Constants in the Settings Panel and then check the box that activates this algorithm. You will notice that additional parameters become available, including a small set of genetic operators especially developed for handling numerical constants (if you are not familiar with these operators, please use the default values for they work very well in all cases). To use the first algorithm, just uncheck the Use Random Numerical Constants box.

Monitoring the Modeling Process
GeneXproTools 4.0 allows you to monitor the modeling process by plotting the essential parameters of a run during the discovery process, including average fitness of the population and the fitness and R-square of the best-of-generation model.

In addition, GeneXproTools allows you to visualize the modeling process by plotting the essential features of a run during the discovery process. In the Run Panel, a total of eight different charts are shown:

1. The Variables Usage map.
   This map shows not only the variables that are being used by the best-of-generation models but also their weight. Not only after a run but also during evolution, by placing the cursor over each square you can access the variable ID and the number of times it appears in the current model. And by clicking the right button of your mouse, you can also change its appearance: Heat Map, Random Colors, or Monochromatic.
2. The Evolutionary Dynamics chart.
   This chart shows the average fitness of the population plus the fitness and R-square of the best-of-generation model.
3. The Average/Best Size chart.
   This chart shows the average program size of the population versus the program size of the best-of-generation model. This chart is especially useful during simplification and can be activated any time during evolution by selecting Avg/Best Size in the rightmost combo box in the bottom.
4. The Curve Fitting chart.
   This chart plots the first 250 data points of the training set and shows how well the evolving models are fitting the target. The Curve Fitting chart can be activated any time during evolution by selecting Curve Fitting in the rightmost combo box in the bottom.
5. The Sub-Program Sizes chart.
   This chart shows the sizes of all the sub-programs in the best-of-generation model. Not only after a run but also during evolution, by placing the cursor over each bar you can access the size of all sub-programs in your model. The Sub-Program Sizes chart can be activated any time during evolution by selecting Sub-Program Sizes in the leftmost combo box in the bottom.
6. The All Sizes chart.
   This chart shows the sizes of all the models in the population. Not only after a run but also during evolution, by placing the cursor over each bar you can access the size of a particular model; the best-of-generation model always occupies the first position so you can also easily see how it fares relatively to the others. The All Sizes chart can be activated any time during evolution by selecting All Sizes in the leftmost combo box in the bottom.
7. The All Fitnesses chart.
   This chart shows the fitnesses of all the models in the population. Not only after a run but also during evolution, by placing the cursor over each bar you can access the fitness of a particular model; the best-of-generation model always occupies the first position so you can also easily see how it fares relatively to the others. The All Fitnesses chart can be activated any time during evolution by selecting All Fitnesses in the leftmost combo box in the bottom.
8. The Program Size chart.
   This chart shows the size of the best-of-generation model. Not only after a run but also during evolution, by placing the cursor over the horizontal bar you can access the size of the best model.

The evolutionary process can be stopped whenever you are satisfied with the results by pressing the Stop button or you can use one of the stop criteria of GeneXproTools for picking up your model exactly as you want it.

When the evolutionary process stops, the best-of-run model is ready either for analysis or for making predictions. And if you are still not happy with the results, you can still try to fine-tune the evolved model by pressing the Optimize button and repeat this process until you are completely satisfied with your model. In addition, you might also wish to try and simplify your model and for that you just have to press the Simplify button.

Analyzing Intermediate Models
GeneXproTools 4.0 gives you the possibility of gaining some insights into the modeling process by analyzing all the best-of-generation intermediate models.

All the best-of-generation models discovered during a run are saved and you can pick them up for analysis at the History Panel by checking the model you are interested in. Each of these intermediate models can then be analyzed exactly as you do for the best-of-run model, that is, you can either check their performance in the testing samples if you are working on Testing Mode or use them to make predictions if you are in Prediction Mode, check their vital statistics, automatically generate code with them, visualize their parse trees, use them as seed to create better models from them and so forth.

Simplifying a Model
GeneXproTools 4.0 allows you to simplify an existing model (which could have been either generated by GeneXproTools or by another modeling tool) either by pressing the Simplify button on the Run Panel or by choosing a fitness function with parsimony pressure and then press Optimize or Simplify on the Run Panel.

For models created outside of GeneXproTools or for GeneXproTools models modified by the user, the starting model is fed to the algorithm through the Change Seed window where both the fitness and structural soundness of the model are tested.

Then, in the Run Panel, by clicking the Simplify button, an evolutionary process starts in which all the subsequent models will be descendants of the model you want to simplify. Keep in mind, however, that the simplification algorithm used by GeneXproTools is an evolutionary one and, despite parsimony pressure being applied, models continue to be selected preferentially by fitness and sometimes their complexity might even increase temporarily if the gain in fitness outweighs the loss in simplicity.

For models created in the GeneXproTools 4.0 environment, you just have to select the model you want to simplify (either the best-of-run or an intermediate model picked up in the History Panel) and then click the Simplify button and let the algorithm create better descendants not only in terms of fitness but also in terms of size.

Comparing Actual and Predicted Values
GeneXproTools 4.0 offers two different ways of analyzing and comparing the output of your model with the actual or target values both for training and testing.

In the first, the target or actual values are listed in a table side by side with the predicted or model values. In the second, the target and predicted values are plotted in a Curve Fitting chart for easy visualization. And both table and chart are shown simultaneously in the Predictions Panel. By moving the scroll bar in the right, you can see the results for all the data points.

If you are testing the predictive power of your model using past known observations, the comparison of the predictions with the actual values are listed at the end of the table and highlighted in blue and are also plotted in the chart. In addition, these testing predictions can either be observed together with the training results or separately in a different Curve Fitting plot by selecting Testing in the Chart options.

Essential Statistics
GeneXproTools 4.0 allows a quick and easy assessment of a wide set of statistical functions. Most of them are immediately computed and shown every time you check the Predictions Panel (for instance, mean squared error, root mean squared error, mean absolute error, relative squared error, root relative squared error, relative absolute error, R-square, and correlation coefficient).

If you are testing the predictive power of your model using past known observations, you can also check the same set of statistical indexes (mean squared error, root mean squared error, mean absolute error, relative squared error, root relative squared error, relative absolute error, R-square, and correlation coefficient) obtained for the recursive testing, by selecting Testing in the Chart options. You’ll notice that the values of the statistical functions are updated, and they refer to the results obtained in the testing.

These and other statistical functions (R-square, correlation coefficient, and fitness) are also shown in the Report Panel so that they can be used for future reference, but you must check them first in the Predictions Panel.

Modeling from Seed
GeneXproTools 4.0 allows the use of an existing model (which could have been either generated by GeneXproTools or by another modeling tool) as the starting point of an evolutionary process so that more complex, finely tuned models could be created.

For models created outside of GeneXproTools or for GeneXproTools models modified by the user, the starting model or seed is fed to the algorithm through the Change Seed window where both the fitness and structural soundness of the model are tested.

Then, in the Run Panel, by clicking the Optimize button, an evolutionary process starts in which all the subsequent models will be descendants of the seed you introduced. A note of caution though: if your seed has a very small fitness, you risk losing it early in the run as better models could be randomly created by GeneXproTools and your seed, most probably, would not be selected for breeding new models. If your seed has zero fitness, though, you will receive a warning so that you could modify your seed until it becomes a viable seed capable of breeding new models.

For models created in the GeneXproTools 4.0 environment, the seed is fed to the algorithm every time you click the Optimize or Simplify buttons during modeling, that is, the seed (in this case, the best model of the previous run) is introduced automatically by GeneXproTools.

Adding a Neutral Gene
The addition of a neutral gene to a seed might seem, at first sight, the wrong thing to do as most of the times we are interested in creating efficient and parsimonious models. But one should look at this as modeling in progress as, for really complex phenomena, it is not uncommon to approximate a complex function progressively.

Thus, being able of introducing extra terms in your seed is a powerful modeling tool and GeneXproTools 4.0 allows you to do that by selecting Add Neutral Gene in the Edit Menu or through the Change Seed window.

Here, by pressing the Add Neutral Gene button, you will see a neutral gene being added to your model. By doing this, you are giving the learning algorithm more room to play and, hopefully, a better, more complex program will evolve.

Neutral genes can also be introduced automatically by GeneXproTools 4.0 using the Complexity Increase Engine.

Complexity Increase Engine
GeneXproTools 4.0 also allows you to introduce neutral genes automatically by activating the Complexity Increase Engine in the Settings Panel -> General Settings Tab.

Whenever you are using the Complexity Increase Engine of GeneXproTools 4.0, you must fill the Generations Without Change box to set the period of time you think acceptable for evolution to occur without improvement in best fitness, after which a mass extinction or a neutral gene (an extra term) is automatically added to your model; the Number of Tries box corresponds to the number of consecutive evolutionary epochs (defined by the parameter Generations Without Change) you will allow before a neutral gene is introduced in all evolving models; in the Max. Complexity box you write the maximum number of terms (genes) you’ll allow in your model and no other terms will be introduced beyond this threshold.

The Complexity Increase Engine of GeneXproTools 4.0 might be a very powerful modeling tool, especially for time series prediction models where good models could take longer to come by, but you must be careful not to create excessively complex models as a greater complexity does not necessarily imply a greater predictive power.

Visualizing Models as Parse Trees
GeneXproTools 4.0 comes equipped with a parse tree generator that automatically converts your models into diagram representations or parse trees.

Indeed, the models evolved by GeneXproTools in its native Karva language can be automatically parsed into visually appealing expression trees, allowing a quicker and more complete understanding of their mathematical intricacies.

Generating Code Automatically Using GeneXproTools Built-in Grammars
GeneXproTools 4.0 offers a total of 15 built-in grammars so that the models evolved by GeneXproTools in its native Karva language can be automatically translated into the most commonly used programming languages (Ada, C, C++, C#, Fortran, Java, Java Script, Matlab, Pascal, Perl, PHP, Python, Visual Basic, VB.Net, and VHDL). This code can then be used in other applications.

Generating Code Automatically Using User Defined Grammars
GeneXproTools 4.0 allows the design of User Defined Grammars so that the models evolved by GeneXproTools in its native Karva language can be automatically translated into the programming language of your choice if you happen to prefer a programming language not already covered by the 15 built-in grammars of GeneXproTools 4.0 (Ada, C, C++, C#, Fortran, Java, Java Script, Matlab, Pascal, Perl, PHP, Python, Visual Basic, VB.Net, and VHDL).

As an illustration, the C++ grammar of GeneXproTools 4.0 is shown here. Other grammars may be easily created using this or other GeneXproTools built-in grammars as reference.

Making Predictions
The algorithms of GeneXproTools 4.0 used for Time Series Prediction allow not only the evolution of models but also to use these models to make predictions.

GeneXproTools 4.0 allows you to make two kinds of predictions: one for testing past known events and another for making predictions about the future. In both cases, though, predictions are made recursively, by evaluating the forecast at t +1, then using it to forecast t+2, and so on.

You must choose either one of these methods while loading your time series data, as this imposes some constraints on the restructuring of the time series for training. Namely, n testing records (the n last ones) are saved for testing and sometimes a small number of records from the top must be deleted. However, you can also change these parameters later in the GeneXproTools modeling environment.

Thus, the first type – Testing Mode – can be used for research or pre-evaluation purposes, as it allows you to test the forecasting capabilities of your model on a set of test observations.

The second type – Prediction Mode – is used obviously to predict unknown behavior, and GeneXproTools 4.0 allows you to venture into the future as far as you see fit, by setting the number of predictions you want to make and then click the Predict button. Like for the Testing Mode, the predictions can be plotted together with the Target and the Model output on the training set or be shown separately in a different plot.