For the new project "New Project: Multi-class Classification & Trading Strategies" we implemented 2 new genetic operators: Fixed-Root Mutation and Conservative Fixed-Root Mutation. These genetic operators were implemented in all modeling categories and core algorithms of GeneXproTools (GEP and GEP-RNC). In this post I'll talk about Fixed-Root Mutation and its uses in the context of this new project.

As the name implies, the Fixed-Root Mutation operator implements a different version of the Mutation operator, where the root of each sub-tree is conserved. So, if we have a multigenic system encoding n trees, with the Fixed-Root Mutation operator we can change all the elements in the chromosome except the ones in the first position of each gene (this position encodes the root of the sub-tree).

The Fixed-Root Mutation operator is interesting because it gives us more control over the root position, which might come in handy in problems like the ones we set out to solve with this new project "New Project: Multi-class Classification & Trading Strategies". For example, if we are using a single-tree system for discovering buy-sell-wait rules, by fixing the function of interest (the buy-sell-wait function in this case) at the root of the tree, we might get a faster and more efficient evolution.

The Fixed-Root Mutation operator has also other applications besides fixing the root function in a unigenic system. For example, in multigenic systems we can use it to impose certain structural constraints on our solutions, especially if we use it in combination with other genetic operators. For example, we can warm start the design process with a weak linear solution and then let the learning algorithm discover a better solution to our problem.

In the next post I'll talk about the related Conservative Fixed-Root Mutation operator and its uses.

Both the new classifier functions and the new linking functions introduced with this mini-release give rise to new ways of doing classification in GeneXproTools.

For example, the new classifier functions of 2-6 discrete outputs greatly simplify the creation of classification models in the Regression Framework when we are interested in crisp outcomes for the model outputs.

Furthermore, these new classifier functions can be used both in unigenic and multigenic systems not only in binary classification but also in multi-class classification. It's worth noting, however, that for multi-class classification better results are usually obtained by decomposing the problem in k sub-tasks (k is the number of classes). But as usual in these situations it all depends on the problem and on the data. For example, all the new classifier functions of 3-6 outputs perform great on the Iris dataset (a 3-class classification problem), but on the Balance Scale dataset (also with 3 classes and the same number of input variables) they perform poorly, mainly due to the unbalanced distribution of classes in the original data. This is obviously a problem for which we can find a solution, like for instance, creating a balanced dataset, but the recommended course of action for multi-class classification is to divide the problem into k binary sub-tasks and solve each one of them in the Logistic Regression Framework or Classification Framework where we have access to the probabilities for each class.

Notwithstanding, for multi-class classification problems where these new 3-6 output classifier functions work well, you can either use these new discrete functions in combination with other math functions in systems with just one tree or in systems with multiple trees, linking the sub-trees with the most appropriate linking function. For example, the Min & Max linking functions work particularly well in those cases.

The other alternative is to use the new discrete output linking functions. In this case, you can either add some of the new discrete output functions to your function set or use just a discrete output function for the linking. Particularly interesting is the setup with just 2 trees linked by a function like the buy-sell-wait function described in the post "Function Design: The BUY-SELL-WAIT Function". But this obviously extends to all kinds of 3-class classification problems as long as the data is amenable to this kind of approach.

And to finish, just a quick note on the uses of these new discrete output linkers in the Logistic Regression Framework and the Classification Framework.

GeneXproTools allows you to use the new linking functions both in Logistic Regression and Classification, but you must be aware that the models you're creating are much simpler (which might be great if you're interested in just that) than the ones created with non-discrete linkers (addition, min, max, Avg2 and others). Therefore these simpler models won't give you much differentiation in terms of probabilities. But again, this might prove useful in certain problems or as a form to gain insight into your data.

So you're free to explore all these new classification algorithms by exploring different combinations of fitness functions (the fitness functions of Regression, Logistic Regression and Classification were all fine-tuned for different purposes), visualization tools (the model fitting charts of Regression and Logistic Regression/Classification are all different), and analytics (the measures of fit used in Regression and Logistic Regression/Classification are also different).

Have fun!

The new linking functions added to GeneXproTools with this mini-release include a total of 6 discrete output classifier functions of 2 and 3 outputs. All these functions showed good performance on the Iris dataset both as normal building blocks when added to the default function set and also as linking functions in systems with 2 and more genes.

In the 3-output category we implemented 3 new linking functions, all of them with outputs of the form {-1, 0, 1}. One of these linkers is the buy-sell-wait function (represented in GeneXproTools as CL3A) described in the post "Function Design: The BUY-SELL-WAIT Function". The other two are the new 3-output classifier functions CL3B and CL3C introduced in the post "Function Design: More 3-Output Classifier Functions".

In the binary category, we also added 3 new linking functions: CL2A, CL2D, and the argmin function of 2 arguments (represented as AMin2 in GeneXproTools). I didn't talk much about these functions, but they all perform very well on the Iris dataset, sub-problem Virginica/Not Virginica. But the main reason for their inclusion and not others (for example the functions CL2B and CL2C) was the fact that they have a neutral gene.

The C++ code for all these new linking functions is shown below: