This is a story about failures. And I'm telling you about them because without them I would never have had the flash of inspiration that was needed to come up with good 4-6 discrete output classifier functions.

I first started my exploration with functions of 2 arguments with 4-6 discrete outputs. The reason for this was that I also wanted functions that worked well as linking functions in GeneXproTools, which as you know is for now limited to functions of 2 arguments. The idea was to then scale up these 2-argument functions to 3- and 4-argument functions, which is the current arity limit in GeneXproTools.

And here's the C++ code for the first failure:

    if ((arg[0] > 0.0 && arg[0] <= 1.0) && (arg[1] > 0.0 && arg[1] <= 1.0))
        output = 1.0;
    else
        if ((arg[0] > 1.0 && arg[0] <= 2.0) && (arg[1] > 1.0 && arg[1] <= 2.0))
            output = 2.0;
        else
            if ((arg[0] > 2.0 && arg[0] <= 3.0) && (arg[1] > 2.0 && arg[1] <= 3.0))
                output = 3.0;
            else output = 0.0;
    return output;

As usual, I tested it on the Iris dataset with just 3 classes and it just didn't work: most of the time it got stuck indefinitely on just 2 classes. I also noticed that the extra redundancy was playing a positive role and so I decided to extend this design to 6 discrete outputs. But again, most of the time it got stuck on just 2 classes!

I tried other designs, exploring symmetry around zero, but again the functions performed poorly. Here's the code of another failure:

    if (arg[0] >= 1.0 && arg[1] >= 1.0)
        output = 1.0;
    else
        if (arg[0] <= -1.0 && arg[1] <= -1.0)
            output = 2.0;
        else
            if (arg[0] >= 1.0 && arg[1] <= -1.0)
                output = 3.0;
            else output = 0.0;
    return output;

At this point I was starting to see why things weren't working the way I wanted them to work, and frankly I couldn't see how I could change this. How could I get around the fact that some lucky class output (usually the one under the last ELSE clause) got all the slack that it needed while the others got just a tiny little fraction of the real line to work with? And was this the real problem here? Or was it something else?

At this point I was scratching my head and resisting implementing the argmin and argmax functions of 3 and 4 arguments (they were on my long list of functions to be added to GeneXproTools) because they could not solve my problem for the linking functions (remember, they must be functions of 2 arguments)!

So out of desperation I decided to implement a new class of functions based on modulo n functions (an old untested idea of mine for the built-in math functions of GeneXproTools) that would fairly divide the real line estate among the different outputs. And if it worked I could scale it up to as many discrete outputs as I wanted and implement it as a function of 2, 3, 4 and even 1 arguments! Who needed the argmin and argmax functions?

Well, they work great both as linking functions and as normal functions when meshed with the other math functions in unigenic systems. These functions are definitely worth exploring and worth adding to the built-in math functions of GeneXproTools, but I'm leaving that to another project as there are some thorny issues with neutral genes if Random Numerical Constants are not turned on.

Here's the C++ code for a 4-output function of 2 arguments:

    output = fabs(arg[0]+ arg[1]);
    output = floor(((output/4.0) - (int)(output/4.0))*4.0);
    return output;

As you can see these functions can be easily adapted to accommodate 2-6 discrete outputs and can be easily implemented as functions of 1-4 arguments, and if we cover all the combinations we get a total of 20 new functions. Or we could implement just the functions with 2-4 outputs, giving a total of 12 new functions. Any ideas?

Well, after all these creative failures and the small success with modulo n functions, I was super curious to see how the argmin and argmax class of functions of 3 and 4 arguments performed on the Iris dataset. But I'll tell that story in the next post.