The APS Environment
Testing the Generalizing Capabilities of a Model
 
In order to test the generalizing capabilities of your model, you must provide an additional set of sample data (testing set) which is only used after the learning process is complete.

To Test the Generalizing Capabilities of a Model with APS 3.0 
  1. On the Edit Menu, select Add Testing Data.
    The Change/Add Testing Data wizard appears. It asks for the type of data source and the path for your testing file. Click on Finish to load the testing set.
  2. In the Run Panel, after a run, check the best-of-run values of Fitness and R-square obtained for the testing set.
    After a run, APS evaluates the performance of the evolved model on the testing set and shows both the fitness and the R-square obtained on the testing set.
  3. In the Results Panel, select the Testing option.
    Depending on the problem category, you can analyze a varied set of statistical functions of your model on the testing set. These statistics include: mean squared error, root mean squared error, mean absolute error, relative squared error, root relative squared error, relative absolute error, correlation coefficient, and R-square for all types of problem category; and the classification error, classification accuracy, confusion matrix (true positives, true negatives, false positives, and false negatives), sensitivity, specificity, positive predictive value, and negative predictive value for Classification problems.
  4. In the Report Panel, check the entry Statistics - Testing.
    Depending on the problem category, a wider set of statistical functions of your model on the testing set can be found. These statistics include: mean squared error, root mean squared error, mean absolute error, relative squared error, root relative squared error, relative absolute error, correlation coefficient, and R-square for all types of problem category; and the classification error, classification accuracy, confusion matrix (true positives, true negatives, false positives, and false negatives), sensitivity, specificity, positive predictive value, and negative predictive value for Classification problems.



Home | Contents | Previous | Next