How does TreeNet® work and what does a TreeNet® model look like?
A TreeNet® model normally consists of from several dozen to several hundred small trees, each typically no larger than two to eight terminal nodes. The model is similar in spirit to a long series expansion (such as a Fourier or Taylor's series) - a sum of factors that becomes progressively more accurate as the expansion continues. The expansion can be written as:
where each Ti is a small tree. An example of the first few terms of a model to predict home values based on the 1970 Census Boston Housing data set is:
The model tells us that we start with the mean home value (in 1970) of $22,533 and adjust that estimate upwards by $13,541 for larger homes, and adjust it upwards again by $2,607 for neighborhoods with good socioeconomic status indicators. In practice the adjustments are usually much smaller than shown in this regression example and hundreds of adjustments may be needed. The final model is thus a collection of weighted and summed trees. For binary classification problems, a yes or no response is determined by whether the sign of the predicted outcome is positive or negative. For multi-class problems a score is developed separately for each class via class-specific expansions, and the scores are converted into a set of probabilities of class membership.
The example above uses the smallest possible two-node tree in each stage. More complicated models tracking complex interactions are possible with three or more nodes at each stage.