Products > CART > Technical Overview > Frequently Asked Questions > CART and Large Datasets

CART and Large Datasets


CART is capable of determining the number of records in your data sets, and uses this information to predict the memory and workspace requirements for trees that you build. Also, CART will read your entire data set each time a tree is built. At times these actions may be problematic, especially if you have enormous data sets.

If you only wish to use the first N records, perhaps due to memory limitations or because you wish for faster turnaround during early exploratory analysis, you can direct CART to treat your data sets as if they have fewer records than actually exist in the data. (Another option is to contact Salford Systems regarding a memory compile upgrade so that CART can accommodate all your data; CART can be compiled to utilize up to 32 gigabytes of RAM. For further info on problems sizes and scalability, see CART Technical Overview - Scalability).

There are two options on the LIMIT command to consider:

LIMIT DATASET = N, ERRORSET = N

These options tell CART to act as if your main data set (and error/test data set if you have one) has fewer observations than it actually does. For instance, if your data set has 500,000 observations but you wish to only use the first 25,000, issue the command:

LIMIT DATASET = 25000

Similarly, if you have an enormous separate test set and wish to only use 75,000 records of it, issue the command:

LIMIT ERRORSET = 75000

CART will now treat these data sets as if they were only 25000 and 75000 records in length. Any other records will be totally ignored.


Steinberg, Dan and Phillip Colla. CART--Classification and Regression Trees. San Diego, CA: Salford Systems, 1997.