Principal Components Analysis of Resampling Results

Performs a principal components analysis on an object of class resamples and returns the results as an object with classes prcomp.resamples and prcomp.

# S3 method for resamples
prcomp(x, metric = x$metrics[1], ...)

# S3 method for prcomp.resamples
plot(x, what = "scree", dims = max(2,
  ncol(x$rotation)), ...)

Arguments

x	For prcomp, an object of class resamples and for plot.prcomp.resamples, an object of class plot.prcomp.resamples
metric	a performance metric that was estimated for every resample
...	For prcomp.resamples, options to pass to prcomp, for plot.prcomp.resamples, options to pass to Lattice objects (see Details below) and, for cluster.resamples, options to pass to hclust.
what	the type of plot: "scree" produces a bar chart of standard deviations, "cumulative" produces a bar chart of the cumulative percent of variance, "loadings" produces a scatterplot matrix of the loading values and "components" produces a scatterplot matrix of the PCA components
dims	The number of dimensions to plot when what = "loadings" or what = "components"

Value

For prcomp.resamples, an object with classes prcomp.resamples and prcomp. This object is the same as the object produced by prcomp, but with additional elements:

metric

the value for the metric argument

call

the call

For plot.prcomp.resamples, a Lattice object (see Details above)

Details

The principal components analysis treats the models as variables and the resamples are realizations of the variables. In this way, we can use PCA to "cluster" the assays and look for similarities. Most of the methods for prcomp can be used, although custom print and plot methods are used.

The plot method uses lattice graphics. When what = "scree" or what = "cumulative", barchart is used. When what = "loadings" or what = "components", either xyplot or splom are used (the latter when dims > 2). Options can be passed to these methods using ....

When what = "loadings" or what = "components", the plots are put on a common scale so that later components are less likely to be over-interpreted. See Geladi et al. (2003) for examples of why this can be important.

For clustering, hclust is used to determine clusters of models based on the resampled performance values.

References

Geladi, P.; Manley, M.; and Lestander, T. (2003), "Scatter plotting in multivariate data analysis," J. Chemometrics, 17: 503-511

See also

resamples, barchart, xyplot, splom, hclust

Examples

if (FALSE) {
#load(url("http://topepo.github.io/caret/exampleModels.RData"))

resamps <- resamples(list(CART = rpartFit,
                          CondInfTree = ctreeFit,
                          MARS = earthFit))
resampPCA <- prcomp(resamps)

resampPCA

plot(resampPCA, what = "scree")

plot(resampPCA, what = "components")

plot(resampPCA, what = "components", dims = 2, auto.key = list(columns = 3))

clustered <- cluster(resamps)
plot(clustered)

}