Choose a model by AIC in a Stepwise Algorithm

Performs stepwise model selection by AIC.

stepAIC(object, scope, scale = 0,
        direction = c("both", "backward", "forward"),
        trace = 1, keep = NULL, steps = 1000, use.start = FALSE,
        k = 2, ...)

Arguments

object	an object representing a model of an appropriate class. This is used as the initial model in the stepwise search.
scope	defines the range of models examined in the stepwise search. This should be either a single formula, or a list containing components upper and lower, both formulae. See the details for how to specify the formulae and how they are used.
scale	used in the definition of the AIC statistic for selecting the models, currently only for lm and aov models (see extractAIC for details).
direction	the mode of stepwise search, can be one of "both", "backward", or "forward", with a default of "both". If the scope argument is missing the default for direction is "backward".
trace	if positive, information is printed during the running of stepAIC. Larger values may give more information on the fitting process.
keep	a filter function whose input is a fitted model object and the associated AIC statistic, and whose output is arbitrary. Typically keep will select a subset of the components of the object and return them. The default is not to keep anything.
steps	the maximum number of steps to be considered. The default is 1000 (essentially as many as required). It is typically used to stop the process early.
use.start	if true the updated fits are done starting at the linear predictor for the currently selected model. This may speed up the iterative calculations for glm (and other fits), but it can also slow them down. Not used in R.
k	the multiple of the number of degrees of freedom used for the penalty. Only k = 2 gives the genuine AIC: k = log(n) is sometimes referred to as BIC or SBC.
...	any additional arguments to extractAIC. (None are currently used.)

Value

the stepwise-selected model is returned, with up to two additional components. There is an "anova" component corresponding to the steps taken in the search, as well as a "keep" component if the keep= argument was supplied in the call. The "Resid. Dev" column of the analysis of deviance table refers to a constant minus twice the maximized log likelihood: it will be a deviance only in cases where a saturated model is well-defined (thus excluding lm, aov and survreg fits, for example).

Details

The set of models searched is determined by the scope argument. The right-hand-side of its lower component is always included in the model, and right-hand-side of the model is included in the upper component. If scope is a single formula, it specifies the upper component, and the lower model is empty. If scope is missing, the initial model is used as the upper model.

Models specified by scope can be templates to update object as used by update.formula.

There is a potential problem in using glm fits with a variable scale, as in that case the deviance is not simply related to the maximized log-likelihood. The glm method for extractAIC makes the appropriate adjustment for a gaussian family, but may need to be amended for other cases. (The binomial and poisson families have fixed scale by default and do not correspond to a particular maximum-likelihood problem for variable scale.)

Where a conventional deviance exists (e.g. for lm, aov and glm fits) this is quoted in the analysis of variance table: it is the unscaled deviance.

Note

The model fitting must apply the models to the same dataset. This may be a problem if there are missing values and an na.action other than na.fail is used (as is the default in R). We suggest you remove the missing values first.

References

Venables, W. N. and Ripley, B. D. (2002) Modern Applied Statistics with S. Fourth edition. Springer.

See also

addterm, dropterm, step

Examples

quine.hi <- aov(log(Days + 2.5) ~ .^4, quine)
quine.nxt <- update(quine.hi, . ~ . - Eth:Sex:Age:Lrn)
quine.stp <- stepAIC(quine.nxt,
    scope = list(upper = ~Eth*Sex*Age*Lrn, lower = ~1),
    trace = FALSE)
quine.stp$anova
#> Stepwise Model Path 
#> Analysis of Deviance Table
#> 
#> Initial Model:
#> log(Days + 2.5) ~ Eth + Sex + Age + Lrn + Eth:Sex + Eth:Age + 
#>     Eth:Lrn + Sex:Age + Sex:Lrn + Age:Lrn + Eth:Sex:Age + Eth:Sex:Lrn + 
#>     Eth:Age:Lrn + Sex:Age:Lrn
#> 
#> Final Model:
#> log(Days + 2.5) ~ Eth + Sex + Age + Lrn + Eth:Sex + Eth:Age + 
#>     Eth:Lrn + Sex:Age + Sex:Lrn + Age:Lrn + Eth:Sex:Lrn + Eth:Age:Lrn
#> 
#> 
#>            Step Df Deviance Resid. Df Resid. Dev       AIC
#> 1                                 120   64.09900 -68.18396
#> 2 - Eth:Sex:Age  3 0.973869       123   65.07287 -71.98244
#> 3 - Sex:Age:Lrn  2 1.526754       125   66.59962 -72.59652

cpus1 <- cpus
for(v in names(cpus)[2:7])
  cpus1[[v]] <- cut(cpus[[v]], unique(quantile(cpus[[v]])),
                    include.lowest = TRUE)
cpus0 <- cpus1[, 2:8]  # excludes names, authors' predictions
cpus.samp <- sample(1:209, 100)
cpus.lm <- lm(log10(perf) ~ ., data = cpus1[cpus.samp,2:8])
cpus.lm2 <- stepAIC(cpus.lm, trace = FALSE)
cpus.lm2$anova
#> Stepwise Model Path 
#> Analysis of Deviance Table
#> 
#> Initial Model:
#> log10(perf) ~ syct + mmin + mmax + cach + chmin + chmax
#> 
#> Final Model:
#> log10(perf) ~ mmin + mmax + cach + chmin + chmax
#> 
#> 
#>     Step Df    Deviance Resid. Df Resid. Dev       AIC
#> 1                              82   3.551742 -297.7732
#> 2 - syct  3 0.005265993        85   3.557008 -303.6251

example(birthwt)
#> 
#> brthwt> bwt <- with(birthwt, {
#> brthwt+ race <- factor(race, labels = c("white", "black", "other"))
#> brthwt+ ptd <- factor(ptl > 0)
#> brthwt+ ftv <- factor(ftv)
#> brthwt+ levels(ftv)[-(1:2)] <- "2+"
#> brthwt+ data.frame(low = factor(low), age, lwt, race, smoke = (smoke > 0),
#> brthwt+            ptd, ht = (ht > 0), ui = (ui > 0), ftv)
#> brthwt+ })
#> 
#> brthwt> options(contrasts = c("contr.treatment", "contr.poly"))
#> 
#> brthwt> glm(low ~ ., binomial, bwt)
#> 
#> Call:  glm(formula = low ~ ., family = binomial, data = bwt)
#> 
#> Coefficients:
#> (Intercept)          age          lwt    raceblack    raceother    smokeTRUE  
#>     0.82302     -0.03723     -0.01565      1.19241      0.74068      0.75553  
#>     ptdTRUE       htTRUE       uiTRUE         ftv1        ftv2+  
#>     1.34376      1.91317      0.68020     -0.43638      0.17901  
#> 
#> Degrees of Freedom: 188 Total (i.e. Null);  178 Residual
#> Null Deviance:	    234.7 
#> Residual Deviance: 195.5 	AIC: 217.5
birthwt.glm <- glm(low ~ ., family = binomial, data = bwt)
birthwt.step <- stepAIC(birthwt.glm, trace = FALSE)
birthwt.step$anova
#> Stepwise Model Path 
#> Analysis of Deviance Table
#> 
#> Initial Model:
#> low ~ age + lwt + race + smoke + ptd + ht + ui + ftv
#> 
#> Final Model:
#> low ~ lwt + race + smoke + ptd + ht + ui
#> 
#> 
#>    Step Df Deviance Resid. Df Resid. Dev      AIC
#> 1                         178   195.4755 217.4755
#> 2 - ftv  2 1.358185       180   196.8337 214.8337
#> 3 - age  1 1.017866       181   197.8516 213.8516
birthwt.step2 <- stepAIC(birthwt.glm, ~ .^2 + I(scale(age)^2)
    + I(scale(lwt)^2), trace = FALSE)
birthwt.step2$anova
#> Stepwise Model Path 
#> Analysis of Deviance Table
#> 
#> Initial Model:
#> low ~ age + lwt + race + smoke + ptd + ht + ui + ftv
#> 
#> Final Model:
#> low ~ age + lwt + smoke + ptd + ht + ui + ftv + age:ftv + smoke:ui
#> 
#> 
#>         Step Df  Deviance Resid. Df Resid. Dev      AIC
#> 1                               178   195.4755 217.4755
#> 2  + age:ftv  2 12.474896       176   183.0006 209.0006
#> 3 + smoke:ui  1  3.056805       175   179.9438 207.9438
#> 4     - race  2  3.129586       177   183.0734 207.0734

quine.nb <- glm.nb(Days ~ .^4, data = quine)
quine.nb2 <- stepAIC(quine.nb)
#> Start:  AIC=1095.32
#> Days ~ (Eth + Sex + Age + Lrn)^4
#> 
#>                   Df    AIC
#> - Eth:Sex:Age:Lrn  2 1092.7
#> <none>               1095.3
#> 
#> Step:  AIC=1092.73
#> Days ~ Eth + Sex + Age + Lrn + Eth:Sex + Eth:Age + Eth:Lrn + 
#>     Sex:Age + Sex:Lrn + Age:Lrn + Eth:Sex:Age + Eth:Sex:Lrn + 
#>     Eth:Age:Lrn + Sex:Age:Lrn
#> 
#>               Df    AIC
#> - Eth:Sex:Age  3 1089.4
#> <none>           1092.7
#> - Eth:Sex:Lrn  1 1093.3
#> - Eth:Age:Lrn  2 1094.7
#> - Sex:Age:Lrn  2 1095.0
#> 
#> Step:  AIC=1089.41
#> Days ~ Eth + Sex + Age + Lrn + Eth:Sex + Eth:Age + Eth:Lrn + 
#>     Sex:Age + Sex:Lrn + Age:Lrn + Eth:Sex:Lrn + Eth:Age:Lrn + 
#>     Sex:Age:Lrn
#> 
#>               Df    AIC
#> <none>           1089.4
#> - Sex:Age:Lrn  2 1091.1
#> - Eth:Age:Lrn  2 1091.2
#> - Eth:Sex:Lrn  1 1092.5
quine.nb2$anova
#> Stepwise Model Path 
#> Analysis of Deviance Table
#> 
#> Initial Model:
#> Days ~ (Eth + Sex + Age + Lrn)^4
#> 
#> Final Model:
#> Days ~ Eth + Sex + Age + Lrn + Eth:Sex + Eth:Age + Eth:Lrn + 
#>     Sex:Age + Sex:Lrn + Age:Lrn + Eth:Sex:Lrn + Eth:Age:Lrn + 
#>     Sex:Age:Lrn
#> 
#> 
#>                Step Df   Deviance Resid. Df Resid. Dev      AIC
#> 1                                       118   167.4535 1095.324
#> 2 - Eth:Sex:Age:Lrn  2 0.09746244       120   167.5509 1092.728
#> 3     - Eth:Sex:Age  3 0.11060087       123   167.4403 1089.409