Combining models for predictions
Much of applied research in agriculture is concerned with selecting models for describing observed data, testing hypotheses and predicting future scenarios. Many criteria have been proposed to assist in model selection from both frequentist and Bayesian perspectives, e.g., Akaike information criterion (AIC), Schwartz criterion (also known as Bayesian information criterion), Mallows criterion, and the coefficient of determination (R2). When a hypothesis and its alternatives can be expressed explicitly as models, relative goodness of fit of these competing models to observed data can then be used as evidence for or against the hypothesis. When observed data fail overwhelmingly to support any one of the competing models, model averaging may be used to estimate model parameters based on various model averaging schemes.
Given the availability of multiple models, the question is whether and, if so, how to select a best one (or indeed to develop another one that fits new data better than published ones), or whether to combine models for prediction. Statisticians have advocated the use of model averaging to improve predictions.
IN collaboration with Professor Larry Madden of Ohio State University, we are studying whether model averaging results in predictions that are more accurate than the ‘best’ single model and, if so, under what conditions. Specifically, we are interested in two situations: (1) a modeller has access to all data sets, i.e. the modeller knows the relative performance of alternative models in explaining the same set of observed data; and (2) a modeller can only access published ‘best’ models without knowing their relative performance.