Model Selection and Multi-Model Inference

Rating:
Summary: authoritative and thorough treatment
Review: Burnham and Anderson have put together a scholarly account of the developments in model selection techniques from the information theoretic viewpoint. This is an important practical subject. As computer algorithms become more and more available for fitting models and data mining and exploratory analysis become more popular and used more by novices, problems with overfitting models will again raise their ugly heads. This has been an issue for statisticians for decades. But the problems and the art of model selection has not been commonly covered in elementary courses on statistics and regression. George Box puts proper emphasis on the iterative nature of model selection and the importance of applying the principle of parismony in many of his books. Classic texts on regression like Draper and Smith point out the pitfalls of goodness of ift measures like R-square and explain Mallows Cp and adjusted R-square. There are now also a few good books devoted to model selection including the book by McQuarrie and Tsai (that I recently reviewed for Amazon) and the Chapman and Hall monograph by A. J. Miller.

Burnham and Anderson address all these issues and provide the best coverage to date on bootstrap and cross-validation approaches. They also are careful in their historical account and in putting together some coherence to the scattered literature. They are thorough in their references to the literature. Their theme is the information theoretic measures based on the Kullback-Liebler distance measure. The breakthrough in this theory came from Akaike in the 1970s and improvements and refinement came later. The authors provide the theory, but more importantly, they provide many real examples to illustrate the problems and show how the methods work.

They also refer to the recent work in Bayesian methods. Chapter 1 is a great introduction that everyone should read. Being a fan of the bootstrap I was interested in their coverage of it in chapters 4, 5 and 6 (much of which is the authors' own work).

Because the authors work in biological fields they cover survival models as well as the standard time series and regression models where most of the emphasis has been placed on model selection in the past.

It is a great reference source and an important book for learning about model selection as part of the inferential process. The pictures of the famous contributors inserted throughout the book is also nice to see. We have Akaike, Boltzmann, Shibata, Kullback, and Liebler brought to life in photographs or sketches.

Rating:
Summary: A breakthrough book on statistical modeling building
Review: Statistical data analysis usually goes through cycles of exploring and looking for patterns in data, often through model construction, analyzing residuals and modifying model fits, until all unusual features being explained. Though this practice has been going on for more than 100 years, it has not been closely examined to see whether the fact that your analysis based on the best fitted model using the same data set should be biased, or plainly you cheated by over-analyzing your data. This book by the two productive authors say yes, and you should rethink about what you have been doing. A highly applaudable and timely efforts on the part of the authors, considering that the trend of over-analyzing your data is increasing rapidly with recent explosion of data and intensive computer analysis in the data mining industry. It's not as hopeless or bad as you think, and there are ways to avoid pitfalls and there may exist ways of making some valid inference out of this model selection process. So enjoy reading this book and think!