Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research

Rating:
Summary: The days of significance testing are numbered...
Review: First off, this book is written in plain English for those of us with less than stellar mathematical backgrounds. Also, the level at which it is written suggests it would make a great masters level stats course supplimental reading choice. The author walks us through the history of statistical testing and shows us where social scientists have gone wrong. Tests of statistical significance are often considered the most important indication of publication worthiness, or of experimental success whereas they really are not. For instance, what good is finding a significant difference between two groups (or in regression, finding an independant varaible significantly predicting a dependant varaible) if the effect size is so small as to be trivial? Kline discusses this in terms of "statistical significance vs. practical significance". This is one example of the very logical arguments he presents regarding aspects of behavioral statistics that most of us probably take for granted.
He argues (convincingly) that there will come a day when tests of statistical significance are no longer the be all end all of our work. Effect sizes, confidence intervals etc... will be the kings. For those of us just getting into the social scientific world, these new techniques will be vaguely familiar, but generally unexplored (remember when your professor talked about confidence intervals for 15 minutes that day you learned about t-tests?). For older behavioral scientists, (at least those that I know) the suggestions made will appear to be heresy (don't test for significance? Preposterous!). Both groups can benefit from reading the compelling arguments that Kline makes (and he is not alone) regarding the discontinuation of statistical testing, or rather, the limited use of it. This new wave in our experimental lives will take us from the barren island of null hypothesis testing to set us down in an ancient, less explored wonderland of confidence intervals, effect sizes, and more. This will be our new statistical paradise. (Yes, it was corny, but I write technical papers all day so I need the outlet).
In closing, a great read, if only to think about what will happen and where we have gone wrong in the past. I have a love of the history of statistics and its evolution, and found this book to link that aspect of my interests with the practical information about how the field is changing and how to begin approaching an acceptance of those changes. The book is reccomended for those with at least an elementary stats background. I would suggest you have taken one masters level stats course (know about p values, R-squared, basic regression, ANOVA, the null hypothesis etc...). While someone with an undergrad behavioral stats course might be familar with these terms, I find it less likely that they have puzzled about such wondrously dull topics as "what does the t-test mean" or "how do between and within subjects variance relate to one another". Such prior puzzling would help the reader fully grasp concepts Dr. Kline presents the first time through.