<< 1 >>
Rating: 
Summary: Highly academic with excellent theoretical insights
Review: The six chapters in this book contain two introductory papers and 11 technical papers on benchmarking in general. Each paper is written by one or more authors, making this a compendium of current research. Most of the book is slanted towards methods associated with Standard Performance Evaluation Corporation (SPEC) benchmarking. The central theme is the creation and management of benchmarks for computer performance, and some of the challenges faced by the creators and maintainers to ensure the validity and accuracy of the benchmarks. Chapter 1 is an overview of the book, discusses related work and provides brief insights on performance evaluations in the real world. Chapter 2 introduces SPEC and current issues, and moves into a second paper on performance evaluation of large-scale systems. The most theoretical chapter, 3, devotes two papers to SPEC benchmarks in the research environment. To give an indication of how esoteric this chapter is, one of the papers is titled "The Allocation Behavior of the SPECjvm98 Java Benchmarks". This is the stuff of pure research; however, the discussion yields conclusions that can be used by development companies, particularly in the quality and architecture domains. Chapter 4 stands out as the most practical for real world applicability. It covers tools and methodologies for benchmarking and performance evaluation, with one paper devoted to a framework that can be applied to parallel applications, and the second paper in the chapter addressing performance coupling through case studies for measuring interactions of kernals. Large development organizations will find much useful information here. The final two chapters are a blend of theoretical and practical, with chapter 5 addressing related benchmarking efforts and chapter 6 devoted to needs for new benchmarks.
The material is well researched; reading through it is somewhat ponderous. However, if you are involved in the creation of performance benchmarks, regardless of whether you intend to use SPEC or related models, this book will prove valuable. If you are seeking information on performance benchmarking in an operational environment this book may be too theoretical. As someone who is on the operational side, but with an interest in theoretical and academic works I found this book to be interesting, but not particularly useful. On the other hand, there is much in this book that *can* be directly applied by those who design, develop or verify and validate benchmarks. Moreover, this book represents the some of the latest research findings and thoughts (as I write this in Sep 2001) in this highly specialized field, making it invaluable to the right audience.
Rating:
Summary: Highly academic with excellent theoretical insights
Review: The six chapters in this book contain two introductory papers and 11 technical papers on benchmarking in general. Each paper is written by one or more authors, making this a compendium of current research. Most of the book is slanted towards methods associated with Standard Performance Evaluation Corporation (SPEC) benchmarking. The central theme is the creation and management of benchmarks for computer performance, and some of the challenges faced by the creators and maintainers to ensure the validity and accuracy of the benchmarks. Chapter 1 is an overview of the book, discusses related work and provides brief insights on performance evaluations in the real world. Chapter 2 introduces SPEC and current issues, and moves into a second paper on performance evaluation of large-scale systems. The most theoretical chapter, 3, devotes two papers to SPEC benchmarks in the research environment. To give an indication of how esoteric this chapter is, one of the papers is titled "The Allocation Behavior of the SPECjvm98 Java Benchmarks". This is the stuff of pure research; however, the discussion yields conclusions that can be used by development companies, particularly in the quality and architecture domains. Chapter 4 stands out as the most practical for real world applicability. It covers tools and methodologies for benchmarking and performance evaluation, with one paper devoted to a framework that can be applied to parallel applications, and the second paper in the chapter addressing performance coupling through case studies for measuring interactions of kernals. Large development organizations will find much useful information here. The final two chapters are a blend of theoretical and practical, with chapter 5 addressing related benchmarking efforts and chapter 6 devoted to needs for new benchmarks.
The material is well researched; reading through it is somewhat ponderous. However, if you are involved in the creation of performance benchmarks, regardless of whether you intend to use SPEC or related models, this book will prove valuable. If you are seeking information on performance benchmarking in an operational environment this book may be too theoretical. As someone who is on the operational side, but with an interest in theoretical and academic works I found this book to be interesting, but not particularly useful. On the other hand, there is much in this book that *can* be directly applied by those who design, develop or verify and validate benchmarks. Moreover, this book represents the some of the latest research findings and thoughts (as I write this in Sep 2001) in this highly specialized field, making it invaluable to the right audience.
<< 1 >>
| 
