Customer Reviews

How to Measure Anything: Finding the Value of "Intangibles" in Business

5 star:		(48)
4 star:		(8)
3 star:		(1)
2 star:		(0)
1 star:		(0)

 

 

Hubbard explains how to "find the value of intangibles in business." An excellent book and one which should be on every manager's book shelf.

Hubbard has made what can be a deadly dull subject interesting and accessible. I found several examples for measuring exactly what I needed and always felt I could not measure. This book is a must read for leaders including the Master Six Sigma Blackbelt on your staff. Finding the value of intangibles in business has always been a challenge. How to Measure Anything is full of practical ideas for getting to a measurement.

Measurement: reducing the uncertainty. As long as we are not willing to accept a best guess, or educated estimate, or range of possibilities for a difficult to measure item we will not move forward. Our decisions will be flawed. Hubbard put forth these four assumptions which I found to be most useful when thinking about measuring:

1. Your problem is not as unique as you think
2. You have more data than you think
3. You need less data than you think
4. There is a useful measurement that is much simpler than you think.

Numbers can be used to confuse people; especially the gullible ones lacking basic skills with numbers. Therefore we, as leaders, must be committed to making sure the whole organization is data driven and understands the way we can reduce uncertainty through the straight forward techniques Hubbard explains. As he states, "The fact is that the preference for ignorance over even marginal reductions in ignorance is never the moral high ground."

Hubbard gives us a very useful check list for a Universal Approach to Measurement:
1. What are you trying to measure? What is the real meaning of the alleged "intangible?"
2. Why do you care -- what's the decision and where is the "threshold?"
3. How much do you know now -- what ranges or probabilities represent your uncertainty about this?
4. What is the value of the information? What are the consequences of being wrong and the chance of being wrong, and what, if any, measurement effort would be justified?
5. Within a cost justified by the information value, which observations would confirm or eliminate different possibilities? For each possible scenario, what is the simplest thing we should see if that scenario were true?
6. How do you conduct the measurement that accounts for various types of avoidable errors (again, where the cost is less than the value of the information)?

I especially enjoy the approach Hubbard takes to quantify the cost of making measurement based on the value of the information obtained. Too often, I have seen projects founder on either inaction to get data which would be of great value and little cost or, perhaps, the exact opposite -- spending great amounts of time and money to obtain relatively useless information.

To emphasize: After reading Hubbard's excellent book on `How to Measure Anything,' I was able to immediately solve several measurement challenges for my CEO and Business Owner colleagues. This book makes accessible measurement techniques that have eluded many of my colleagues. It should be on every manager's desk. - Dave Kinnear, CEO dbkAssociates, Inc. and Vistage Chair.

Douglas Hubbard covers a broad landscape but does exactly what the title claims; it provides a guide to measuring anything. Hubbard builds from simple concepts to show the practical yet intuitively simple application of some rather advanced statistical techniques. The author's skill is in communicating complex ideas in an easy to follow and motivational flow that builds in a series of seemingly obvious steps.

The book is both philosophical and practical. If one read no more than the first three chapters one's view of the world would be changed forever. Yet the later chapters cover many extremely simple illustrations of some complex statistical concepts. I particularly enjoyed his discussion of the value of information (chapter 7), Bayesian Statistics (chapter 10), and some advanced concepts such as measuring value via observable trade-offs and using prediction markets. No one reading just a portion of this book would walk away without a new insight.

This book would be extremely useful to students in an MBA program or to those pursuing an advanced degree in one of the social sciences. It would provide a valued motivational reference to anyone studying Computer Science, Economics, or Applied Statistics. Anyone teaching or mentoring students in these disciplines might want to review this book for inclusion in their curriculum.

The book also has considerable potential at helping someone working the area of Data Warehousing and Business Intelligence. For example, Steve and Nancy Williams have written a great book titled "The Profit Impact of Business Intelligence". In it they explain the case for managing BI projects as a portfolio of risky investments. They talk of the need to measure the business value of a BI project and to coordinate changes in information flow, workflow, and decision structure so as to maximize that value. Hubbard's book offers ways that business value, cultural change, and process impact can be measured, and therefore managed. The Williams' book talks of the need for Decision Engineering. Hubbard's book gives one the understanding of what a Decision Engineering group would do on a routine basis. The two books together would be of high benefit to any manager trying to develop a "value management" culture.
The Profit Impact of Business Intelligence

The field of Human Capital Management and Workforce Analytics is receiving a lot of attention today. Two books that are particularly helpful in understanding how analytics can help Human Resource managers better support (and in fact drive) organizational performance are "The HR Scorecard" (by Brian E. Becker, Mark A. Huselid, and Dave Ulrich) and "The Workforce Scorecard" (by Mark A. Huselid, Brian E. Becker, and Richard W. Beatty). These books provide a comprehensive list of elements that could be included in a workforce analytics program from recruiting and retention to compensation and talent development. Hubbard's book helps the HR executive identify the critical metrics, what metrics add value and what metrics do not in their environment, and how to prioritize one measurement need over another. Again, Hubbard's book in combination with others will be a great compliment for the leader interested in building a fact-based value oriented decision culture.
The Workforce Scorecard: Managing Human Capital To Execute Strategy
The HR Scorecard: Linking People, Strategy, and Performance

Perhaps the most frequent question from decision analysis team members is, "How do we get the inputs?" In most evaluations, there are several key variables about which we know little. Consider oil price, for example. We have abundant historical data, yet forecasting future prices is a daunting challenge.

Doug Hubbard has written an entire book about capturing quantitative judgments. His approach differs from the usual decision analysis process. In a conventional analysis, we assume that that a subject matter expert (SME) can be identified for each key variable. Then, a skilled interviewer carefully elicits the SME's judgment through an interview process.

Hubbard takes a different approach. People familiar with the type project are assembled and given calibration training. Becoming calibrated might take perhaps a half-day of practice exercises and feedback. Basically, being "calibrated" means that one can consistently provide judgments of 90% confidence intervals that avoid the "overconfidence" bias. The book provides several example quizzes for the reader to self-assess.
Even though I was well-aware of the overconfidence bias, I still performed poorly on the self-assessment tests (history was never my strong subject!). Of course, the questions for a technical group would be crafted from topics within the area of interest. Whether (a) expert in the quiz subject matter or not and (b) being told in advance that people tend to be overconfident about the quality of their knowledge doesn't seem to affect the overconfident bias. Practice and feedback are the antidotes.

Hubbard's training and consulting examples are engaging. It has been years since I've devoured a technical book so thoroughly. While the reader will pick-and-choose methods of most interest, the "measurement" topic is well-covered.

The book contains many shortcuts and heuristics for rapid problem-solving. Many people never attempt to quantify intangibles. Yet, most people with some modest training are able to provide credible judgments in quantitative form.

A sampling of topics includes:
* Modeling and Monte Carlo simulation
* Designing experiments for measurement
* Decomposition
* Heuristics for obtaining simple statistics
* Value of perfect information, for screening which variables are worthwhile measuring
* Bayes' rule (because we almost always have some prior information about the subject of the observation)
* Cognitive biases

How to Measure Anything is well-written and carefully edited. The companion Web site, [..], offers additional calibration questions, several calculation spreadsheets, and additional information.

Persons reading this book will be the better for it.

Excellent, practical treatment of a critical issue in managing technology investments--namely, how can we reliably assess IT project ROI? This question lies at the heart of virtually every strategic IT decision. Yet, answering it is often viewed as being either too difficult to pull off or simply not worth the effort. As a result, organizations are left with project portfolios that are underperforming and wasting resources that could otherwise be applied to lower risk and higher return efforts.
Doug's straightforward treatment of this important topic directly addresses what our clients view as their biggest obstacles to achieving better returns on IT investments:
* Intangibles, esp. potential benefit streams
* Lack of proper instrumentation and measurement infrastructure
* Difficulty in applying the proper financial and statistical concepts
The examples, spreadsheet templates, and download links provide almost any IT organization with the basic tools and know-how to quickly assess what to measure, how to measure, and then how to turn that information into a powerful decision-making tool for your IT organization. The material in this book offers a compelling blueprint for IT governance. This book is accessible to everyone.

This book addresses the measurement problem that managers face when it comes to placing a value on the important but vague factors that affect the decision-making process. I like the way he clearly defines the steps that the manager should take. I also like the conversational style of the book.
In general, he shows that measuring associated costs and other phenomena that affect the decision-making process is not that big a deal once you are able to define the aspects that you feel affect your processes and also determine the context in which these aspects are important to you, as a manager.
However,as an accountant, I find that it does not address the issue of intangibles from my perspective. How does the accountant measure physical intangibles that are not accounted for on the books, but that clearly affect the market value of the company? Intangibles like R&D, intellectual property and internally developed software should be accounted for but are not for lack of a dependable method of measurement. The book doesnt provide an answer to this question, so if you are an accountant looking to measure an intangible for purposes of reporting them, this book will not provide an answer, but if you are a manager or just someone who likes placing a value on the happenings that you find interesting, go for it. You will not be disappointed.

As someone moderately well versed in math, statistics, economics, and business, I was not convinced that I was going to find something useful in reading the book, but decided to pick it up anyway. Not only was I wrong...I picked up a great deal when I read the book, but on a second reading, I picked up even more.

On first reading, I heard loud and clear:

1. Measurement doesn't quite mean what you think it means, and what it does mean is really important.
2. Measures (and estimates) are always ranges + probabilities. They are not just numbers.
3. If you ask someone to give you a 90% range, they will be wrong (on the too narrow side). You have to practice to get these ranges right.
4. People mostly measure stuff they don't need to measure at high cost, and don't measure the important stuff at all.
5. For anything you would like to measure, you can...it's not too hard.
6. Statistics + excel can give you the rest of the information you need, both about what to measure and what the results mean.

On a second read (and a conversation with Doug), I got one more piece of information that seems really simple, but isn't.

The things that you want to measure are the things that have the highest information value. High information value measurements are ones that are cheap compared to the decisiveness of the information. Effectively, how little money do you have to spend on measurement to make a good decision. This is likely even more important than the stuff I learned on the first read.

All in all, if you make decisions, and you are worried about making correct decisions, rather than just decisions that you like, or that feel right, then you ought to be reading this book. If you're interested in protecting your old way of making decisions, on the other hand, don't read it. This will tell you where you're making your mistakes.

"I wrote this book to correct a myth that permeates many organizations today: that certain things can't be measured." Douglas Hubbard goes on to note that he has made a career out of measuring the sorts of things many thought were immeasurable. Intangibles, for example, "that appear to be completely intractable to be measured...in a way that is economically justified." Hubbard notes that there are several common misconceptions about intangibles. He offers what he characterizes as a "universal approach," Applied Information Economics (AIE), to measure an intangible, providing with that explanation some "interesting methods for particular problems."

He duly recognizes that only what is most important (tangible or intangible) should be measured; also, that what is currently most important may not retain that importance; and, that information needs change, sometimes significantly and unexpectedly. That said, basic questions must constantly be asked and answered:

1. What are our most important information needs? Why?

2. How best to obtain and then verify that information?

3. What will we then do with that information?

4. How can we then measure (accurately, consistently, and sufficiently) the impact of actions taken based on that information?

To his credit, Hubbard makes every effort to provide information, explanations, and recommendations that are (in his words) as "simple as can be"; nonetheless, some of the material may prove daunting, at least it did to me. I appreciate the inclusion of dozens of real-world examples that illustrate key points. Hubbard also makes effective use of other reader-friendly devices, such as checklists inserted throughout his narrative. In his own words, here is how he organizes his material:

In Section One (Chapters 1-3), he "makes the case that everything is measurable and offers some examples that should inspire readers to attempt measurements even when it seems impossible."

In Section Two (Chapters 4-7), he "begins to get into more specific substance about how to measure things - specifically uncertainty, risk - and the value of information."

In Section Three (Chapters 8-10), he "deals with how to reduce uncertainty by various methods of observation including random sampling and controlled experiments."

And then in Section Four (Chapters 11-14), Hubbard offers "an eclectic collection of interesting measurement solutions and case examples."

Many readers will appreciate having the Appendix (Pages 269-278) which provides both the questions and answers for various calibration tests, including "Calibration Survey for Binary: B" that also includes percentages to indicate degree of confidence that the respondent is correct.

Earlier, I suggested that this is by no means an "easy read." It isn't. Nor will this book respond directly to every executive's immediate needs and objectives. However, it will generously reward those who need assistance with finding and measuring the intangibles in business if they absorb and digest the material with appropriate care. To those about to begin reading this book, Douglas Hubbard offers this recommendation: Write down those things they believe are immeasurable or, at least they are not sure to how to measure. "After reading this book, my goal is that you are able to identify methods for measuring each and every one of them." I presume to add another recommendation: Highlights key passages and titles of checklists. By doing so, you will be able to facilitate, indeed expedite frequent review of key concepts and insights later.

I use this book as a primary reference for my measurement class at MIT. The students love it because it provides practical advice that can be applied to a variety of scenarios; from aerospace & defense, healthcare, politics, etc.

Last week I noticed that the person sitting next to me at the airport was reading the book and we had a great conversation about it. She is the CEO of an environmental engineering firm and is applying some of the principles to organizational performance measurement.

In the often over-saturated field of metrics literature, plain and simple this book makes sense of measurement. Starting with a common grounding definition of measurement for the decision maker "a set of observations that reduce uncertainty where the result is expressed as a quantity" - Hubbard unveils many simple truths about measurement that are helpful in every day practice for practitioners at all levels.

Some of the simple nuggets of wisdom to extract from this book entail the:

CLARIFICATION CHAIN: (i) if it matters at all, it is detectable and observable; (ii) if it is detectable, it can be detected as an amount (or range of possible amounts); and (iii) if it can be detected as a range of possible amounts, it can be measured

FOUR USEFUL MEASUREMENT ASSUMPTIONS: (i) your problem is not as unique as you think; (ii) you have more data than you think; (iii) you need less data than you think; (iv) there is a useful measurement that is much simpler than you think

QUESTIONS TO ASK BEFORE MEASUREMENT: (i) what is the decision this is supposed to support; (ii) what really is the thing being measured; (iii) what does this thing matter to the decision being asked; (iv) what do you know about it now; and (v) what is the value to measuring it further?

And many other heuristics that one can consult on an everyday basis or use as a cornerstone for a holistic measurement analysis.

Last, for any position relying expert opinion - Hubbard's depiction of how optimism bias can be calculated is particularly insightful and in fact a must read.

In general, this book is a great read for not just measurement people, but for management people and the everyday practitioner. Enjoy!

I first read Hubbard's book in pre-publication form when asked by the author to write a recommendation for the book. Two weeks ago I had occasion to read it again as I was planning an introductory lecture on measurement for health care measurement experts. I routinely recommend the book along with Denny Borsboom's Measuring the Mind. Upon my rereading I felt compelled to write this brief note.
This book "works" on several levels. First, it is very well written and comparably well edited. Second, it covers a wide range of material without dabbling. Every key point is illustrated with a memorable application or example. And, finally, the book treats complex material like the Rasch model and Baysean inference elegantly.
My favorite parts of the book are Chapter 3, The Illusion of Intangibles: Why Immeasurables Aren't, which could have been an alternate title for the book. Chapter 14, New Measurement Instruments for Management, especially the section on the Internet as an unobtrusive measurement device and Chapter 10, Bayes: Adding to what you Know Now. Of course, we all are Bayseans!
Graduate students in almost any field that deals with numbers, and business executives across the spectrum will benefit from reading this book. Like USA Today the book measures at a comfortable 1240 Lexiles.

A. Jackson Stenner
CEO, MetaMetrics, Inc
Durham, NC