This post describes four notable technical advantages of Value Ratings.
An Intuitive and Informative Rating Scale
A Value Rating is a number that ranges from 0% to 100%. So, an option with a Value Rating near 100% performs very well in the areas that matter most to you. An obvious use of Value Ratings is simply to rank your options. However, the Value Rating score itself offers much more. For instance, it reveals whether there is a clear leader, a close or distant contender, whether the overall field is competitive, if there is an absence of a strong entrant, and so on. And for especially critical decisions, Value Ratings offer an assessment of Evidence Strength in addition to sequential rankings.
The Preference Survey is a series of qualitative questions that quantify your preferences for different kinds of performance. You might ask: Can I simply sit down and, after some thought, arrive at similar quantities? Beware, because this is a risky and unreliable approach. Relying on sheer reflection exposes you to mental pitfalls. One example is called the Availability Heuristic, in which easy-to-recall situations become over-represented. We call this pitfall Introspection Risk, and the Preference Survey defends against it.
Missing Data are Less Problematic
When comparing complex options, it seems reasonable to “collect all the data” before any evaluation. However, data are often not available for each category of performance and for every option. This tends to stall progress. The typical thinking is that complete data is required. Value Ratings lessen this problem. When missing data is associated with areas of little importance to you, some choices may still garner high Value Ratings – if they are performing well in the areas you do care about.
Consistency Checks for Your Responses
As noted above, the Preference Survey converts your subjective responses into statistics representing how much you favor one type of performance over the others. Naturally, it is important to be consistent in responding to the survey. As a precaution, rules automatically check the consistency of your responses. If your responses imply changing preferences over the course of the survey, you will receive a notice to review them before proceeding.