Why Do We Use Subjective Ratings Scales?

By relative rating scales, we mean that each successive rating value has the same relative relationship with the rating value above it. For example, consider the following sample rating scale:

  • Exceptional
  • Very Good
  • Acceptable
  • Poor

In the example above, Very Good is half as good as Exceptional; Acceptable is half as good as Very Good, and so on. This is simply the start-point for the dialogue regarding the development of rating scales. We start here because it applies a consistent approach across the entire scale and gives us the best opportunity to make meaningful distinctions among the alternatives with respect to each criterion. An alternative is well ‘compensated’ for getting an Exceptional score, and alternatives that really should not be scored very highly are not over-valued.

In addition, as humans we all make comparison judgments to create scales of relative values of the qualities we experience. Because there is no single scale built into the brain to measure things, we need to derive scales by making comparisons in relative terms. In Decision Lens, we see how this is applied when we go through pair wise. But it is also applicable when we think about the rating scales. This approach is offered in contrast with an interval scale. Consider the following scale:

  • 1.0
  • 0.75
  • 0.5
  • 0.25
  • 0

In the example above, the interval of change with the progression of rating scales from 1 to 0 is 0.25. This is a linear curve, and provides less ability to make a meaningful distinction among your alternatives (as compared to a relative scale). If it becomes evident that, for example, ‘Very Good’ is better than half as good as ‘Exceptional,’ we can modify the rating values accordingly. For example, in the example above, ‘Very Good’ now has a much closer value to ‘Exceptional,’ and the most significant drop-offs in value now occur between ‘Very Good’ and ‘Acceptable,’ and even more so between ‘Acceptable’ and ‘Minimal.’