Can ordinal scale numbers kill you?

This May 19, 2013 article in USA Today might make you nervous. According to a study of 1400 sunscreen products by the Environmental Working Group, many continue to carry SPF ratings that some experts consider misleading and potentially dangerous. The reason, the article states is that:

"SPF numbers like 100 or 150 can give users a false sense of security, leading them to stay in the sun long after the lotion has stopped protecting their skin. Many consumers assume that SPF 100 is twice as effective as SPF 50, but dermatologists say the difference between the two is actually negligible. 

"Where an SPF 50 product might protect against 97% of sunburn-­causing rays, an SPF 100 product might block 98.5% of those rays. There is a popular misconception that the SPF figure relates to a certain number of hours spent in the sun. However this is incorrect, since the level of exposure varies by geography, time of day and skin complexion."

In other words, people believe (as did I until I read this), that SPF numbers were ratio scale and that 100 provided twice the protection of 50. In fact, as this article postulates, they are ordinal and SPF 100 is only 1.5% more protection than SPF 50.

This is the problem with ordinal scales. They confuse people into interpreting numbers the wrong way.

QFD experts have known this problem for many years, and in our early days (1966-1985), we didn't have an easy solution to obtaining ratio scale values from subjective judgments. In the House of Quality (HoQ), rating customer needs or competitive performance on a 1-5 ordinal scale, or enumerating relationship weights using 1,3,5 or 1,3,9 ordinal numbers are examples of ordinal scale subjective judgments. Like SPF, you cannot meaningfully add, subtract, multiply, or divide them.

The solution to the problem came to us QFD folks in 1986 when Dr. Saaty's Analytic Hierarchy Process (AHP) became available as a PC program, allowing us to accurately convert subjective judgments into ratio scale values. It took me a few projects to see the difference in accuracy. For example, a judgment of "4" on the ordinal scale is usually perceived to be two times a judgment of "2."  But if you convert the 1-5 to ratio using AHP, it turns out that the judgment of " 4" is 26.0% and "2" is 6.8%.  26/6.8= 3.82 meaning that the judgment of a "4" is almost four times the judgment of a "2." Imagine the impact this mistake could have on a multi-million dollar project!

So, if you are not using AHP to calculate ratio scale judgments in your QFD, switch now, before the summer sun kills you.