22 May 2013

Can ordinal scale numbers kill you?

This May 19, 2013 article in USA Today might make you nervous. According to a study of 1400 sunscreen products by the Environmental Working Group, many continue to carry SPF ratings that some experts consider misleading and potentially dangerous. The reason, the article states is that:

(photo of sunscreens, source FDA)"SPF numbers like 100 or 150 can give users a false sense of security, leading them to stay in the sun long after the lotion has stopped protecting their skin. Many consumers assume that SPF 100 is twice as effective as SPF 50, but dermatologists say the difference between the two is actually negligible. 

"Where an SPF 50 product might protect against 97% of sunburn-­causing rays, an SPF 100 product might block 98.5% of those rays. There is a popular misconception that the SPF figure relates to a certain number of hours spent in the sun. However this is incorrect, since the level of exposure varies by geography, time of day and skin complexion."

In other words, people believe (as did I until I read this), that SPF numbers were ratio scale and that 100 provided twice the protection of 50. In fact, as this article postulates, they are ordinal and SPF 100 is only 1.5% more protection than SPF 50.

This is the problem with ordinal scales. They confuse people into interpreting numbers the wrong way.

QFD experts have known this problem for many years, and in our early days (1966-1985), we didn't have an easy solution to obtaining ratio scale values from subjective judgments. In the House of Quality (HoQ), rating customer needs or competitive performance on a 1-5 ordinal scale, or enumerating relationship weights using 1,3,5 or 1,3,9 ordinal numbers are examples of ordinal scale subjective judgments. Like SPF, you cannot meaningfully add, subtract, multiply, or divide them.

The solution to the problem came to us QFD folks in 1986 when Dr. Saaty's Analytic Hierarchy Process (AHP) became available as a PC program, allowing us to accurately convert subjective judgments into ratio scale values. It took me a few projects to see the difference in accuracy. For example, a judgment of "4" on the ordinal scale is usually perceived to be two times a judgment of "2."  But if you convert the 1-5 to ratio using AHP, it turns out that the judgment of " 4" is 26.0% and "2" is 6.8%.  26/6.8= 3.82 meaning that the judgment of a "4" is almost four times the judgment of a "2." Imagine the impact this mistake could have on a multi-million dollar project!

So, if you are not using AHP to calculate ratio scale judgments in your QFD, switch now, before the summer sun kills you.


AHP books by Thomas Saaty, Ph.D.


2 comments:

  1. You wrote that a sun lotion "50" lets pass trough 3% of the sunrays, and a lotion "100" lets trough 1.5%.
    If the "100" lets then pass half of the sunrays of the first one, isn't it twice as protective?

    ReplyDelete
  2. LK raises a good question, that is also answered on the epa.gov (US Environmental Protection Agency, "You should be aware that an SPF of 30 is not twice as protective as an SPF of 15; rather, when properly used, an SPF of 15 protects the skin from 93 percent of UVB radiation, and an SPF 30 sunscreen provides 97 percent protection" See chart SPF vs UVB protection http://www.epa.gov/sunwise/doc/sunscreen.pdf

    The non-linear curve begins to flatten at SPF25. Depending on the rounding, the chart looks like this
    according to skincancer.org:

    SPF 15 blocks 93% UVB rays (not UVA) or passes 7%
    SPF 30 blocks 97% or passes 3%
    SPF 50 blocks 98% or passes 2%
    SPF 100 blocks 98.5 or passes 1.5%

    Further, 2 layers of SPF 5 does not equal SPF25.

    In other words, the intent of the first blog was to demonstrate that ordinal scale numbers (like SPF) can mislead, and that ratio scale numbers (like % rays) are more useful in QFD math.

    ReplyDelete

Comments are welcome and will be posted upon review.