Skip to content

You Can’t Quantify The Unquantifiable

I stole the picture below from YouGov.

A new YouGov study reveals exactly how positively and negatively the population perceives various descriptions to be.

YouGov showed respondents a selection of adjectives from a list of 24 and asked [a group of about 1,000–2,000 Britons] to score each on a scale from 0-10, with 0 being “very negative” and 10 being “very positive”.

Study the picture for a moment.

Even more important for us is a second picture at their site which compares average responses of Britons and Americans. There are many differences: abysmal had a difference (by my eye) of about 1.5; others were smaller. But even the small ones represent mean differences, meaning there is some variability in the differences, which they didn’t picture.

Now the picture above smacks of “density estimates”, which I won’t explain, but think of it as a way to make fancy histograms. It (over-)smooths the actual results. (I was in grad school when smoothing methods were going to save the world.) Ignore that twist. It’s clear enough that substantial variation exists for each word.

It’s true there is little overlap in scores in words at the extremes, such as perfect and appalling. Nobody I know who speaks English would ever confuse these words. Then I don’t know a lot of people. The vocabulary of benighted college students is particularly thin these days, so one never knows.

The point is this. That the variance seen is in no way unusual or unexpected, not in the different “scores” given the individual words nor in the differences in the scores by country. Of course, the so-called country differences could be partly genuine (they speak a weird English over there) and partly no different than if you were to take just the Britons and split them in two using whatever marker you like. In other words, it’s another measure of variation.

The differences within in any word are well withing the differences touted in wee-p research. Meaning, of course, that many of the results touted to be caused by theories favored by the researchers could just as well be caused by different understandings on the words. Yes, even with these variabilities differences in means between words can exist, but there is no proof this comes about because of the theory or because of the differences in understanding.

Attempting quantification of unquantifiable mental states and feels is far too common.

How much do you agree with this assessment on a scale of -42 to 3 in increments of 1/e before -10 and 1/pi after it, though those in Britain, may use mostly whole numbers.

13 thoughts on “You Can’t Quantify The Unquantifiable Leave a comment

  1. Examining the chart, one in particular stands out. This exceptional word is “average”. It appears to be a nearly perfect mean, in the center of the chart, with tiny, well balanced deviations.

  2. > “Attempting quantification of unquantifiable mental states and feels is far too common”

    Sounds like most subjective priors to me. I’m still interested in a coherent & constructive definition of a logical prior. Data priors we all like…

  3. The distribution calculations show that a goodly proportion of folks, decent or better that is, scored words greater than 10. Hmmm.

  4. It’s true there is little overlap in scores in words at the extremes, such as perfect and appalling. Nobody I know who speaks English would ever confuse these words.

    I wonder what perfectly appalling and appallingly perfect might mean.

  5. Do I detect a tiny negative blip in the distribution for ‘awesome’. As in ‘I find the crass, witless scientific illiteracy of the BBC awesome’. I also wonder what the distribution for ‘crass’ and witless’ might be. Just a thought.

  6. There are many words in English that have fuzzy meanings. Take cold, cool, warm and hot. What temperature ranges might they mean?

    Years ago during a winter, my airplane partner and I flew to Florida to pick up an airplane. On the way back we stopped at a place a bit north of Daytona to have lunch with a friend of ours who had moved there about a year before to be the airport manager. When our friend came to lunch he was wearing a t-shirt, flannel shirt, sweater, a fleece coat, scarf and gloves. After lunch, he invited us to use the phone in his office to call Flight Service so we “didn’t have to call while outside in the cold.” We explained to him that when we had left that morning, it was 5 below zero and 75 wasn’t cold. His office had three running quartz heaters.

    So, when someone tells you it’s cold outside, what do they mean?

  7. Per:

    The graphs seem to indicate responses from -1 to 11. But no doubt that is an artifact of the smoothing method. If, for some reason I can’t fathom, one were to take this business seriously, one would need to get hold of the raw data.

  8. Words can’t be quantified, except by extremes or difference, or by contrast, which isn’t really quantifying but ordering, sequencing by comparison then applying numbers afterwards. Hence,
    “compare and contrast” from non scientific essays.

    The word abysmal gets negative points because of it’s ‘juiciness in the second syllable, where it dips to a low.
    Like ‘DISmal.
    Although for exQUISite, it works the other way for the same reason. Unless you say all three words one after the other! Then exQUISite sounds sarcastic! Anyway, I’ve gone off that word, it’s got creepy connotations. Just so happens it’s used in medical terminology to explain certain things and people don’t understand if their mind is in the dustbin.

    I’m adding to a growing list of words that need debriefing, in the military sense.
    They are top secret words and cannot be revealed.

  9. Why isn’t the word “fair” in that list? Fair is one of the politicians favorite words. Politicians are always going to make things fair.

  10. IS certainty of a proposition quantifiable? This is something I don’t get about Briggs’ interpretation of probability. I lean towards the frequentist interpretation and I believe unpacking of certainty would ultimately require invoking frequentist interpretation at some point.

  11. It’s a circular situation, quantifying certainty in a proposition, even given ‘some evidence’. Or it’s a hall of mirrors or some other analogy.
    You either know or you don’t. If you’re not sure, numbers ain’t gonna help. The uncertainty is objective, the quantification is subjective, or it would be certain! So it can’t be described as a final quantity. Just described.

    So the hard sums look like a smoke screen, or a placebo. Not contrived deliberately, it’s just the way it looks.

Leave a Reply

Your email address will not be published. Required fields are marked *