Tweet Hate Map: Awful, Really Awful Use Of Statistics

I hate this kind of thing
I hate this kind of thing
I give you “Dr” Monica Stephens from Humbolt State University and her widely dispersed Twitter “Hate Map.”

The thumbnail here doesn’t give the picture the full injustice it deserves, so you’ll have to click on the link to view the real map, which is interactive.

She had her students count “hate” tweets over eleven months, aggregate them at the county level, and then plot the results on a map. She called this counting an “algorithm sentiment analysis,” which is like calling the guy who collects pop cans from the trash an “aluminum reorientation environment engineer.”

First sin, and grounds for automatic disqualification: her definition of what counts as “hate.” Tweets which had “homophobic”, “racist”, or “disability” words in them (supposedly in context). Mouse over the headings on the map to see the brief list, or go to their FAQ to see all words.

These included ni**er1 and bitch, words which are not always used hatefully, at least if we understand hip hop, rap, and black popular culture. Why, take these words away and there would be no modern music!

Indian, they swear, is a hate word. So are monkey, gringo, cripple, and honky.

Actually, strike that. The list is not the list. Turns out the list was only a starting point, because, for example “honky/honkey/honkie was discarded, as most of the tweets were positive references towards honky-tonk music and not slurs”.

Bitch also had to go. Too many instances, you see, 55 million-plus, since each tweet had to be read by a student. What about other words? Here Stephens became whiny and evasive. She wanted to include more words, she really did!, but her “research funds, and thus the scope of this project, are extremely limited. It’s not like we have billions of dollars in funding lying around.”

I weep for her, I really do. But after reading all her explanations, it appears that she only used 10 words: dyke, fag, homo, queer; chink, gook, ni**er, wetback, spick; cripple.

Apparently none that were disparaging to whites, males, or females in general made the cut. That means, even if accurate, the “hate” map only pertains to animosity toward those with nonstandard sexual desires, blacks, East Asians, and Latinos. Oh, and the “differently abled”.

Her excuse for excluding most people? “If you are a disgruntled white male who feels that the persistence of hatred towards minority groups is a license to complain about how discrimination against you is being ignored, just stop.” Shut up, she explained.

Second, the ridiculous artifact caused by the mapping application she used and her sampling scheme. Look at those red blobs of hate! The entire Eastern USA is sliding into Hades.

Stephens did normalize the number of “hate” tweets by the number of total tweets in each county, but this did not help, as we’ll see.

Go to the map and zoom in the maximum extent possible. Look at Chicago and Detroit, including their suburbs. Nary a hater to be found. Can this be? Surely New York, or at least New God-Help-Us Jersey? Nope. Dallas, L.A., Indianapolis? Uh-uh. Any city where lots of people live? Sorry.

Instead, navigate up to the darkest blob in northernmost lower Michigan, north of the town Gaylord (where yours truly was bred). Looks like Cheboygan county, population 26,000.

Stephens only collected 150,000 tweets from the entire USA over eleven full months. There are 3,143 counties in the USA (Alaska and Hawaii are on the map; just zoom out to see; look how red everything now is!). Population isn’t evenly dispersed, but that’s an average of fewer than 47 “hate” tweets per county. Obviously counties with major cities will have had lots more “hate” (and normal) tweets than counties with small towns. This means small counties had to have many fewer than 47 “hate” tweets.

Now how many total tweets could have come from the mostly older not-too-tech-savvy folk in tiny Cheboygan county? A hundred? Two? A thousand? Not too many. All it would take was one lunatic with a sour mouth making just one intemperate tweet (once in eleven months) and voilà! we have a center of hate.

There are many more problems, the chip on Stephens’s shoulder not the least, but forget it: this study is so awful that I want to weep.


Update WordPress ate the first version of this. I have no idea what happened. It was up and then just disappeared. The original is not anywhere. Thank God for a reader who received an email version, so I was able to restore it.

—————————————————————————-

1I do not spell out some words because of squeamishness—I am, after all, military trained in language arts—but because of what the presence of these words would do in search engines. You behave yourself too.

Thanks to Roger McDermott for alerting us to this topic.

18 Comments

  1. I couldn’t help but start laughing that at the border of Idaho-Utah-Wyoming… there appears to be… a block and tackle *nudge nudge*. You know, meat and two vegs *wink wink*.

  2. Briggs

    Nate,

    How surprising that the only places that mentioned zombie where those containing people who knew the word zombie. But since this “research” was undertaken on a computer there must be something to it.

  3. Why did she have her students count the tweets? If her criteria was the use of specific words in tweets, has she not heard of a filter to pull out all tweets with the designated words? One could just add additional filters to refine the lists. Then she wouldn’t have needed billions in research money–just enough to pay a computer programmer to write the filters.

  4. Briggs,

    I know, right? At least we can feel secure knowing all us internet warriors are keeping us safe.

  5. DAV

    Look at all other maps the teacher has made

    Hey, if you’ve found a way to milk the cow, keep going until it no longer works. Sadly, though, she likely thinks what she’s doing is of any value.

    Briggs,
    Look at Chicago and Detroit, including their suburbs. Nary a hater to be found.

    Obvious centers of enlightenment. Way to Go, Detroit!
    See? Her efforts and those of others are making an impact.

    Sheri,
    Then she wouldn’t have needed billions in research money–just enough to pay a computer programmer to write the filters.

    And why wouldn’t that require billions? Would you work for free?

  6. Ken

    There’s some interesting commentary addressing the analysis at the map-maker’s website: http://www.floatingsheep.org/2013/05/hatemap.html

    A typical recurring Comment-Response pattern is an Objective comment about tallying hate directed against a sub-/group in a larger majority group (e.g. white hillbilly/redneck–majority because they’re white) invariably receives Angry & Hostile attacks, often personally directed. Within a majority group, per that bunch, in-group stratifications cannot exist, which of course is disconnected from reality.

  7. DAV–No, I wouldn’t work for free. However, if I thought I could get billions for writing a simple filter for Twitter and then writing a paper, I wouldn’t be sitting here commenting on blogs. I’d be out there making those billions.

  8. DAV

    Sheri,

    Really? You wouldn’t just make a couple then sit back and do nothing but comment on blogs?

    Actually, I think she used students because a filter would have a hard time determining context. At least, that’s what she would likely say. I’ll bet the real reason is they are cheap. I don’t think her interest is in statistical integrity, though. Look at the beer/MJ map. The price/oz. of marijuana required a minimum of two (count them!) reported price samples to be included.

  9. You know how it is–you make a couple of billion and then you start getting hooked on it….Or maybe that was the MJ study? 🙂

    (I did consider that the context might be a factor, but I think a couple of passes with various filters could do at least as well as the students!)

  10. Doug M

    What offends me about this map are the fuzzyness of the blobs…if you are counting the % of racist tweets in a county, keep the colors inside the lines.

    This may be a fault of the cartographers at google, rather than Dr. Stephens, but it still looks all wrong to me.

  11. DAV

    Sheri,

    Money is indeed addictive. Probably why the government is so hellbent on taking it from us. It’s for our own good.

  12. Ray

    With such a great talent for discovering hate, Dr. Stephens should have a great future with the Southern Poverty Law Center. I am surprised Dr. Briggs hasn’t been declared a hate group by the SPLC. Any criticism of a woman means you have declared war on women.

  13. David L. Hagen

    The real case said “Dr” Monica Stephen has on “hate” is with the Almighty:
    Exodus 18:21 (NIV)

    But select capable men from all the people—men who fear God, trustworthy men who hate dishonest gain—and appoint them as officials over thousands, hundreds, fifties and tens.

    Exodus 23:5

    If you see the donkey of someone who hates you fallen down under its load, do not leave it there; be sure you help them with it.

    Leviticus 19:17
    “‘Do not hate a fellow Israelite in your heart. Rebuke your neighbor frankly so you will not share in their guilt.

    2 Chronicles 19:2

    But Jehu the son of Hanani the seer went out to meet him and said to King Jehoshaphat, “Should you help the wicked and love those who hate the Lord? Because of this, wrath has gone out against you from the Lord.

    Psalm 5:5 ESV

    The boastful shall not stand before your eyes; you hate all evildoers.

    Psalm 11:5 ESV

    The Lord tests the righteous,
        but his soul hates the wicked and the one who loves violence.

    Malachi 2:16 NASB

    For I hate divorce,” says the Lord, the God of Israel, “and him who covers his garment with wrong,” says the Lord of hosts. “So take heed to your spirit, that you do not deal treacherously.”

    Romans 12:9

    Love must be sincere. Hate what is evil; cling to what is good.

    Hebrews 1:9
    You have loved righteousness and hated wickedness; therefore God, your God, has set you above your companions by anointing you with the oil of joy.”
    Isaiah 5:20 (NIV)

    Woe to those who call evil good
        and good evil,
    who put darkness for light
        and light for darkness,
    who put bitter for sweet
        and sweet for bitter.

    How will she plead?

  14. Noblesse Oblige

    Two can play at this game. How about a “whacko map” generated by the right key words. For example:

    gay marriage
    global warming
    contraception
    rights
    fair
    fair share
    environment
    bullet train
    subsidy

  15. Bill S

    Maybe I am dense or maybe dinner was too filling – – –
    But I was not able to find any statistics in this mess.
    Just doggerel.

  16. Sander van der Wal

    Calling people names is different from beating people up. “Hate” Is a much stronger word than, for instance, “dislike”. So, if an American uses the word “hate”, is it about somebody you would not invite to a birthday party, somebody who’s car tyres you would puncture, or somebody you would try to murder if you think you can get away with that.

  17. Fundamental flaws.

    1. No clear sample frame.
    2. Lousy definitions.
    3. No attempt to correct for population.
    4. Use of a commercial service as a proxy measure for whatever hate is.

    And that is before we start on how she did her analysis. This paper defines massive fail

Leave a Reply

Your email address will not be published. Required fields are marked *