Statistics

Scientism Of The First Kind: Science Shows Laptops Distract Students

https://twitter.com/mattstat/status/722028191582445568

Here’s the headline. (It’s NPR, so don’t expect much.) “Attention Students: Put Your Laptops Away”. And here’s the relevant sad bit:

For one thing, research shows that laptops and tablets have a tendency to be distracting — it’s so easy to click over to Facebook in that dull lecture. And a study has shown that the fact that you have to be slower when you take notes by hand is what makes it more useful in the long run.

Research shows. Research shows. Research shows. Research shows that two of the most dismal words in the English language are “research shows.”

Only scientism of the first kind accounts for the necessity to undertake a study to discover whether laptops can distract students.

Scientism of the first kind, as pithily described in the tweet linked at top (or here), is the idea that only facts certified by scientists, or by experiment, or that have been studied in some official way, are worthy of attention and that all other knowledge is suspicious, dangerous, or false.

Commonsense and routine observation can’t be trusted because they might—might—lead to error. So even though all experience shows laptops, and other “devices”, are distracting at some times to some students in some situations, laptops might not be distracting in some future group at some time in some place. Or something.

This is confused because it’s not clear what the objection is to the commonplace observation that laptops distract. Perhaps it’s that knowing, via plain observation, laptops distract does not quantify the phenomenon. We can’t put a number on the precise number of students in some future class who will be distracted. Hunger for needless quantification is another symptom of scientism.

Heck, we don’t even know if all races, sexes, sexual desires, class types, times of day, income statuses, and on and on and on are equally distractable. There may be disparities! Can you imagine the horror if white men were to be discovered, via a formal study, to be less distracted than, say, female transsexuals?

I’m teasing, but my joke has a good chance of being true (I refuse to check).

Obviously, one reason for scientism is the relentless desire for academics to publish something, anything, that resembles scholarship. Since there are so many academics, publications rise like a tsunami. A peer in need of a paper will see the journal article “Laptops distract (p < magic number)” and will realize that, while this is a fine study, it hasn’t been conducted in introductory sociology courses at universities of the type he coincidentally works at. And is there a difference between tablets and laptops? Let’s test!

So he’ll repeat the study, and add to the world’s knowledge by an infinitesimal amount. Given that he’ll use statistics, he’ll, like everybody, forget that correlation isn’t causation, and use the certified-by-hypothesis-tests correlations to “prove” various theories about why laptops distract.

Suddenly, then, there are two papers on this most important subject. There are many more than two academics, however, and they all need papers (their hunger for them is insatiable), and so a few more will enter the burgeoning new sub-sub-sub-field of laptop distraction studies. Soon, one will speak of the “literature” of laptop distractions.

The whole thing will take the patina of science. It all looks so formal! At that point, none would dare write a news story in a respected outlet without seeking a quote from an authority, a quote which will begin with the words “Research shows…”

Too gloomy an outlook? NPR continues:

In the study published in Psychological Science, Pam A. Mueller of Princeton University and Daniel M. Oppenheimer of the University of California, Los Angeles sought to test how note-taking by hand or by computer affects learning.

“When people type their notes, they have this tendency to try to take verbatim notes and write down as much of the lecture as they can,” Mueller tells NPR’s Rachel Martin. “The students who were taking longhand notes in our studies were forced to be more selective — because you can’t write as fast as you can type. And that extra processing of the material that they were doing benefited them.”

Look at all those words to say what everybody already knew! Note the serious tone. The story continues:

Mueller and Oppenheimer cited that note-taking can be categorized two ways: generative and nongenerative. Generative note-taking pertains to ‘summarizing, paraphrasing, concept mapping,’ while nongenerative note-taking involves copying something verbatim.

We finally reach theory, the most essential part of any scientific work, because it is theory that leads to the four most-cherished words in any paper: “More research is needed.”

Categories: Statistics

25 replies »

  1. “…you can’t write as fast as you can type.”
    At another time and place Mueller and Oppenheimer might have quantified the differences between typists using QWERTY and Dvorak keyboards, answering an engineering design question important to manufacturers. That seems like useful research. So what makes it different from understanding impediments to efficient learning? What’s an appropriate way to study how note-taking might be made more effective? Those are serious questions. The complaint here is that M & O really are interested in the wrong thing — their careers, rather than the success of their students — and they dress it up in new clothes (with a hat) to make it look important.

  2. (From GKC’s “Songs of Education”)

    Form 394411102, Sub-Section X

    “All practical Eugenists are agreed on the importance of sleep.”–The Eugenic Congress.

    When Science taught mankind to breathe
    A little while ago,
    Only a wise and thoughtful few
    Were really in the know:
    Nor could the Youth his features wreathe,
    Puffing from all the lungs beneath:
    When Duty whispered softly “Breathe!”
    The Youth would answer “Blow!”

    When Science proved with lucid care
    The need of Exercise,
    Our thoughtless Youth was climbing trees
    Or lightly blacking eyes:
    To reckless idlers breaking bounds
    For football or for hare-and-hounds,
    Or fighting hard for fourteen rounds,
    It came as a surprise.

    But when she boldly counsels Sleep
    To persons when in bed,
    Then, then indeed men blush to see
    The daybreak blushing red:
    The early risers whom we term
    Healthy, grow sickly and infirm;
    The Early Bird who caught the Worm
    Will catch the Germ instead.

    For this at least be Science praised
    If all the rest be rot,
    That now she snubs the priggish child
    That quits too soon his cot:
    The pharisaic pachyderm
    Of spiritual pride shall squirm:
    The Early Bird catches the worm,
    The Worm that dieth not.

  3. “Research shows…” usually translates as, “Liberals have been sitting around brainstorming, and have reached the conclusion that…”

  4. I must note here that the corollary to this is “There MUST be a link” to whatever one says in a comment box or anywhere else for that matter. I have spent four days now in exchanges with a commenter elsewhere asking for a link to a theory I mentioned in passing (He obviously found the theory majorly upsetting) and he continues to demand a link. Telling him there are books and professional journals did no good. it is NOT true if there is no link to prove it.

    Actually, hidden deep in the text of some articles I’ve seen lately are the words “correlation does not equal causality” BUT it might. I was actually surprised at the CYA tactic of saying correlation and causality are not equivalent. Maybe a study showed you could be sued for this if enough distracted students happened to land on the Goldwater Law Firm page and in their distraction thought up new and different ways to sue people.

    Gary: What’s wrong with having a study is a guy with a seating chart, standing in the back of the room where he can see what the students are doing, can accurately tally who is surfing Facebook, who is taking notes on computer, who by hand and then look at the grades the students get. With a large sample size, you won’t need a p-value to see how well which method works. So, yes, the problem is the motivation and the dressing it up. It’s back to the idea that observation is sufficient in many cases. Even your keyboard question can be answered not by a formal study, but by looking at sales of keyboard types or a survey of users. That’s marketing research, which, if that is what this study is, it needed to be labeled as such

  5. I think that this is a better example of artism, which is the idea that whatever teaching techniques are useful for an arts course (e.g. English) will work in all courses (e.g. Physics). In an equation and diagram intensive lecture, computers are much slower than hand notes. You still can’t beat the blackboard and students can easily keep up unless the instructor is pathological which is regrettably not uncommon. Students have been known to use a phone camera to record images especially for overheads. The bottom line is that each student much adapt to the circumstances of each course and instructor and there are no universal techniques that well work for all students and all courses. The last person to know what technique works best in his course is the instructor. You stated in a previous post that students prefer the traditional lecture method over on-line courses. Did you determine this through scientism or artism? 😉 Maybe the students just prefer the system that they already know how to game.

  6. “Scientism: a BELIEF that you can’t BELIEVE until the BELIEF has been quantitatively certified by a believing scientist.” [emphasis added]
    – One of many similar definitions by adherents of “scientism”

    Does that mean that a non-believing scientist that quantitatively certifies a belief isn’t good enough for some?

    Why is “belief” the central issue; why isn’t an acceptance of objective fact the central focus?
    – Like, once upon a time, the then-heretical acceptance-as-fact that the Earth orbited the Sun
    — A fact-not-accepted in some circles because that conflicted with an ancient cherished belief that the Earth was the center of the Universe
    — Which, some assert, that clash between fact & belief did not get Galileo in trouble…

    Sometimes, the study of obvious facts, or truths, generates new insights and discoveries.

    Sometimes, accepted facts or beliefs turn out to be false, and often enough verifying objective obvious facts prove beneficial:

    “It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.” (Mark Twain)

    So, often enough, when things so-obvious-there-can’t-be-anything-to learn-there are actually studied, some unexpected and meaningful and useful insights often enough are learned.

    It’s interesting that those invoking “scientism” (as opposed to ‘flawed studies,’ ‘pseudo-science,’ etc.) invariably associate it with “belief” or “faith” rather than objective fact and irrefutable truth. The definition provided invokes “belief” or “believe” three times in 18 words and makes “belief” and a relationship between believers the central issue — not the discovery of & acceptance of credible fact.

    That’s a clue:

    Invoking “scientism” is not about discipline & rigor…about stopping sloppy, pointless and even flawed science … much less learning about reality as reality is … “scientism” is all about thought-stopping and keeping pesky facts from being found out & brought to a “believer’s” conscious awareness.

    Adherent’s of “scientism” are not those that demand scientific proof to accept a belief as the common definition asserts — note that those people do not invoke the term/concept, “scientism.”

    Adherent’s of “scientism” — those prone to invoke the term & apply its concept — are invariably a subgroup of “believers” who’d prefer science didn’t find out any objective fact that might force them to have to recognize and accept that a cherished belief was wrong all along. Since they cannot do that they invoke “scientism” as an ad hominem sweeping generalization, which in their mind is convenient excuse for dismissing any fact credible science might produce that might also undermine a cherished belief — by preemptively giving them an excuse to be dismissive of science & the entire scientific field itself.

    Doubt that? Observe who uses the term/concept & in what relations they apply it vs those that don’t.

  7. From AAAS concerning the definition of scientism:
    1) But this formulation is so broad as to render it virtually useless. Philosopher Tom Sorell offers a more precise definition: “Scientism is a matter of putting too high a value on natural science in comparison with other branches of learning or culture.” (2) MIT physicist Ian Hutchinson offers a closely related version, but more extreme: “Science, modeled on the natural sciences, is the only source of real knowledge.”

    That seems to be what is being discussed here—can science measure those things which have no physical forms, such as success, learning, etc? Scientism says it can, but does not seem to explain why that is so.

  8. … “scientism” is all about thought-stopping and keeping pesky facts from being found out & brought to a “believer’s” conscious awareness.

    Ken, this describes the motivation of those who would criminalize “climate denial”. Couldn’t have said it better myself.

  9. Ken,
    That argument is against common sense in favour of a waste of time.
    Like the notorious ‘the computer says no!”;
    Or the minion who daren’t use their own initiative for fear of stepping outside of their role and losing their job because they had a thought.
    These are symptoms of a widely held idea that we must wait for the authorities to tell us how to act what to think and how to feel. It’s all part of the same syndrome.
    I call it insulting the intelligence of people. It’s also an argument against truth in that something can’t be true if there hasn’t been a study that showed it true. This has been said above.
    It has reached the material of comedians though, so it won’t be long before it goes out of fashion. Science will be a pariah before it returns to the respected field it used to be.

    If only students would take better notes, more of them would succeed. It’s the laptops.
    If only designated people within an organisation think, those employed to think, the company or organisation will succeed.
    Now we know about school based water intervention they have built on the knowledge base. They could introduce it into the workplace and see if it applies. It doesn’t do to assume.

    I’d like to do a study of the swimmers versus non swimmers on the islands of the Maldives and compare them with those of Bora Bora. Careful analysis of confidence in water and swimming stroke would be needed not just to ask people to fill in a form. Physical observation would be necessary.
    After I discover the facts. Then I would be there in an overseeing capacity in the places where swimming lessons were needed most. With provision of equipment I could probably fulfil the oversight from elsewhere.
    In the winter I would turn my attention to northern hemisphere issues in particular those of safety on the ski slopes. There’s far too much hip flask activity. I’d particularly like to show that snowboarders are a danger, are too scruffy and must be banned. Change of snowboarding attire will be considered a sign of improved attitude towards other slope users and careful monitoring on a regular basis in order to update the database.

    When I was a student I recorded lectures and took notes later. I split the tapes with another student who wrote braille so we both saved time.

  10. In his book At the End of an Age, historian John Lukacs commented that the vast explosion of documentation at the end of the Modern Age meant that inflation set in, and the individual documents became worth less and less.

    why isn’t an acceptance of objective fact the central focus?

    Because a fact has no meaning absent a theory (or “interpretation”).
    We observe the sun crossing the sky, but does this mean that the sun goes round the earth, or does it mean the earth rotates?
    We observe a geological layer with a higher concentration of iridium than is normal; but does this mean that an (iridium-rich) asteroid struck the earth, vaporized and deposited its detritus, or does it mean the Deccan Traps erupted for a geological age spewing iridium-rich ejecta into the upper atmosphere atmosphere, later deposited everywhere?
    We observe (as Xenophanes did) marine fossils in the high-up hills of Greece, but does it mean that a world-girdling flood washed fish and shellfish onto the mountains, or does it mean that the mountains used to be sea floor and then by some magical process were uplifted into mountains? (Or do we assume with ibn Sinna that they are only rocks that happen by coincidence to look like shells and bones?)
    There is nothing so rare as an “objective” fact. The very word “fact” implies “something done or accomplished,” factum est.

    A fact-not-accepted in some circles (that the Earth orbited the Sun) because that conflicted with an ancient cherished belief that the Earth was the center of the Universe

    Actually, the belief was that the Earth was at the bottom of the world, the most ignoble place. That’s why the Copernican theory was adopted enthusiastically by the humanists, who desired to elevation mankind into the heavens.

    The theory had not been accepted because it appeared to conflict with the data. There was no discernible stellar parallax (which there ought to have been if the Earth revolved around the sun), and there was no discernible Coriolis effect (which there ought to have been if the Earth rotated on itself diurnally). Nicholas Oresme had shown in the 14th century that the motion of the heavens was ambiguous and could mean either that the heavens revolved or that the Earth rotated; but there were other objections: if the Earth sped around the Sun, why wasn’t the Moon left behind? If the Earth rotated at great speed why was there not a strong, steady headwind from the east? Etc. IOW, these objective facts could not be dismissed until entirely new mental furniture was put in place in order to see the facts from a different angle.

  11. “And a study has shown that the fact that you have to be slower when you take notes by hand is what makes it more useful in the long run. ”

    Unless you are so much slower writing that you can’t keep up with the lecture and effectively miss half the lecture trying to capture 1/4th of it in notes. At that point, hand written notes are useless.

    I had exactly that kind of problem taking notes in classes. By the time I graduated High school I had pretty much stopped trying to take notes at all.

    If a teacher insisted that I try to take notes, I would scribble in my note book so it looked like I was trying to take notes.

  12. “can science measure those things which have no physical forms, such as success, learning, etc? Scientism says it can, but does not seem to explain why that is so.”

    No, scientism doesn’t say that science can measure success, learning, etc. What scientism says is that if science can measure something it doesn’t exist. So it’s not that science must necessarily be able to measure learning, but that if it can’t, then learning does not exist.

  13. MattS,
    “I had exactly that kind of problem taking notes in classes.” Exactly, there is always the choice between listening carefully to the lecture or taking notes and understanding, hopefully, later. Some instructors are impossible to take notes from – my first year physics instructor was the prototype. He lectured without notes himself and filled the blackboard with an error prone mess of equations. One of my daughter’s instructors was a triple threat – no textbook, wrote little on the blackboard, and spoke too softly to be heard clearly. Others are very easy to take notes from. Then there is the relationship between exams and lectures which can vary from a close relationship to none at all. I remember a discussion I had with a university colleague who was complaining that the students did not follow his advice about preparing for exams. I claimed that that was the safest position for a student to take since there were three basic instructor types – those who give good advice, those who give unreliable advice, and those whose advice is the exact opposite of reality. All three are commonly encountered (I could name names) and since it is near impossible to determine ahead of time it is best to go your own way. Sometimes the guidance of previous students is helpful or a cache of the exams of previous years.

  14. Did anyone actually read, or skim, the actual study paper as opposed to (like Briggs’ self-reports) reading only a news article about it … as if a news report is in any way authoritative?

    Here’s some bottom-line findings most of us probably didn’t know — from the actual paper, not the news report (though its there), or, Briggs summary of …who knows what…:

    – Using a laptop to take notes results in transcribing lectures (somewhat mindlessly and/or indiscriminately), as opposed to synthesizing lecture content as must occur with handwritten notes;

    – If the student cannot/does not study, either method generates similar results;

    – If the exam addresses factual material (e.g. recall of historical events) then either note-taking approach works about as well;

    – However, if a student has a) time to study, and b) the exam addresses conceptual material (e.g. applying case law to a legal scenario) those taking handwritten notes perform measurably better.

    In a highly competitive school (e.g. where students are vying for limited upper class positions, internships, etc.) any small edge may make the difference in tipping the outcome in one’s favor — a couple of A-‘s instead of B+’s could be the difference between a GPA that renders one eligible vs ineligible for a PhD. The study’s findings imply (insufficient details are provided) that the actual benefit is greater than this — that tiny edge might translate into a significant difference in a highly competitive program, especially where grading curves are applied.

    Not to mention the potential in business situations were most fully appreciating a prospective client’s position matters — better to write, not type, one’s notes of their feedback when one’s proposal will be based on those notes….

    That’s not “scientism” (by any definition) — in a competitive world where the difference between victory & ignominy may be milliseconds, a 100th of a GPA (yes, that has put some above a cut line), etc., every little edge helps. That’s capitalism in action (capitalism — free market competition — is always striving for an edge).

    The study, in other words, had nothing at all do with how much students are distracted — though that’s the theme Briggs’ harps on, mentioning “distract/ing/ion/etc.” some 13 time in his essay, plus the title.

    So, how many people genuinely believe he actually read the actual study paper he critiqued?

  15. Ken: I don’t see where what you have presented in any way shows the study is not scientism. Both MattS and Scotian stated they found hand writing notes to be very ineffective for them. On the other hand, I hand wrote notes on everything in every class and even outlined chapters of my books by hand. I still write blog entries out in long hand most of the time. The study in no way addresses the very personal differences in the students, but rather generalized what worked in the study. People are individuals. Science may provide some guidance on how people learn, but individual variation makes that knowledge very limited in use. Students who read the study or professors who relate the information in the study, or professors who insist on note taking via some particular method may actually damage some students’ learning, resulting in decreased scores, the opposite of what is hoped for.

  16. Sheri,
    “Both MattS and Scotian stated they found hand writing notes to be very ineffective for them.”

    I hate to interrupt someone when they are on a roll, but I didn’t say that and it is, in fact, not true.

  17. @acricketchirps,

    “Matt S – Yet here you are today in a rewarding career as an internet comboxer. It just goes to show.”

    Actually, I have a very rewarding and well paid (100K+/year) career in IT, but thanks for trying.

  18. Oh. I hadn’t seen any of your IT work–it must be good to command such remuneration. And you’re very welcome; I’m not proud but, as you say, I am trying.

  19. Ken,
    What did the study show you that you did’t already know?
    What Is objectionable is the money and time wasted on such work which originates in the public purse.
    Companies can waste their money as can individuals but they don’t have to expect everybody to be interested.
    If they froze all journalism and media studies courses for ten years there’s be nobody to report on studies that show and it would help cut the vicious triangle between the media, the politicians and the scientists, or rather ‘academia’.

    As for actually reading the study? You are joking, it’s bad enough reading Briggs summary. This could be a template for all the bad and unnecessary studies to which media think our attention needs to be drawn.

    When they’ve got something really important to tell the public they won’t want to know.

    To pass exams, listen, learn by writing what you hear or by dictation and
    past papers, past papers past papers.
    If the course has practical application such as business or in my case clinical application, don’t write about what is considered the most correct methods. Write what academia wants to hear about, i.e. ten years out of date.
    “write about phlegm, and lots of it. That’s what the CSP wants to hear about. Don’t write about what I’m telling you but when you get out there, you can do things properly.” Alexandra Howe.
    A certain cynicism is required to get on in business Ken, everybody understands this.

  20. @acricketchirps

    “Oh. I hadn’t seen any of your IT work”

    You wouldn’t. I spend 17 years in the IT department of an electric utility. Now I work for a firm that does IT consulting for utility clients. I work in a very specialized area (geographic information systems)

  21. Well, that explains it. Still if I were to put a value on your work here in the comment box, I think it would be right up there. My own comments, I’m sure, are worth something less than $90K per year (and that’s Canadian).

Leave a Reply

Your email address will not be published. Required fields are marked *