5 Reasons It’s So Hard To Think Like A Scientist

By Christian Jarrett

Thinking like a scientist is really hard, even for scientists. It requires putting aside your own prior beliefs, evaluating the quality and meaning of the evidence before you, and weighing it in the context of earlier findings. But parking your own agenda and staying objective is not the human way.

Consider that even though scientific evidence overwhelming supports the theory of evolution, a third of Americans think the theory is “absolutely false”. Similarly, the overwhelming scientific consensus is that human activity has contributed to climate change, yet around a third of Americans doubt it.

We Brits are just as blinkered. In a recent survey, over 96 per cent of teachers here said they believed pupils learn better when taught via their preferred learning style, even though scientific support for the concept is virtually non-existent. Why is it so hard to think like a scientist? In a new chapter in the Psychology of Learning and Motivation book series, Priti Shah at the University of Michigan and her colleagues have taken a detailed look at the reasons, and here I’ve pulled out five key insights:

We’re swayed by anecdotes
When making everyday decisions, such as whether to begin a new treatment or sign up to a particular class at uni, most of us are influenced more powerfully by personal testimony from a single person than by impersonal ratings or outcomes averaged across many people. This is the power of anecdote to dull our critical faculties. In a study published last year Fernando Rodriguez and his colleagues asked dozens of students to evaluate scientific news reports that drew inappropriate conclusions from weak evidence. Some of the reports opened with an anecdote supporting the inappropriate conclusion, other reports lacked an anecdote and acted as a control condition. Regardless of their level of university training or knowledge of scientific concepts, the students were less competent at critically evaluating the reports when they opened with an anecdote. “Anecdotal stories can undermine our ability to make scientifically driven judgements in real-world contexts,” the researchers said. Of course much health and science news in the mainstream media is delivered via anecdotes, increasing the likelihood that news consumers will swallow any claims whole.

We’re overconfident
Confronted with a scientific claim, another reason many of us find it hard to reflect on it scientifically is that we overestimate our comprehension of the science. A study from 2003 asked hundreds of university students to read several science news stories, to interpret them and rate their understanding. The students made many interpretative errors – for example, confusing correlation for causation – even though they thought they had a good understanding. This is redolent of a survey from the 1980s of thousands of British and American citizens: nearly 60 per cent stated they were moderately or very well-informed about new science findings and yet far fewer were able to answer easy questions about elementary science. Part of the problem seems to be that we infer our understanding of scientific text based on how well we have comprehended the language used. This means that popular science stories written in layman’s language can contribute to false confidence. This “fluency bias” can also apply to science lectures: a recent study found that students overestimated the knowledge they’d derived from a science lecture when it was delivered by an engaging speaker.

We’re biased by our prior beliefs
This obstacle to scientific objectivity was demonstrated by a now-classic study from the 1970s in which participants were asked to evaluate scientific research that either supported or conflicted with their prior beliefs. For instance, one of the to-be-evaluated studies supposedly showed that murder rates tended to be lower in US states with the death penalty. Participants demonstrated an obvious bias in their evaluations. For example, if they supported capital punishment, they tended to evaluate the death penalty study favourably, whereas if they were against capital punishment, they were more likely to see the studies’ flaws. Scientific skills offer little protection against this bias, in fact they can compound it. A 2013 study asked participants to evaluate a piece of research on gun control. Participants with greater numeracy skills were especially biased: if the findings supported their existing beliefs, they were generous in their evaluation, but if the findings went against their beliefs, they used their skills to (in the words of Shah et al) “slam” the findings – a phenomenon dubbed “identity-protective cognition”.

We’re seduced by graphs, formulas and meaningless neuroscience
It doesn’t take a lot to dazzle the average newspaper or magazine reader using the superficial props of science, be that formulas, graphics or jargon. Consider a study due for publication soon (Ibrahim et al, cited in the new chapter): researchers asked their participants to consider a news story about a correlational study into genetically modified foods that was either consistent with the bulk of past research showing that they are safe, or was inconsistent, suggesting that they are harmful. Additionally, the story was either accompanied or not by a scatterplot of the new findings. When the news story had a graphic visualisation of the correlational evidence, which was inconsistent with the weight of past research (i.e. it implied a possibility of harm), participants were far more likely to interpret the new evidence as showing genetically modified foods cause harm, than if they had read the same story without a graphic. “This is especially worrisome,” write Shah et al, “since it demonstrates how easily people can be convinced by new data, regardless of the actual scientific merit of the result.” Similar research into readers’ critical skills has shown that they are blinded in a comparable manner by gratuitous neuroscience jargon and meaningless formulas.

Being smart isn’t enough
Even expert researchers suffer from the human foibles that undermine scientific thinking. Their critical faculties are contaminated by their agenda, by their ultimate motives for doing their experiments. This is why the open science revolution occurring in psychology is so important: when researchers make their methods and hypotheses transparent, and they pre-register their studies, it makes it less likely that they will be diverted, even corrupted by, confirmation bias (seeking out evidence to support their existing beliefs).

Take the example of systematic views in psychotherapy research: a recent analysis found that the conclusions of many are spun in a way that supports the researchers’ own biases. Other times, the whole scientific publishing community, from journals editors down to science journalists, seem to switch off their critical faculties because they happen to agree with the message to emerge from a piece of research.

In their chapter, Shah and her colleagues point out that raw cognitive ability (IQ) is not a good predictor of a person’s ability to think like a scientist. More relevant is mental attitude, such as a person’s “need for cognition” and their ability or motivation to override gut instinct and reflect deeply. On a positive note, these mental dispositions may be more malleable, that is more trainable, than basic intelligence. But we’ll need plenty of solid evidence to test that.

What Makes Everyday Scientific Reasoning So Challenging?

Image: Dr. Maria Goeppert Mayer of the University of California was named a co-winner of the 1963 Nobel Prize for Physics (Getty Images).

Christian Jarrett (@Psych_Writer) is Editor of BPS Research Digest

33 thoughts on “5 Reasons It’s So Hard To Think Like A Scientist”

  1. Man I love the digest, but this article reads like you’ve just entirely side stepped the science wars of the past half century.

    No discussion of subjective versus objective analysis; no mention of embodied cognition as an empirical diagnositc tool; blah blah blah..

    Not that the science you review isn’t fascinating, it’s just written for a public stuck in the 1950’s, when feelings were bad and logic was good.

    To catch everyone up to the 21st century, even many of those without degrees recognise that knowledge gained outside evidential hypothesis testing can be both reliable and valid. And you could even call that scientific.

    1. “Knowledge gained outside evidential hypothesis testing can be both reliable and valid” is true enough. If my thermometer says it’s 46C outside, it probably is – with no hypothesis required.

      But “reliable and valid” knowledge is only part of the scientific story. It is only by devising consistently accurate models that a scientific idea is sufficiently proven.

  2. Au contraire, it seems to me that the article is precisely about the issues you raise in your second sentence. I am sharing this summary and the original book chapter with many of the science educators I work with. They know this, but it is always a pleasure to get more evidence.

  3. Where to start? You use totally unscientific thinking to supposedly validate your point that people cant think like a scientist! You use the “fact” that overwhelming scientific consensus is that human activity has contributed to global warming (please, its poor sport to change the terms to “climate change” just because you’re losing, that global warming hasn’t occurred in 18 years!) In other words you are saying that it is ‘settled science: which is the antithesis of science! In astrophysics scientists actively try to disprove Einsteins theorem as a way to expand our knowledge, yet in global warming any dissent is shut down. If you really want to “think like a scientist” examine all history showing fluctuating temps much hotter and colder than now, discard all the lying scientists uncovered in the environmental computer hack like Mann and his totally disproved hockey stick, examine all the computer models which have never been right, discount all the charlatans on the dole for shilling man made global warming and then listen to the real scientists who say its bunk. Analyze both sides and come to the rational conclusion that many factors influence global temperatures (and those of other planets) far more than man – that’s how you think like a scientist!

    1. Very good comment. The article misses the whole point about science which is to try and falsify hypotheses (Thank you Karl Popper). If you can do it by experiment or observation great. Evolution is accepted because in Darwin’s great book Origin of the Species he spends nearly all the 576 pages trying to disprove the theory of evolution and finally gives up and says we have no choice but to accept it. That is what makes the Theory of Evolution acceptable not that 2/3rds of the population believe it.
      Ditto for the hypothesis of anthropogenic global warming due to increasing carbon dioxide – if there is evidence or observations that appear to disprove it, and there appears to be plenty the hypothesis should be rejected.

      1. I think you misrepresent Popper here. It is not that scientists are driven to disprove, but rather that a statement that cannot be disproved is not a scientific one, but rather one based more or less on belief. For example statement about non-natural phenomena by definition have no means for disproof in nature, so they are not acceptable scientific statements.
        We cannot underestimate the power of our own belief systems, and this article points that out very well. There is ample evidence in the physics education literature, for example, that we see what we believe, and not that we believe what we see. That is what makes helping students learn in the presence of their beliefs so difficult.

    2. Of course temperatures fluctuate. That’s not the point. The point is the overall trend not the fluctuations that take place in between. A business that’s losing money might have occasional windfalls and jumps that temporarily turn back or slow the losses, but the business is still losing money regardless.

  4. Very interesting article but one comment has me wondering…

    “We Brits are just as blinkered. In a recent survey, over 96 per cent of teachers here said they believed pupils learn better when taught via their preferred learning style, even though scientific support for the concept is virtually non-existent.”

    Perhaps 96 per cent of teachers are thinking like scientists, and hypothesising that the one-dimensional approach we currently have is not getting results. The UK ranks lowest of all developed nations for literacy and numeracy. Meanwhile, Finland has an education system that wholly values the individual and consistently outperforms every other country in the world.

    If you give me 100 schools in the UK, I’ll do the science to prove these teachers are right!

    1. “If you give me 100 schools in the UK, I’ll do the science to prove these teachers are right!”

      Scientifically speaking, wouldn’t that reflect small sample bias?

    2. Since we are thinking about thinking like a scientist, let me suggest that these statements are not thinking like a scientist, but rather speaking like a politician, since few are the words use have good operational definitions that we all agree on. That vagueness allows for all sorts of statements that should be challenged.
      The lack of a preferred learning style means that in different circumstances children learn in different ways, although there are some activities that clearly enhance learning such as active engagement and good feedback.
      I am not sure what you mean by a one-dimensional approach, that is open to interpretation and so not meaningful.
      Your assessment of education in Finland is to narrow. A key factor in the Finnish system is that teachers have significant planning time as individuals and in small groups, something that doesn’t exist in the US, and as far as I can tell in the UK.
      If you had 100 schools, depending on how they were chosen, you would likely have a biased study, so simply have 100 is not enough.

  5. It is odd that people are saying this article missed the mark. Each point seems valid with respects to checking our own biases and relying more on empiricism and critical thinking. For instance:

    “We’re swayed by anecdotes”

    This is basically be wary of relying on heuristics.

    “We’re overconfident”

    This is overconfidence effect.

    “Being smart isn’t enough”

    Potentially research bias.

    “We’re biased by our prior beliefs”

    This is confirmation bias.

    “We’re seduced by graphs, formulas and meaningless neuroscience”

    Misleading statistics and correlation doesn’t always equal causation.

  6. As this piece refers to THINKING like a scientist in the title rather than ACTING like a scientist can anyone provide a clear and also comprehensive description of how scientists think?

Comments are closed.