Category: Replications

Can excessive small talk really make us miserable?

GettyImages-693777214.jpgBy Emma Young

If you want to feel happier, avoid small talk and aim instead for profound conversations. That was the message the mainstream media took from a well-publicised paper published in Psychological Science in 2010 (e.g. Talk Deeply, Be Happy? asked the New York Times). But now an extension of that study, in press at the same journal (available as a pre-print), and involving two of the psychologists behind the original work, has found no evidence that how much – or little – time  you spend chatting about the weather or what you’re having for dinner will affect your life satisfaction. “The failure to replicate the original small talk effect is important as it has garnered considerable scientific and lay interest,” note the authors.

Continue reading “Can excessive small talk really make us miserable?”

“Strongest evidence yet” for ego depletion – the idea that self control is a limited resource

By Christian Jarrett

For years, “ego depletion” has been a dominant theory in the study of self control. This is the intuitive idea that self control or willpower is a limited resource, such that the more you use up in one situation, the less you have left over to deploy in another. It makes sense of the everyday experience of when you come home after a hard day at the office, abandon all constructive plans, and instead binge on snacks in front of the TV.

The trouble is, the theory has taken some hard knocks lately, including a failed joint replication attempt by 23 separate labs. Critics have pointed out that most supportive studies – and there are over 200 of them – are small and underpowered. A meta-analysis that corrected for a positive bias in the existing literature concluded that ego depletion is not real. A study in India – where there’s a cultural belief that exercising self-control is energising – even found evidence for “reverse ego depletion“.

It’s not easy to weigh the evidence for and against, but perhaps the science is tipping back in favour of ego depletion. Two new studies, made publicly available on the PsyArXiv preprint website, provide what the researchers at Texas A&M University, led by Katie Garrison, describe as “the strongest evidence yet of the ego depletion effect”.

Continue reading ““Strongest evidence yet” for ego depletion – the idea that self control is a limited resource”

Replication failure: No evidence that big smilers live longer

GettyImages-659908692.jpgBy Emma Young

What’s in a smile? According to a widely reported 2010 study of US major league baseball players, which we covered here at BPS Research Digest, one important answer is: an indication of how long the smiler will live.

By analysing official individual photos of players from the 1952 baseball season, and then looking at subsequent death records, Ernest Abel and Michael Kruger at Wayne State University, Detroit, concluded that players who’d smiled like they meant it – with full “Duchenne smiles“, which involve muscles around the eyes as well as the mouth – lived on average seven years longer than players who’d posed with less convincing grins.

The result was taken to support existing evidence that happier people tend to live longer. It also seemed to show that smiles in posed photos – even on just one occasion – are a fairly reliable signal of people’s underlying emotional disposition and therefore their likely longevity.

But a new replication and extension of the Baseball photo study has produced very different results. This is important, because the idea that happier people live longer is widely promoted, and has implications both for individuals and policy-makers.

Continue reading “Replication failure: No evidence that big smilers live longer”

We are witnessing a renaissance in psychology

The future looks bright

By Christian Jarrett

There’s been a lot of talk of the crisis in psychology. For decades, and often with the best of intentions, researchers have engaged in practices that have made it likely their results are “false positives” or not real. But that was in the past. The crisis is ending. “We do not call the rain that follows a long drought a ‘water crisis’,” write Leif Nelson at UC Berkeley and Joseph Simmons and Uri Simonsohn at the University of Pennsylvania. “We do not call sustained growth following a recession an ‘economic crisis'”.

In their paper, due for publication in the Annual Review of Psychology, the trio observe that had any psychologists been in hibernation for the last seven years, they would not recognise their field today. The full disclosure of methods and data, the pre-registration of studies, the publication of negative findings, and replication attempts – all of which help reduce risk of false positives – have increased immeasurably. “The improvements to our field have been dramatic,” they write. “This is psychology’s renaissance.”

As well giving the field of psychology a pep talk, their paper provides a useful review of how we got to this point, the reasons things are getting better, and the ongoing controversies.

Continue reading “We are witnessing a renaissance in psychology”

These nine cognitive psychology findings all passed a stringent test of their replicability

By Christian Jarrett

The failure to reproduce established psychology findings on renewed testing, including some famous effects, has been well-publicised and has led to talk of a crisis in the field. However, psychology is a vast topic and there’s a possibility that the findings from some sub-disciplines may be more robust than others, in the sense of replicating reliably, even in unfavourable circumstances, such as when the participants have been tested on the same effect before.

A new paper currently available as a preprint at PsyArXiv has tested whether this might be the case for nine key findings from cognitive psychology, related to perception, memory and learning. Rolf Zwaan at Erasumus University Rotterdam and his colleagues found that all nine effects replicated reliably. “These results represent good news for the field of psychology,” they said.

Continue reading “These nine cognitive psychology findings all passed a stringent test of their replicability”

Is something rotten in the state of social psychology? Part Two: digging through the past

Victorian lifeboat men rowing to rescue a stricken shipBy Alex Fradera

A new paper in the Journal of Personality and Social Psychology has taken a hard look at psychology’s crisis of replication and research quality and we’re covering its findings in two parts.

In Part One, published yesterday, we reported the views of active research psychologists on the state of their field, as surveyed by Matt Motyl and his colleagues at the University of Illinois at Chicago. Researchers reported a cautious optimism: research practices hadn’t been as bad as feared, and are in any case improving.

But is their optimism warranted? After all, several high-profile replication projects have found that, more often than not, re-running previously successful studies produces only null results. But defenders of the state of psychology argue that replications fail for many reasons, including defects in the reproduction and differences in samples, so the implications aren’t settled.

To get closer to the truth, Motyl’s team complemented their survey findings with a forensic analysis of published data, uncovering results that seem to bolster their optimistic position. In Part Two of our coverage, we look at these findings and why they’re already proving controversial.

Continue reading “Is something rotten in the state of social psychology? Part Two: digging through the past”

Was the “crisis” in social psychology really that bad? Have things improved? Part One: the researchers’ perspective

Antique illustration of By Alex Fradera

The field of social psychology is reeling from a series of crises that call into question the everyday scientific practices of its researchers. The fuse was lit by statistician John Ioannidis in 2005, in a review that outlined why, thanks particularly to what are now termed “questionable research practices” (QRPs), over half of all published research in social and medical sciences might be invalid. Kaboom. This shook a large swathe of science, but the fires continue to burn especially fiercely in the fields of social and personality psychology, which marshalled its response through a 2012 special issue in Perspectives on Psychological Science that brought these concerns fully out in the open, discussing replication failure, publication biases, and how to reshape incentives to improve the field. The fire flared up again in 2015 with the publication of Brian Nosek and the Open Science Collaboration’s high-profile attempt to replicate 100 studies in these fields, which succeeded in only 36 per cent of cases. Meanwhile, and to its credit, efforts to institute better safeguards like registered reports have gathered pace.

So how bad did things get, and have they really improved? A new article in pre-print at the Journal of Personality and Social Psychology tries to tackle the issue from two angles: first by asking active researchers what they think of the past and present state of their field, and how they now go about conducting psychology experiments, and second by analysing features of published research to estimate the prevalence of broken practices more objectively.

The paper comes from a large group of authors at the University of Illinois at Chicago under the guidance of Linda Skitka, a distinguished social psychologist who participated in the creation of the journal Social Psychological and Personality Science and who is on the editorial board of many more social psych journals, and led by Matt Motyl, a social and personality psychologist who has published with Nosek in the past, including on the issue of improving scientific practice.

Psychology research is the air that we breathe at the Digest, making it crucial that we understand its quality. So in this two-part series, we’re going to explore the issues raised in the University of Illinois at Chicago paper, to see if we can make sense of the state of social psychology, beginning in this post with the findings from Motyl et al’s survey of approximately 1,200 social and personality psychologists, from graduate students to full professors, mainly from the US, Europe and Australasia.

Continue reading “Was the “crisis” in social psychology really that bad? Have things improved? Part One: the researchers’ perspective”

New Milgram replication in Poland finds 90 per cent of participants willing to deliver highest shock

By guest blogger Ginny Smith

Fifty years ago, in Connecticut, a series of infamous experiments were taking place. The volunteers believed they were involved in an investigation into learning and memory, and that they would be administering shocks to a test subject whenever he answered questions incorrectly. But despite pretences, the scientist behind the research, Stanley Milgram, wasn’t actually interested in learning. The real topic of study? Obedience.

Milgram recorded how far his participants were willing to go when told to deliver larger and larger shocks. In one version of the study, 26 out of 40 participants continued to the highest shock level – two steps beyond the button labelled “Danger: severe shock”.

But this was 50 years ago – surely the same wouldn’t happen if the experiment were conducted today? That’s what a group of researchers from SWPS University of Social Sciences and Humanities in Poland aimed to find out, in a “partial replication” of Milgram published recently in Social Psychological and Personality Science.

Continue reading “New Milgram replication in Poland finds 90 per cent of participants willing to deliver highest shock”

Personality differences uncovered between students at different US universities

Students at more expensive institutions tended to score higher in trait Neuroticism 

By Christian Jarrett

Psychology is overly dependent on student samples, but on the plus side, you might assume that one advantage of comparing across student samples is that you can rule out the influence of complicating background factors, such as differences in average personality profile. In fact, writing in the Journal of Personality, a team of US researchers led by Katherine Corker at Kenyon College has challenged this assumption: their findings suggest that if you test a group of students at one university, it’s not safe to assume that their average personality profile will match that of a sample of students from a university elsewhere in the same country.

Continue reading “Personality differences uncovered between students at different US universities”

Replication success correlates with researcher expertise (but not for the reasons you might think)

Old businessman holding his glassesBy Christian Jarrett

During the ongoing “replication crisis” in psychology, in which new attempts to reproduce previously published results have frequently failed, a common claim by the authors of the original work has been that those attempting a replication have lacked sufficient experimental expertise. Part of their argument, as explained recently by Shane Bench and his colleagues in the Journal of Experimental Social Psychology, is that “just as master chess players and seasoned firefighters develop intuitive expertise that aids their decision making, seasoned experimenters may develop intuitive expertise that influences the ‘micro decisions’ they make about study selection … and data collection.”

To see if there really is any link between researcher expertise and the chances of replication success, Bench and his colleagues have analysed the results of the recent “Reproducibility Project” in which 270 psychologists attempted to replicate 100 previous studies, managing a success rate of less than 40 per cent. Bench’s team found that replication researcher team expertise, as measured by first and senior author’s number of prior publications, was indeed correlated with the size of effect obtained in the replication attempt, but there’s more to the story.

Continue reading “Replication success correlates with researcher expertise (but not for the reasons you might think)”