To mark 100 email issues of the Research Digest – the British Psychological Society’s free roundup of the world’s best new psychology research – and to inspire the next generation of researchers, I asked leading psychologists and bloggers to write about ‘The most important psychology experiment that’s Never been done.’
This feature is sponsored by the not-for-profit Centre for Applied Positive Psychology. CAPP Press, their publishing arm, is proud to announce the forthcoming publication of two titles in its “Strengthening the World” series. See www.cappeu.org.
I’d like to thank our contributors sincerely for taking their time to participate in this special feature and being prepared to put their ideas on the line.
We have 13 contributions in all, which have been published daily over the past fortnight.
Martin Seligman: “For one hundred years psychotherapy has been where you go to talk about your troubles. Looking over hundreds of controlled outcome studies, it is a moderately effective process. But does the troubles part matter?
We have recently been looking at a process—called Positive Psychotherapy—in which talking about what is good in your life is the central focus: strengths, virtues, flow, meaning, positive emotion, gratitude, hope and the like. We do not neglect troubles (depressed patients are socialized into the belief that troubles must be discussed and rapport would be undermined otherwise), but they are not the central focus and often form a segue into talking about strengths and meaning. Similarly trouble-focused psychotherapy does not wholly neglect the positive side of life, but damage and its repair are the central focus. Trouble-focused therapy, unlike strength-focus therapy, is not much fun (worse, sometimes patients unravel and cannot be ravelled up again), is stigmatizing, and has a considerable drop-out rate.
So let us finally test experimentally if it is troubles and repairing damage or building strength, meaning, and positive emotion that is the (more) active ingredient in psychotherapy: 200 depressed patients, randomly assigned to therapists, trained to deliver either trouble-focused or strength-focused psychotherapy. It could even be within subjects in an ABAB design. And how would these compare to medication or medication plus strength or trouble-focus?” —
Dr Martin Seligman is Leadership Professor of Psychology at the University of Pennsylvania. He has written over twenty books and two hundred articles on motivation and personality.
Jeremy Dean: “While the greatest psychology experiment imaginable has never been done, it has been filmed. The film is The Truman Show in which the main character Truman Burbank, played by Jim Carrey, lives in an entirely manufactured world, and has done since birth. The island on which he lives is a stage, his wife is an actress along with all his friends, neighbours and acquaintances – indeed everyone on the island is playing a part.
In the film, Truman’s every move is broadcast to an adoring global audience of millions – a run-of-the-mill Hollywood dream. But in the psychologist’s dream, Truman’s every move would be broadcast back to a waiting team of analysts.
All breeds of psychologists would be in on the act. Developmental psychologists examining how Truman changes over his life-span, social psychologists testing his obedience, conformity and social identity and cognitive psychologists checking his memory and attention. All in a controlled environment, and over a controlled lifespan.
The problem is that a sample size of one doesn’t play well in the academic journals. So more ‘participants’ would be required. Truman is soon joined by an ever-growing cast of participants. He would need a ‘real’ wife, ‘real’ children and ‘real’ in-laws. But as more participants are added, the environment becomes less controlled, more chaotic, open to the vagaries of human behaviour. All those nicely controlled experiments would start to break down as real participants interacted with each other in unexpected ways.
No, we need many Truman’s all in separate artificial environments – each subjected to slightly different environments and having slightly different genetic make-ups…
Right, I’m off to write my grant application. Do you think a few billion pounds ought to cover it?” —
Alex Haslam: “The invitation to design the ‘most important experiment that’s never been done’ is an interesting one, but one that entails a great many dangers. One of these is that by inviting researchers to focus on experiments per se, it encourages them to forget that the fundamental purpose of experiments is to test theoretically derived ideas. Really, then, the question should be ‘what is the most important theoretical idea that has never been empirically challenged or supported?’ An answer to this question needs to be informed by an analysis of both (a) ideas that dominate the field but which are misleading, and (b) ideas which offer an alternative, superior understanding. In my own field of social psychology, there are a great many candidates for (a), but relatively few for (b). In this regard, I would assert that the most important class of ideas that need to be challenged in our understanding of social life are those which lead us to believe that social problems encountered in the world at present (e.g. tyranny, prejudice, abuse, discrimination) are the ‘natural’ manifestation of inherent processes (e.g. evolutionary, socio-biological, or social psychological). The most important studies are therefore those which take influential studies that appear to support such ideas, and turn them on their heads.
In this regard, one study that I would like to conduct would involve using the paradigm of the Stanford Prison Experiment (SPE, Haney et al., 1973) as the basis for a study showing that, under certain conditions, people (e.g. prisoners) can resist oppression as well as commit and fall victim to it — thereby challenging the idea that tyranny is an inevitable consequence of assigning people to powerful and powerless roles. In fact we were given the opportunity to design and conduct just such a study several years ago (the BBC Prison Study — BPS; Reicher & Haslam, 2006). The findings to this both (a) challenged received models of tyranny, (b) supported an alternative analysis (derived from social identity theory; Tajfel & Turner, 1979), and (c) bore striking resemblance to patterns of resistance displayed in real-life prisons (and elsewhere in society).
The problem here, though, is that because our findings diverged markedly from those of Zimbardo, he (and others) assumed that the study must have had an inherent design flaw. Of course, as an experiment the study had certain limitations (e.g. lack of a control condition, small sample size) that it would be good to try to address in follow up research. Indeed, it would be great to conduct an experiment which resolved this debate conclusively. Such a study might involve two conditions with multiple prisons in each: in one, the prisons would recreate and replicate patterns of tyranny observed in the SPE; in the other a relevant manipulation would heighten shared social and political identification among the prisoners while weakening that of the guards in order to show that, over time, this was a basis for resistance of the form displayed in the BPS (and in prisons like Robben Island and the Maze; Buntman, 2001; McEvoy, 2001).
Here again, though (as we have found out), there is a danger in thinking that the resolution of such matters is only ever an empirical issue — a question of ethics, resources, and careful design. These things are important, but ideology, politics, group memberships and vanity also have a role to play. You can lead an experimentalist to data, but you can’t always make them think. The most important experiments are those which make such disengagement harder, and which encourage fresh minds to change the world not just reproduce it.
Dr Alex Haslam is Professor of Social and Organisational Psychology at the University of Exeter, UK.
Vaughan Bell: “In 1684, the famous writer, Nathaniel Lee, was becoming increasingly disturbed and was promptly admitted to Bethlem Hospital. While protesting his sanity, he described the situation as one where ‘they called me mad, and I called them mad, and damn them, they outvoted me.’ Over three hundred years later, the difficulty of agreeing on whether someone’s belief is a paranoid delusion, a sign of psychotic mental illness, is still troubling psychologists.
Delusions are broadly defined as false, fixed beliefs that are held despite obvious evidence to the contrary. Sometimes, it’s a clear-cut case. If someone believes they are dead, a condition known as Cotard’s Syndrome, you can be confident that the belief is a delusion. On other occasions (and these occasions are by far the most common) the question relies on a judgement of how well the evidence supports the person’s belief. This is where it gets tricky, because what counts as evidence, and what counts as ‘well supported’ are often a matter of opinion.
Someone goes to a mental health professional and says ‘I feel awful. I’m being targeted by my neighbours, and they’ve implanted microphones in my house to listen to my breathing.’ It certainly sounds strange, and maybe they’ve already been diagnosed with a mental illness in the past, so we might just think it sounds unlikely enough to count as a delusion. But we know, for example, that people diagnosed with schizophrenia are much more likely to suffer violence and discrimination. Perhaps, some of the paranoia is driven by genuine persecution.
So here’s the experiment. Everyone who walks into a mental health clinic is interviewed and their seemingly paranoid beliefs are noted. The mental health professional is asked to make a judgement on how delusional the belief might be. Then, each client is assigned a world-class private investigator, who is given the job of checking out all aspects of the belief, no matter how unusual. Is anyone in the neighbourhood persecuting the person? Are there really microphones in the house? Is there anyone who might have an interest in listening to their breathing patterns?
At the end, the professionals’ judgements are compared to the evidence from the investigation, and we get to see how good we are at distinguishing paranoia from realistic concerns. Just as importantly, the study would indicate where the borderlands of paranoia lie, giving us a better understanding of how the mind exaggerates our fears. Further research could look precisely at how genuine threats spark, ignite and become inflamed by the cognitive distortions of psychosis.
The experiment, of course, will never be run. Even ignoring the practical difficulties, it’s simply too intrusive and risks breaking client confidentiality. To do their job, the private investigators would have to ask questions which would give away personal details. Thankfully though, good mental health care focuses on psychological distress, no matter what causes it, but the issue raises the important question of how much we rely on guesswork to judge other people’s reality.” —
Annette Karmiloff-Smith: “Two lines of research have motivated my choice of a psychology experiment that has hitherto never been done. The first involves scientifically-controlled studies of the typically developing foetus during the last three months of intra-uterine life, pioneered in Belfast by Peter Hepper and his collaborators. Measurement of intra-uterine changes in rate of heart beat or of limb movement can be used to ascertain whether the foetus is sensitive to changes in auditory stimuli such as music, words, male/female voices. The second line of research comes from a PhD experiment in my lab (Paterson, etal., 1999) which indicated that infants and toddlers with Williams syndrome were as sensitive to changes in small numbers (1, 2, 3) as chronological-age-matched controls, whereas those with Down’s syndrome (DS) performed more poorly than both chronologically age-matched and mental-age-matched controls. How could these seemingly disparate lines of research meet?
The experiment I propose has three steps:
Step 1) Hitherto, work on the foetus has not involved number. Can the typically developing foetus notice changes in small number in the auditory domain (we know, for instance, that young infants can do cross-modal matching of small numbers from auditory to visual stimuli)? If we repeatedly tap two sounds until the foetus habituates, will the foetus renew heart-rate or limb movements when we change to three sounds (or from repeated three to two sounds)?
Step 2) I would test the DS foetus in the final 3 months of intra-uterine life for sensitivity to changes in auditory stimuli, as has already been carried out with the typically developing foetus (music, language, etc.). Then I would test the DS foetus for sensitivity to changes in small number discriminability, as in step 1.
Step 3) If some of the DS foetuses are sensitive to auditory changes in numerosity, and subsequently perform well in the visual number domain, whereas those who fail to react to changes during intra-uterine life also fail to notice change in the visual domain after birth, a training study for the DS foetus would be set up to ascertain whether this induces changes in the trajectory of number development after birth.
This experiment is important in my view because it really takes development seriously, i.e., that the roots of all cognitive development are in low-level mechanisms operative at the very outset of development (Karmiloff-Smith, 1992, 1998, 2007). There are, however, ethical issues involved. How would expectant parents of DS children react to a request to participate in research? In my view, parents who have decided to carry a Down syndrome foetus to term are likely to be happy to be involved in any research that might encourage their foetus to process auditory stimuli as a possible preparation for life outside the womb. Moreover, even if the training were not successful during intra-uterine life, the experiment might lay the foundations for postnatal research placing the focus on early training in the auditory and visual domains for both typically and atypically developing infants.” —
Dr Annette Karmiloff-Smith is Professorial Research Fellow in the Developmental Neurocognition Lab, Centre for Brain & Cognitive Development, Birkbeck College, University of London.
Chris Chatham: “The goal of developmental cognitive neuroscience is to uncover those mechanisms of change which allow the mature mind to emerge from the brain. The term encompasses a wide spectrum of research with one common fundamental assumption: the point of reference for this ‘mature mind’ is the healthy human adult, considered the apex of cognitive development.
Consideration of this assumption inspires a fascinating question. Are such mechanisms merely a kind of one-time developmental programme which self-terminates in the mature adult? Or, more likely, are they an emergent property of young neural networks? In other words, perhaps these developmental mechanisms are no more than a cascading reduction in entropy from the “blooming, buzzing confusion” of young, uncommitted networks to the specialized and entrenched patterns of connectivity in trained networks.
Current theories of lifespan trends in intelligence converge nicely with this idea. This work has shown that while measures of both fact knowledge and problem-solving trend together in early life, problem-solving and other forms of ‘fluid’ intelligence mysteriously plateau by early adulthood although fact knowledge continues its upward ascent. This apparent ‘crystallization’ of previously fluid intelligence may be a cognitive consequence of neuronal specialization. Perhaps, then, the adult mind is an artificial end point in the developmental process, enforced by limitations in neuronal real estate.
Imagine if these limitations were not present – what new vistas might the human mind reach? High-density intracranial electrode arrays, an invasive brain-computer interface technology, may someday progress to the point where they are not so quickly rejected by the body, but are rather greeted with the same axonal-dendritic kiss by which neurons embrace each other. And rather than interfacing the brain with simple computer programmes, as this technology is often used now, it might be used to interface the brain with massive biologically accurate computational models of the cortex.
Such a ‘computational extension system’ could conceivably be connected to nearly any cortical region, and become functional with exposure to adequately complex input. Extensions to sensory cortices might allow for more detailed perception of auditory or visual stimuli, improved spatial processing, or an exquisitely detailed kinaesthetic sense. The system might have its most extreme effects after bidirectional connection with prefrontal cortex and midbrain neuromodulatory systems, a network known to have undergone rapid expansion since homo sapiens diverged from other primates. Might a computational extension of prefrontal cortex affect cognition as profoundly as the evolutionary expansion of prefrontal cortex?
Though clearly plagued by ethical and practical issues, such an effort could have far-reaching consequences for our understanding of the mind. First, it would suggest that cognitive development is a process that unfolds naturally within uncommitted neuronal tissue. Second, it would allow unprecedented observation of the activity of such networks throughout their developmental cycle. Third, it may broaden our understanding of exactly how expansion of the brain – whether cultural, developmental, evolutionary, or computational – influences the emergence of mind.” —
Judith Rich Harris: “In a 1995 paper in Psychological Review, I proposed a new theory of child development, based on the idea that children’s personalities are shaped, not by their parents, but by the environment they encounter outside the home. This proposition, I said, doesn’t imply that children can get along without parents. What it does imply is “that children would develop into the same sort of adults if we left them in their homes, their schools, their neighbourhoods, and their cultural or subcultural groups, but switched all the parents around.”
It was a thought experiment – I wasn’t suggesting that parents should actually be switched around. What I was saying was that, given a child’s genetic makeup, and given the child’s environment outside the home, the environment provided by the parents inside the home would not have any noticeable impact on the child’s adult personality.
The experiment is an important one but it cannot be done, and not only for practical and ethical reasons. For one thing, there’s no control group. We’d need two identical universes so that we could switch the parents around in one and leave them in place in the other. Then we could compare the children in the two universes. We’d have to compare them one by one, because my prediction wasn’t about group averages – it was about individual differences. But that wouldn’t work either, because we already know that two children with identical genes and essentially identical outside-the-home environments – namely, reared-together identical twins – don’t end up with the same adult personalities. (The personality differences between reared-together identical twins is a mystery I address in my 2006 book, No Two Alike.)
There are ways to work around these problems and show that, given a child’s genetic makeup, and given the child’s outside-the-home environment, the environment provided by the parents inside the home makes no noticeable difference in the long run. But it involves putting together evidence from many different sources. This evidence already exists. For example, evidence exists that identical twins reared by different parents are (on average) as similar in personality as those reared by the same parents, and that adoptive siblings reared by the same parents are as dissimilar as those reared by different parents. Evidence exists that children reared by immigrant parents have the personality characteristics of the country they were reared in, rather than those of their parents’ native land. Evidence exists that environmental differences within the family, such as those associated with birth order, leave no long-term marks on children’s personalities. Even in childhood, firstborns do not behave differently from laterborns when they are outside the home, playing with their agemates.
Is it less convincing to put together many little bits of evidence (as I did in The Nurture Assumption and No Two Alike) than to point to a single grand experiment that proves one’s thesis conclusively? It certainly requires more patience from one’s audience. But sometimes the piecemeal approach is all that is possible.”
Scott Lilienfeld: “The most important psychology experiment that’s never been done would determine whether psychology can save the world.
Yes, that statement is admittedly more than a bit hyperbolic. And this experiment will probably never be conducted, at least not in our lifetimes or even our great-grandchildren’s lifetimes. But it is at least worth pondering as a Gedanken experiment. This experiment rests on three premises for which, I contend, there is substantial, although not yet definitive, support.
Premise #1: The greatest threat to the world is ideological fanaticism. By ideological fanaticism, I mean the unshakeable conviction that one’s belief system and that of other in-group members is always right and righteous, and that others’ belief systems are always wrong and wrongheaded – even to the point that others who hold them must be eliminated. Contra Hitchens (2007), religion per se is not a threat to the world, although certain religious beliefs can provide the scaffolding for ideological fanaticism, as we can see in the contemporary wave of Islamic extremism. As many historians have observed, the three most deadly political movements of the 20th century – Hitler’s Nazism, Mao Tse-Tung’s cultural revolution, and Pol Pot’s Khmer Rouge – were largely or entirely secular. What unites all of these movements, including Islamic extremism, is the deeply entrenched belief that one’s enemies are not merely misguided, but so profoundly misguided that they are wicked and must be liquidated.
Premise # 2. Biased thinking is a necessary, although not sufficient, condition for ideological fanaticism. Among the most malignant biases, and those most relevant to ideological fanaticism, are: (1) Naïve realism: the erroneous belief that the world is precisely as we see it (Ross & Ward, 1996). Naïve realism in turn often leads to the assumption that “because I perceive reality objectively, others who disagree with me must be foolish, irrational, or evil” (see Pronin, Puccio, & Ross, 2002); (2) Bias blind spot (“not me” bias): the erroneous belief that we are not biased, although others are (Pronin, Gilovich, & Ross, 2004); and (3) Confirmation bias: the tendency to selectively seek out information consistent with one’s beliefs and to ignore, minimize, or distort information that that is not (Nickerson, 1998).
Premise # 3: Critical thinking is the most effective (partial) antidote against ideological fanaticism. By critical thinking, I mean thinking designed to overcome one’s biases, especially the three aforementioned biases.
Regrettably, malignant biases in thinking are virtually never addressed explicitly or even implicitly in educational curricula, which is troubling given that so much of everyday life – left-wing political blogs, right-wing political talk radio, political book buying habits (Krebs, 2007), ad infinitum – reinforce them. Moreover, our selection of friends can generate not only communal reinforcement for our biases (Carroll, 2003), but the erroneous belief that our views are shared by most or all other reasonable people (i.e., a false consensus effect; Ross, Greene, & House, 1977). In some Islamic countries, of course, much of the educational curriculum comprises indoctrination into a cultural and religious worldview that implies that one’s enemies are mistaken, blasphemous, and despicable. In the United States, some social critics (e.g., Bloom, 1987; Horowitz, 2007) have charged that the higher educational system typically engenders an insidious indoctrination into left-wing ideology. The merits of these arguments aside, it is undeniable that even among highly educated individuals (a group that includes many or most terrorists; Sageman, 2004), the capacity to appreciate views other than one’s own is hardly normative.
So, the most important psychological experiment never done would (1) begin with the construction of a comprehensive evidence-based educational programme of debiasing children and adolescents in multiple countries against malignant biases, (2) randomly assign some students to receive this program and others to receive standard educational curricula, and (3) measure the long-term effects of this debiasing program on well-validated attitudinal and behavioural measures of ideological fanaticism. To some extent, the goal of this program would be to inculcate not merely knowledge but wisdom (Sternberg, 2001), particularly aspects of wisdom that necessitate an awareness of one’s biases and limitations, and the capacity to recognize the merits of differing viewpoints (e.g., Meacham, 1990 see p.181-211 here).
The greatest obstacle to conducting this experiment, aside from the sheer pragmatic difficulty of administering a large scale curriculum across multiple countries, is the surprising paucity of research on effective debiasing strategies. Nevertheless, at least some controlled research suggests that encouraging individuals to seriously entertain viewpoints other than their own (e.g., “considering the opposite”) can partly immunize them against confirmation bias and related biases (Kray & Galinsky, 2003; Wilson, Centerbar, & Brekke, 2002). Whether such educational debiasing efforts, implemented on a massive scale, would help to inoculate future generations against ideological fanaticism, is unknown. But launching such an endeavour by conducting small-scale pilot studies would seem to be a worthwhile starting point.” —
Richard Gregory: “Why is learning slow? Is it set by physiological (hardware) limitations, or is it due to cognitive (software) strategy?
As learning can be single-trial, in dangerous situations, the general slowness may well be a cognitive strategy – following Mill’s Methods for induction. Many instances are needed to establish that A is related to B. The more the noise (and other possibilities etc.), the more instances are needed, so learning should be slower.
The experiment would control environment ‘noise’, to see whether learning is generally faster in a simpler, more easily predictable world.
I tried this 50 years ago on Guppy fish, but they died! (The tank had metal sheets with many regular holes, giving moire patterns, which moved more or less consistently with the movements of the fish.)
If children are brought up in a simpler, more regular environment – do they learn faster, with fewer trials? Would this extend to any learning? How generally is the inductive strategy applied?” —