Intelligence tests are meant to tell you something about a person’s inherent abilities. But what if the results are distorted by the motivation to perform well? That would undermine the tests’ validity and have important implications for their use in education and recruitment.
A simple way to find out whether motivation affects intelligence test performance is to offer people a financial incentive for doing well and see if this helps them get a higher score. A paper in the British Journal of Psychology has done this, finding that while a financial incentive boosted people’s self-declared effort levels, it failed to lift their performance. This result is good news for the validity of intelligence testing. Or as the paper’s author, Gilles Gignac at the University of Western Australia, put it: “The position of a causal effect of test-taking motivation on intelligence test performance in adults does not, yet, appear to be clearly tenable”.
The background to this research is a controversial meta-analysis published in 2011 by Angela Duckworth (of Grit fame) and her colleagues which suggested that motivation distorts IQ test performance, thereby challenging the validity of IQ tests as a pure measure of ability. However, only 2 of the 46 samples included in that meta-analysis featured adults, and the findings from those adult samples were inconclusive.
To plug this gap, Gignac recruited 99 undergrads and asked them to complete various intelligence tests, including of basic “processing speed”, working memory, fluid intelligence (measured using Raven’s progressive matrices) and crystallised intelligence (based on a vocab test).
Gignac split the tests in half so that the participants completed two very similar testing sessions in succession: one without any financial incentive and one with the chance to win $75 if they were among the top 10 per cent of performers (some participants did the session with this incentive first, others did it second). After each session, the participants completed a survey about how much effort they had put in and how important they thought the tests were.
The financial incentive didn’t affect the participants’ perception of the importance of the tests. However it affected their effort levels, at least according to their self-reports. Those participants who had the financial incentive session in their first session subsequently reported less effort in their second session. In contrast, those who had the financial incentive in the second session managed to sustain their effort levels. “It may be claimed with some confidence that financial incentives can, on average, increase the amount of effort testees apply to complete cognitive tasks,” Gignac said.
Crucially, however, this effect on effort did not translate into any difference in performance. Participants’ actual performance on all the various tests was the same across both sessions, suggesting that the financial incentive failed to boost their results.
Many prior studies, in children and adults, have found that people’s self-reported effort correlates with their IQ. That is, people who are more intelligent – at least according to their higher test results – tend to say that they tried harder. Of course this doesn’t mean that greater effort is causing the superior performance. And the new results are among the first in adults to suggest strongly that it does not.
Although this study makes a useful contribution, it doesn’t settle the matter. Remember the financial incentive did not affect participants’ belief in the importance of the tests. Perhaps a bigger or different kind of incentive may affect test performance, especially if it manages to harness people’s intrinsic motivation by making them think it is very important to do well (for instance, imagine feeling highly motivated after being told that your test performance would be used to rate the ability of your country or your university). Another thing is that the participants in the new study all scored fairly high in terms of their effort levels, even without financial incentive. It remains possible that financial incentives may affect intelligence test performance but only among people who have very low motivation to start with.
For now, though, it is safe to say there is little evidence that differences in motivation undermine the validity of intelligence tests taken by adults. However, Gignac concluded, “as absence of evidence is not evidence of absence, we encourage more research to help understand precisely why test-taking motivation and intelligence test scores are correlated positively in both children and adults.”