Psychology is overly dependent on student samples, but on the plus side, you might assume that one advantage of comparing across student samples is that you can rule out the influence of complicating background factors, such as differences in average personality profile. In fact, writing in the Journal of Personality, a team of US researchers led by Katherine Corker at Kenyon College has challenged this assumption: their findings suggest that if you test a group of students at one university, it’s not safe to assume that their average personality profile will match that of a sample of students from a university elsewhere in the same country.
Corker and her colleagues measured the personality of over 8,500 students studying a range of majors (including psychology, business and nutrition) at 30 colleges and universities across 20 different US states. They found some significant differences in average student personality between different sites. The amount of difference that was explained by the site of testing was modest – about 1 to 3 per cent – but Corker and her colleagues said that this “should not be dismissed as necessarily trivial or unimportant”.
Among the site-specific effects, larger universities tended to have more extraverted students; more urban and diverse universities had more open-minded students; universities requiring letters of recommendation had more agreeable students; public colleges had less agreeable students than private colleges; and more expensive colleges had higher trait Neuroticism. Differences like these could reflect students with particular personality profiles being drawn to particular institutions; selection could be at play, in the sense of university selectors showing a preference for particular personality types; and also students’ personalities could be shaped by the culture of their university.
“All told, these results suggest there is more variability between students at different colleges and universities than some researchers might have expected,” Corker and her team said, though they warned their colleagues not to leap to this finding as an explanation for the replication crisis in psychology (the difficulty labs often have in trying reproduce earlier findings reported by researchers based at another institution), at least not until there is more data and a thorough theory in place to account for the site differences in personality.
It’s a shame the current research didn’t include students studying a more diverse set of subjects, but if anything, this makes it likely the observed between-institution differences are an underestimate. Perhaps for now the lesson to take is that if you’re comparing data from two sets of students at different institutions, it’s unwise to be overconfident about how similar they are likely to be.