The data confirmed your expectations? That’s great, you were right from the beginning – or where you tricked by what is called confirmation bias – you only saw what you wanted to see? Although the effect is typically strongest with emotionally charged subjects (think gun control), it can be a serious problem in scientific experiments as well.
Ellen van Wilgenburg and Mark Elgar recently published a great article on the subject. They conducted a meta-analysis of behavioural experiments about nestmate recognition in ants. During these experiments, ants were encountered with other ants who were either nestmates or non-nestmates. In such situations, nestmates are typically treated peacefully while non-nestmates are attacked.
The authors found a clear difference between studies where the experimenter knew whether the ants they observed were nestmates or non-nestmates, and other studies where the experimenter did not have this information, i.e. was blind to the origin of the ants. Studies with blind experiments were more likely to find aggression between nestmates, and the difference between nestmate and non-nestmate aggression was lower in blind than in non-blind experiments. Apparently, researchers are more likely to score a behaviour as aggressive when they expect aggression, and tend to “overlook” aggression when no aggression is expected.
It may seem odd to you to find aggression between nestmates, because if that occurred in nature, life in ant colonies would be quite chaotic. However, as experiments rarely track natural conditions very closely and the ants are under some stress during the tests, recognition errors may occur.
Isn’t this study by van Wilgenburg and Elgar pretty much what many of us who routinely conduct behavioural experiments always thought someone should do? So read it, give it to your students, and do your experiments blindly!
van Wilgenburg E, Elgar MA (2013) Confirmation Bias in Studies of Nestmate Recognition: A Cautionary Note for Research into the Behaviour of Animals. PLoS ONE 8(1): e53548. doi:10.1371/journal.pone.0053548