There’s an article in the SMH this morning reporting that 53% of people have a ‘hidden bias’ against Arab Muslims. I’m more than a little suspicious of the methodology. The tests are based on the Project Implicit implicit association tests. These tests work as follows.

Assume we are trying to work out whether you have a bias towards group X, say the English, over group Y, say the Americans. Words from one of four groups are flashed on the screen:

Positive words: e.g. happy, laughter, joy
Negative words: e.g. awful, suffering, misery
Words associated with X: London, Shakespeare, Queen Elizabeth
Words associated with Y: New York, Ben Franklin, Hollywood

Your task is to group the words into one of two disjunctive categories. The test runs through twice. At first you have to group the words into X or good, on the one hand, or Y or bad, on the other. The second time through, you have to group the words into X or bad, on the one hand, or Y or good, on the other.

The idea is that if your reaction times are quicker (and/or you are more accurate) on the first part of the test, then you find the category X or good more natural than the category Y or good, which reveals a bias for Xs over Ys. On the other hand, if your reaction times are quicker, and/or you are more accurate, on the second part of the test, then for similar reasons you have a bias for Ys over Xs.

So I’ve taken these kinds of tests twice. I just got told I have a strong dislike for Arab Muslims. So I’m apparently one of the 53%. It doesn’t seem very plausible to me, but that’s what implicit preference tests are meant to show. Or maybe not. The other time I took the test I was told I have a bias in favour of the New York Yankees over the Arizona Diamondbacks. Now I have no particular fondness for the D’backs, but the Yankees are one of my few outright bigotries. I think “Yankees Suck” should be added to the Pledge of Allegiance. I’d rather see the Taliban being held in Cuba receive constitutional protection than the damned Yankees fans. Are these conscious reactions just a repression of a deep fondness for all things pinstriped?

Unlikely. What happened in both cases was that my reaction times, and accuracies, were higher in the second part of the test than in the first. I was just getting much better at disjunctive classification through doing the test. So in each case I was listed as having a preference for Ys over Xs. But it really was totally independent of just what Xs and Ys were. I presume this kind of consideration has been factored into the test design, but at least in my case (going by a massive N=2 sample) it looks to have not been given sufficient weight. So I’d be more than a little sceptical of newspaper articles reporting doom and gloom based on a few not necessarily well calibrated internet tests.