I just got back from the annual NCWIT Summit, in NYC. Carleton recently became a member of NCWIT’s Academic Alliance (yay!), and one of the perks responsibilities of membership is attending the annual summit. The summit is a chance for the members of all the alliances (academic, entrepeneur, workforce, K-12, etc) to get together, network, share best practices…and hear fabulous speakers.
This year one of the speakers was Joshua Aronson, who’s done a ton of excellent work on stereotype threat. (He and Claude Steele coauthored the original, seminal paper on stereotype threat.) The talk was excellent—he had a lot of great anecdotes (and is a good storyteller) and ended with some hopeful and promising ideas for mitigating stereotype threat.
However, he told one story that, frankly, still infuriates me.
In 2004, Stricker and Ward wrote a paper on a study sponsored by ETS, the company that is responsible for the AP exams. The study tested whether moving the usual questions on gender, race, etc. to the end of the AP AB Calculus exam, after the students finished answering the questions, would raise the scores of underrepresented groups on this exam, instead of asking these questions before the exam. The idea is that if stereotype threat was an issue, then moving the questions on race and gender to the end of the exam should have a positive effect on test scores of those most likely to be affected by stereotype threat. The study found no statistically significant difference in scores.
In 2008, Danaher and Crandall reexamined the study’s data, and found quite different results. They found that the criterion applied in the original analysis was too conservative. Changing the timing of the question in fact had a great effect on women’s performance, and that as a result 4700 more women would have earned scores high enough to earn placement credit for AB calculus.
Now, you would think that the ETS would have a vested interest in making these tests as fair and equitable as possible. After all, these tests are supposed to test knowledge about a subject, so why wouldn’t we want to make the test conditions as fair and unbiased as possible? So why not move the questions on demographics to the end of the exam, especially if there’s no good reason why they have to be asked at the beginning (and many reasons why they shouldn’t be asked at the beginning)? But even today, ETS has failed to make this simple change in the exam.
One of the messages/themes of the summit this year (and maybe in most years—this is only my second year attending) is “small changes count too”. One or more speakers explicitly mentioned that a small deed is better than no deed, or that small deeds start change, or some variation of this message. Changing the timing of the gender/race questions is a small deed. At worst, it has no effect on scores. At best, it can have an important and measurable effect on reducing stereotype threat.
So why not make the change, ETS? You’ve got so little to lose and so much to gain. As an organization invested in having more students participate in AP examinations and AP courses, why not also invest in a change that will possibly remove some subtle, unconscious, but real barriers to the demonstration of knowledge? What good is having more people at the table if those people aren’t performing up to their best level—particularly when you can make a change that might very well remove this performance barrier?
Why not, ETS?
References:
Steele, Claude M.; Aronson, Joshua (1995), Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology 69 (5): 797–811.
Stricker, L. J. and Ward, W. C. (2004), Stereotype Threat, Inquiring About Test Takers’ Ethnicity and Gender, and Standardized Test Performance. Journal of Applied Social Psychology, 34: 665–693.
Danaher, K. and Crandall, C. S. (2008), Stereotype Threat in Applied Settings Re-Examined. Journal of Applied Social Psychology, 38: 1639–1655.
This is what a computer scientist looks like 
I was at this meeting and was appalled the effects of the gender question placement . It’s unconscionable that ETS hasn’t made this seemingly-simple change. I spoke with the dinner audience afterwards, along with 2 other collegues on the Social Science Advisory Board, asking if anyone was interested in looking into why they’ve persisted. This seems like a perfect issue for NCWIT to get behind.
LikeLike