Edev_502 response wk7_1

[In response to: I have two questions about your post re: 1) your claim that the Dar-Nimrod and Heine (2006) is methodologically unsound 2) the benefits of the proliferation of categorizations you mention in your last paragraph.]

You ask some very critical questions. Your summary of the Dar-Nimrod and Heine’s article is spot on, and you’re right that it I shouldn’t have linked the meshing hypothesis to that. However, I will explain why I found that study methodologically unsound. I followed the links in the article to the numerical data that couldn’t be printed in Science. However, they provided very little that showed how they arrived at their conclusions. I’m sure that it wasn’t, but the data could have been made up. The same is true for Cohen et al.’s. Furthermore (and I do realise that this is not a technical point), the left-hand graph in D&H is suspicious. The ‘No-difference’ white bar is just over 5 (but which number exactly I don’t know) and the ‘Standard Stereotype Threat’ is just under 4. But the visual presentation of the differences between these numbers is suggestive of a much greater gap. The graph starts at 2, but the difference seems almost double. This sort of statistical trickery reduces the trust I put in the study.

Going a bit further into the what data was presented, we see that D&H’s Study 1 had no pre-test, 18 questions on the test and average scores (eyeballed from the graphs) between 3.8 and 5.1. Even if D&H’s claim of statistical significance is valid, the correct answer range is minimal and all scores could be due to chance if the test were an 18-question 4-item multiple-choice. They don’t specify what kind of test they conducted. In other words, Study 1 results are close to meaningless. Word limits stop me from describing D&H Study 2. All I can do here is to repeat my claim that the D&H paper is suspect.

This paper was on the reading list for this week possibly as a demonstration of how claims regarding intervention effects need to be viewed with a very narrow eye.


About theCaledonian

Scot living in north Japan teaching at a national university.
This entry was posted in EDEV_502, individual differences and tagged . Bookmark the permalink.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s