That white firefighters case.....

[nancylebov]

Does anyone have details about the test which the white firefighters scored so much better on?

I can't imagine what would be on a test which legitimately produced such results, so I'm wondering if there was dishonest scoring.

Comments

[cakmpls.livejournal.com]

Given that the entity that gave the test, the city, was so unhappy with the result that they threw the whole thing out, I kind of doubt that there was dishonest scoring.

Do you know how many people took the test? If it was a fairly small number, there's no particular reason (that I can see) that the distribution couldn't have just happened to turn out this way. The larger the number who took the test, the more likely there was some kind of bias.

[avram]

Also, if you give a test to hundreds of mixed groups, you'll get a few cases where all members of one group do better than the other, even if there isn't any overall tendency for members of one group to do better in the society as a whole.

[nancylebov.livejournal.com]

That's another plausible explanation, probably as plausible as bad scoring.

[subnumine.livejournal.com]

New Haven asked that the original Federal court decision be sealed; there's probably something embarassing.

[cakmpls.livejournal.com]

It's possible; it's also possible that it had the individuals' names in it.

[madfilkentist]

There are often significant educational disparities between different ethnic groups within a local area. When this happens, it's quite easy for members of one group to get better scores than another.

After beating racial discrimination in court, why do these people now have to be subjected to groundless accusations of cheating?

[redaxe.livejournal.com]

The more typical reason for test results to have been biased in terms of race/ethnicity in the past is that the test itself contained social/ethnic biases. I don't know that this was the case in Connecticut, but I'd find it easier to believe than I would believe dishonest scoring.

I must go read the opinion at SCOTUSBlog, but that will wait until later.

[avram]

According to Wikipedia, 118 firefighters took the exams. New Haven's city charter has a rule called the "Rule of Three", which states that an open civil service position be filled with one of the three people who scored highest on the qualifying exam. There were seven vacancies for the Captain position, so those would be filled from the top nine scorers, all of whom were white or Hispanic. For the Lieutenant exam, there were eight vacancies, and the ten top scorers were all white.

[cakmpls.livejournal.com]

That entry says further that the Captain exam was taken by 25 whites, 8 blacks, and 8 Hispanics. The top 9 scorers were 7 whites and 2 Hispanics. That means that 28% of whites were in the top scorers, 25% of Hispanics, and 0% of blacks. Given the small numbers who took the test, that doesn't look particularly suspicious to me. 18 whites were NOT in the top ten--more than the total of blacks and Hispanics (14) who were not.

The Lieutenant exam was taken by 43 whites, 19 blacks, and 15 Hispanics. 23% of the whites were in the top ten scorers, and 0% of the blacks and Hispanics. That's somewhat more skewed, but the numbers are still pretty small for drawing any conclusions. 33 whites were NOT in the top ten--and a total of 34 blacks and Hispanics were not.

[nancylebov.livejournal.com]

Is there a statistician in the house? Those results don't look terribly likely to me, but I don't know what the odds are of results like that happening by chance.

[cakmpls.livejournal.com]

I don't think there is any way of determining "odds" for something like this. It isn't a case of drawing lots out of a hat.

On the Captain's test, we're looking at 9 top scorers out of 41, so we're looking for the top 22% (rounded). More than 22% of the white testees scored in that group, but so did more than 22% of the Hispanic testees.

If the spread had been perfectly proportional by race/ethnicity, 4.5 whites, 1.8 blacks, and 1.8 Hispanics would have been in the top 9, compared to 7, 0, and 2 actual. That doesn't seem particularly unlikely to me.

[nancylebov.livejournal.com]

Thanks. The results probably weren't as unlikely as I thought.

[bradhicks.livejournal.com]

For what it's worth, it's also been a consistent result of multiple-choice tests in general, even when comparing two groups of equivalent educational and economic background. For reasons not even vaguely well known, African Americans do much worse on multiple-choice tests than Americans of European or Asian descent do. There are some hypotheses, having to do with dialect, especially vocabulary and grammar rules differences between the people writing the test (who are, unsurprisingly, nearly all white) and those taking the test.

From 1971 until today, to defend a test that had that disparate an impact, you would have had to prove that answering multiple-choice questions was a bona fide occupational qualification (BFOQ) for the job, that the person who got the job was going to have to spend all or part of his day filling out multiple-choice tests. An overtly silly and indefensible argument, when it comes to fire-fighters.

But today's particular big of legislation from the bench (and you can't call it anything else, I'm sorry) is entirely consistent with Antonin Scalia's long-held political philosophy on discrimination cases, which is that no matter what Congress says, and no matter what the results are, the only actionable discrimination is overt, explicit, and intentional discrimination, and no court should accept anything less than the most explicit and compelling evidence with regard to intent or state of mind. For example, the only reason that the Santerians won in Hialeah versus Santeria was that the Hialeah city council's own minutes showed that the intent of the ordinance was to close down a Santeria church. If they had managed to hide this intent from the evidentiary record, they would have won.

[nancylebov.livejournal.com]

http://ta-nehisicoates.theatlantic.com/archives/2009/06/thoughts_on_the_ricci_case.php#comment-217345

The study materials cost $500, but people from fire-fighter families from the area would already have had them. And in a couple of cases, it took a month and a half for the study materials to arrive. So there was some de facto bias there.

I wonder what would have happened if the city had compensated everyone who'd taken the test for their time (an estimate of study time, not just the time spent taking the test) and money.

[cakmpls.livejournal.com]

On the Lieutenant's test, we're looking at picking the ten top scorers out of 77 candidates. That's the top 13% (rounded). Perfectly proportional, it would include 5.59 whites, 2.5 blacks, and 1.9 Hispanics; actual was 10, 0, and 0.

[subnumine.livejournal.com]

Not particularly difficult; the real difficulty is deciding on the exact question being tested (does it matter that some Hispanics got into the range eligible for promotion). The odds are on the order of 1000 to one against, assuming everybody had an equal chance; but if one assumes that there are some 200 similar exams out there, a thousand to one shot will show up about a sixth of the time. (And no statistics of the exam can tell you whether the inequality was the test scoring, the study kits, or the New Haven educational system.)

[doomspark.livejournal.com]

I remember when I was in the military, the unwritten rule for promotion was that if there were two slots open, one white would be promoted and one non-white - regardless of actual number of promotion points held by the candidates.

[(Anonymous)]

It's easy to find sample questions for the SATs, which produce similar results.