Will comments on my previous post and does some math to argue that the level of discrimination estimated in this paper is too small to fuss about. (Of course, I'm sure he would agree that a small amount of discrimination is worth stomping out if the cost is small enough so let me focus on the magnitude.)
The paper finds that a pitcher is 0.34 percentage points more likely to get a strike if the umpire is the same race. Yes, this is a small number. Relative to the probability of getting a strike when the umpire is a different race, the gain is a bit over 1%. Still somewhat small but I don't think that's quite the right comparison.
It's important to keep in mind that more often than not, the strike/ball call is obvious. How many pitches in a game are really "close calls" that "could go either way?" My point is that the small estimate is expected since there so few times in a game when the umpire really has an opportunity to discriminate. He's discriminating a lot more on the margin than he is on average.
I'm sure you get the picture but here's an example. Think about a team that only uses pitchers that match the race of the umpire. How much more likely are they to get a strike on marginal pitches? Assume 150 pitches and 20 close calls. Further, assume that the umpire only discriminates on the close calls.
Increased probability of strike = 0.34 =
(# close-calls / total pitches) * (increased prob. of strike on close calls)
+
(# obvious calls / total pitches) * (increased prob. of strike on obvious calls)
Filling in the knowns,
0.34= (20/150)*(increased probability of strike on close calls) + (130/150)*(0)
Solving, we get that the increased probability of a strike on close calls is 2.55 percentage points. If you think there are less close calls per game (which I think there probably are), then the impact on the margin is greater.
EDIT: It's important to note that the 0.34 estimate is barely statistically significant at the 5% level and that they haven't clustered their standard errors. They should probably be clustering on the umpire. And if they did, my guess is that the estimate would probably no longer be statistically significant. So, despite the fact that I think this magnitude can be meaningful, I'm with Will in the skeptics camp as far as the overall effect is concerned. But the non-questec stadium (where umpires aren't monitored so closely) results are quite compelling.
The paper finds that a pitcher is 0.34 percentage points more likely to get a strike if the umpire is the same race. Yes, this is a small number. Relative to the probability of getting a strike when the umpire is a different race, the gain is a bit over 1%. Still somewhat small but I don't think that's quite the right comparison.
It's important to keep in mind that more often than not, the strike/ball call is obvious. How many pitches in a game are really "close calls" that "could go either way?" My point is that the small estimate is expected since there so few times in a game when the umpire really has an opportunity to discriminate. He's discriminating a lot more on the margin than he is on average.
I'm sure you get the picture but here's an example. Think about a team that only uses pitchers that match the race of the umpire. How much more likely are they to get a strike on marginal pitches? Assume 150 pitches and 20 close calls. Further, assume that the umpire only discriminates on the close calls.
Increased probability of strike = 0.34 =
(# close-calls / total pitches) * (increased prob. of strike on close calls)
+
(# obvious calls / total pitches) * (increased prob. of strike on obvious calls)
Filling in the knowns,
0.34= (20/150)*(increased probability of strike on close calls) + (130/150)*(0)
Solving, we get that the increased probability of a strike on close calls is 2.55 percentage points. If you think there are less close calls per game (which I think there probably are), then the impact on the margin is greater.
EDIT: It's important to note that the 0.34 estimate is barely statistically significant at the 5% level and that they haven't clustered their standard errors. They should probably be clustering on the umpire. And if they did, my guess is that the estimate would probably no longer be statistically significant. So, despite the fact that I think this magnitude can be meaningful, I'm with Will in the skeptics camp as far as the overall effect is concerned. But the non-questec stadium (where umpires aren't monitored so closely) results are quite compelling.
8 comments:
RE: "he would agree that a small amount of discrimination is worth stomping out if the cost is small enough"
I don't think you can call such a small effect a bias. If I wasn't so lazy, I'd take their data and see if you can reject the null that length of surname (or some such inane thing) matters for called strikes.
In any case, here's data on pitch locations.
I'm easier to convince that a small effect can reflect a bias as long as the standard error is small enough. I just looked at their table and it's barely significant at the 5% level. Furthermore, they don't cluster their standard errors. They should probably do that on the umpires. And I have a feeling that doing so would make the standard errors increase enough so that it's no longer statistically significant.
But what about economic significance and all that?
Yes, that's where we disagree. As I explain in this post, I think the economic significance may be much greater than it initially appears.
Wait a minute... a slight change in standard errors gets you from "RACISM!!1!" to "ehh, there's no effect"?
You misrepresent my former position, sir, and my current one as well.
I never actually said my position since I hadn't read the paper in full. Having picked through sections of the paper, I still think there may be something there. The probable lack of statistical significance of the 0.34 percentage points we've discussed just means that there isn't convincing evidence of discrimination over all the games. At the same time, the evidence of discrimination in the non-Questec stadiums is about twice that magnitude and pretty convincing.
On that subject: Why didn't they look at the Questec data? Wouldn't that be an objective measure of balls and strikes to compare the subjective calls of the ump?
I wonder if all the Questec cameras were installed at the same time...
Absolutely. There'd be little disputing the results if they did that. Of course, I'm sure the reason they didn't is that they didn't have access to that data.
Post a Comment