Joe Posnanski teams with Tom Tango to compare the fans scouting report on defense (the eyeball test) to the numbers computed by advanced defensive analysis:
So we compared them player for player from 2011-17. And in roughly 95 percent of the cases, the fans scouting report matches up stunningly well with DRS, UZR or both.
Let’s repeat that: The eye test and the defensive numbers almost always are very close. Tango and I looked at the years 2011-17 to get a larger sample size, and the agreement between eyes and digits was pretty staggering. We’ve been led to believe because of a few examples that the numbers and the eyes see defense in entirely different ways, and it just isn’t true.
It turns out that two players the fans overrate compared to the numbers are Mike Trout and Eric Hosmer:
Well, it could be the numbers are wrong. Hosmer’s greatest skill, by nearly unanimous opinion, is his hands. One theory is that he’s as good as anybody in baseball at saving infield errors by scooping bad throws out of the dirt — again and again people say that he saves 20 to 50 errors a year with the slickness of his glove — and the numbers don’t pick that up.
But is this true? Probably not. If Hosmer was really saving so many errors, wouldn’t this show up in his teammates’ defensive numbers? Wouldn’t we be able to see this in, say, a substantially lower error total for third baseman Mike Moustakas and shortstop Alcides Escobar? But it doesn’t seem like that’s true. Escobar finished tied for the most in errors in the AL in 2012, when Hosmer was 22, and you would think at the top of his defensive powers, and he’s been in the top five in errors two other times. Moustakas has been top five in errors as well.
And Hosmer has been top five in errors among first basemen five times. I don’t like errors as a statistic and feel lousy for using it, but errors are useful here because they are part of the eye test. People who want to say that Hosmer’s advanced numbers miss something have a harder time explaining why he makes quite a few errors.
The whole article is an interesting read.
I think a bigger factor for Hosmer and Trout may well be the teams positioning philosophy as well as pitching staffs command. As for errors, they are so subjective. I’d love to see some analysis on the various official scorers around the league. Unfortunately, for some reason, they refuse to give is H-A defensive splits so we can observe any home town biases (including stringers used for the advanced stuff) or lack thereof
You can be sure Padres had statcast data to analyze Hosmers defense
I know MGL did some calculations years ago and his conclusion was that great scooper may save 3-4 errors during the season, IIRC. However, great scooper may help the team in a way that is a bit harder to quantify:
I would expect that defenders throwing to a statue at 1B, where they know that they have to hit the glove, would make sure they are in a perfect position for straight throw. Throwing to great scooper, they would try to throw ASAP in close situations, knowing that they will get an out even if throw is of-line. So they will have more errors, but also more out in borderline situations. So, great scooper’s influence should be reflected in better defensive numbers of other infielders and better team DER. That may be the way to check if great scooper is contributing more than his numbers.
Pft » Back when I worked for STATS, Inc. (1990s), Craig Wright suggested we put up a report that showed some kind of home/road error break down. I think we did error rates in home games versus road games and/or errors rates for home teams versus visiting teams. Wright was well aware that error rates were highly influenced by the official scorer.
This seems more like herding to me than evidence that the eyeball test is accurate.
Fans vote on fangraphs. People who are on fangraphs are aware of stats. They internalize the stats and respond accordingly. Trout and others are overrated because they get more press coverage for their fielding.
It doesn’t seem like an eyeball test, unless the eyeballs you’re measuring are the eyeballs on the media coverage of baseball.
Here’s something on polling herding that is related: https://fivethirtyeight.com/features/heres-proof-some-pollsters-are-putting-a-thumb-on-the-scale/
Hans » Thanks, Hans, good point.
Thinking about it. It probably wouldn’t be too hard to test. Just regress the difference between the UZR rating and the fans rating on the amount of media coverage that a player gets (maybe even specifically looking at the coverage of webgems–Adam Jones’s WBC catch is an example– or the media coverage of defense), and see if coverage correlates to increases in ratings relative to UZR.
I’ve been getting that comment about “fans are aware of the stats” ever since I started this in 2003.
First off, except for the 2017 season, polling was done through my website.
Secondly, the way I set it up is based on voting for 7 distinct traits. That forces the voters to NOT give an “overall” score.
Thirdly, Fans voting correlates better with future UZR just current UZR on its own: meaning fans are picking up on things that UZR does not.