clock menu more-arrow no yes mobile

Filed under:

Some stuff about catcher framing

Another small, unexpected headache caused by 2020

MLB: Milwaukee Brewers at Pittsburgh Pirates Charles LeClaire-USA TODAY Sports

(Brief introductory note: this post is almost entirely about catcher framing, and is going to treat it as a thing. If you disagree with the premise that it exists, or the idea that it is possible to value framing even on pitches that don’t end a plate appearance [a concept I struggle with myself], there won’t be much here for you.)

Before the 2019 season, the Milwaukee Brewers took advantage of an oddly-dormant market for Yasmani Grandal, who had put up no less than 4.1 fWAR in each of the past four seasons, and signed him to a hefty one-year deal. Grandal rewarded them by being Grandal-esque (5.2 fWAR, featuring his usual combination of well-above-average hitting and great framing) and then hit free agency again, securing $73 million from the White Sox. As a replacement for 2020, the Brewers opted for someone that could perhaps replicate one part of Grandal’s production, if not the whole shebang: Omar Narvaez.

As Fangraphs’ Jake Mailhot wrote regarding the Narvaez acquisition, the former Mariner backstop could potentially give Milwaukee something like Grandal’s offense... but at a serious cost: Narvaez was an atrocious framer. The most generous assessment of Narvaez’ framing at the time was that he cost his team half a win with his lack-of-framing; more aggressive valuations had him losing more than a win relative to an average catcher in this department. Nor was this a one-time blip: the various framing metrics available indicated that over the past four seasons, Narvaez cost his team around six runs per season, framing-wise, with the two most recent seasons (2018-2019) being particularly awful. While Narvaez generally avoided the dubious honor of being the worst framer in baseball, that was about all you could say about that particular “skill” of his.

So, of course, fast-forward to the pandemic-shortened, everything-is-a-small-sample 2020 season, pull up the Baseball Savant catcher framing leaderboard, and uh...

Baseball Prospectus was generally harsher on Narvaez’ framing than other metrics. And yet...

What fueled this dramatic turnaround? Mailhot noted:

Framing is a skill that can be taught, and the Mariners worked with Narváez to develop his receiving. But the improvements weren’t enough to make him even an average receiver. Now the Brewers take up that mantle, hoping to continue his development behind the plate.

Framing improvements certainly do happen. Tyler Flowers went from a really good framer pre-2015 to a rabidly elite one afterwards, potentially due to his dedication to that particular craft. (As always, this Ben Lindbergh article is a must-read for framing-related stuff.) But, they can sometimes be hard to dissociate from noise — Chris Iannetta went from horrendous to great to horrendous to average across his last five full seasons, with different metrics resulting in different degrees of framing value whiplash. J.T. Realmuto is another complicated case of growth in this arena.

So, what’s the deal with Omar Narvaez? What’s the deal with James McCann, whom the Mets recently signed to a four-year deal? I can’t say I know the answer, but maybe you’ll find the below as interesting as I did.

If we’re going to be surprised or startled by rapid swings in a statistic across periods, we should first make sure that rapid swings are startling in the first place. One very basic way to do this (and before anyone gets any pretensions that the below is just a bunch of bamboozling, everything in this post is very basic) is just to look at year-to-year patterns in catcher framing value. Since we have multiple metrics, there’s no reason to look at just one. In particular, the metrics I looked at were the following:

  • Statcast/Baseball Savant: this is likely the most basic of the framing metrics. It simply calculates the rate of expected called strikes in each “edge zone,” compares that to the catcher’s rate of called strikes, figures out total strikes above/below average, and applies a run value to them. From what I can tell, nothing is controlled for — this is a purely mathematical calculation.
  • Jared Cross/Steamer, via Fangraphs: this is the value on the Fangraphs player pages, which is currently used as fWAR for catchers. (Orthogonal note: can someone get Fangraphs to add the other components of catcher defense back into fWAR? They don’t really matter much, but we’re now going on multiple years with them just disappearing without comment. Just use the DRS ones, they’re already right there. Or am I missing something?) You can read more about it here, but the idea is that this model uses some more sophistication by controlling for pitchers (and some other things).
  • Called Strikes Above Average, per Baseball Prospectus: arguably the big mamma-jamma of framing (they had it before it was cool), this metric controls for a bunch of stuff. Basically, in the realm of complexity. this metric sits on the throne.

Note: Sadly, Statcorner could not be included because Statcorner went offline for a while, and does not have 2020 data. I did not include the Sports Info Solutions (DRS) framing data, because documentation for it is sparse, and I couldn’t figure out where it fit into the above paradigm. I’ll leave replicating this analysis with rSZ as an exercise for some particularly adventurous, free-time-laden reader.

The actual year-to-year analysis is simple: grab every catcher with a decent sample of innings caught, calculate a per-inning rate of framing runs above average, and examine the relationship between Year 1 and Year 2 across the various framing metrics. In this case, I set a “decent sample of innings caught” to 650.

Note: Why 650? There are a few different reasons, including the fact that this cutoff usually gives you around 30 catchers per season, and the fact that the ratio of 650 innings to “the amount of innings regular catchers play” is similar to the ratio of 200 PAs to “the amount of PAs guys qualified for the batting title get,” which is an important cutoff because that’s where the median hitter approximates a league-average hitter — too high and you oversample better players, too low and you oversample fringy guys that are bad at most things.

Critically, the actual sample of “catcher-seasons with 650 innings caught in consecutive years” is really small. Across 2015-2019 (the starting point is because Statcast does not go back further than 2015), we’re talking just 71 catcher-seasons, or only around 18 catchers per set of paired seasons. Catcher turnover is real. While expanding the sample might be useful in some ways, it’s also at odds with our purpose here, which is to get a fairly robust framing value to compare year-to-year. We’re already basically letting half-time catchers into the sample, cutting it down further would make rates somewhat meaningless. Anyway, this is a lot of words, and all we care about here are some basic numbers.

Basically, what this tells us is that framing is at least somewhat “sticky,” as in, Year 1 framing informs Year 2 framing an okay amount. The left table can be read as “how much of the variation in Year 2 framing is explained by variation in Year 1 framing,” and it’s somewhere between one third and one half. One interesting but inconsequential thing to me is that the higher the value, the “better” the predicator is... and Year 1 Steamer is actually a better predictor of Year 2 Statcast/Baseball Savant and BP framing value than Year 1 of those respective metrics. The difference isn’t huge, but still. The rightmost table is just the coefficient or slope of the relationship between Year 1 and Year 2 — it’s positive and less than one, which is basically just a way of saying that if you want to know a guy’s Year 2 framing value, assuming it regresses to the mean around 40 percent is a safe bet. There are other things that drive the variation (one half to two thirds of it!), but it would be way weirder if the rightmost table was populated with anything other a positive value below one. Oh, and hey, the relationship generally looks pretty linear. Below is just Steamer Year 1 and Year 2, but you get the idea.

One very important thing that might be clear in the above chart, but also might not be: there are very few dots in the bottom-right and top-left quadrants! In fact, where they do occur, a lot hug at least one axis. It’s not that there are none, but fundamentally, this suggests something fairly obvious — good framers stay good, bad framers stay bad. Except when they don’t. Of the two dramatic swings from good to bad, one is Gary Sanchez in 2018-2019, who nearly missed being included in 2018. The other is Chris Iannetta in his perpetual framing weirdness. Of the three dramatic swings from bad to good, one is J.T. Realmuto 2017-2018, one is Mitch Garver with relatively small innings caught totals in both years, and the last is Tucker Barnhart, who seems like he might be Realmuto-esque in turning it around between 2018 and 2019.

One thing that’s really important to keep in mind is that the pattern in the scatterplot above is not an in-aggregate phenomenon where individual years look messy, but as you add more data, a pattern emerges. Instead, the individual years look pretty good:

While the 2018-2019 correlation is a bit weaker than the ones preceding it, they’re all in the same ballpark, and reasonably consistent with year-to-year correlations in other metrics. Where it completely falls apart, though, is 2020, which brings us partly full circle to the question from the beginning of the post. Here’s a visual illustration — the 2017-2018 and 2018-2019 data, small sample set of catcher-seasons and all, still kind of look like your expected, y=x function. 2019-2020, though...

If you’ve been following along, this might not feel like a particularly noteworthy outcome. Of course the correlation is going to be weaker for 2020, given that we had to relax our innings caught parameter given that the season relaxed its length. It’s debatable what we should have expected given said relaxation, and I don’t think anyone really knows the answer to that given that we’ve pretty much never had pandemic-cuts-season-to-a-third-of-its-length before (and hopefully never again...). But, I think it’s worth going a little deeper. (And if I didn’t, this post wouldn’t exist.)

Without spending a lot of words going over the method, what I did was calculate the residual for each catcher-season, based on the relationship modeled from 2015-2019 (without 2020 polluting the data). What’s a residual? It’s literally just the difference between what the model says a value should be, and what it actually is. (In other words, if I have a [stupid] model that says that every player whose first name starts with a “V” will earn $7 million per season in free agency, and Victor Von Doom secures a $11 million AAV deal, the residual in this stupid example for that one datapoint is -$4 million.) In particular, I focused on the residuals associated with the 2020 season, to see if there was any other explanation for the correlation being awful, beyond “welp, teeny sample.”

I won’t waste too many more words getting to the point. See the table below, and keep in mind that a positive number means “the player was better at framing than predicted by the model,” and a negative number means “the player was worse at framing than predicted by the model.”

Hey, look, there’s Omar Narvaez! The table is sorted such that catchers in the same division are grouped together. Sure, this is taking a very small sample of 20 “qualifying” catchers in 2019 and 2020, and dicing them into three divisions, but it’s at the very least suggestive, innit? Just in case averages are helpful:

  • AL/NL Central: Average catcher overshot the modeled framing value based on 2019 performance by 9.3/8.5/7.4 runs per 1,000 innings caught. Only Roberto Perez undershot his modeled value, and in only two of the three metrics.
  • AL/NL East: Average catcher undershot the modeled framing value by 5.4/2.9/3.3 runs per 1,000 innings caught. Of the nine catchers, three overshot by all three metrics, five undershot by all three metrics (by far more than the overshooters), and Gary Sanchez was weird.
  • AL/NL West, which sadly has only four catchers in this array: Average catcher undershot the modeled by framing value by 0.9/3.1/0.8 runs per 1,000 innings caught. Carson Kelly did way better than modeled, the other three did worse, especially Austin Hedges (who moved to the Central in the Mike Clevinger trade, but caught almost no innings in Cleveland).

Or, if you’re really keen on silly visuals:

The point is not that the small samples in 2020 didn’t hopelessly mess with this analysis. They did, and probably in an expected way. But this divisional breakdown, well, that could be a real (fairly minor, inconsequential) problem. (Note: the difference between non-Central 2019-2020 catchers having a model comparable to the other annual or total models is literally just whatever happened with Austin Hedges in 2020; if you remove him you can even eyeball a pretty obvious line through the other data points.)

At this point, you might be wondering, “Hey, wait, why are you grouping the AL and NL Central together?” In case you are, I won’t tease you: during the weirdness of the 2020 season, umpires did not cross divisions. In short: some chunk of this thing could be an umpire effect. Omar Narvaez, 2020 framing king? Maybe, Omar Narvaez, substantial beneficiary of specific umpires, who may have similarly aided James McCann and Willson Contreras. Turn the logic on its head, and you might get a partial explanation for Mike Zunino, Jorge Alfaro, Danny Jansen, and Yan Gomes. Those gains had to come at the expense of someone. Of course, none of this is anywhere near bulletproof. It’s small samples and only a few catchers. I didn’t bother torturing the data with actual regressions and dummy variables, game-by-game catcher comparisons with an umpire, and so on and so forth. One interesting thing is that I would have expected the more complex models (lookin’ at you, Baseball Prospectus) to be more resistant to this potential effect. That sort of appears to be the case, but the difference is small.

There’s a lot more that can be done with this starting point, but the small sample renders most of it somewhat moot. There just aren’t enough catchers with sufficient innings to run through this wringer. Still, in the future, if you’re scrolling through historical framing numbers and wondering “what happened to this guy in 2020?” maybe check his division first. That could be a part of the puzzle.

The Brewers acquired a bat-first Omar Narvaez. He didn’t hit at all, putting up a 59 wRC+, and got non-tendered. Still, his boatload of framing value gave him 0.5 fWAR (a 2.4/600 pace) on the year. The Brewers re-signed Narvaez in early December, but for a salary decrease relative to his (pre-pandemic prorated discount) 2020 guarantee. James McCann turned his great 2020 (144 wRC+, 1.5 fWAR in 111 PAs, which is a ridiculous 8.1/600) into a four-year deal with the Mets, despite that half-season-of-a-third-of-a-season being the first time he ever posted positive framing value, and him outhitting his xwOBA by over .040.

There probably isn’t too much to say about the Braves’ catchers in this regard; neither made the innings cut for this analysis. Travis d’Arnaud was a strong framer before his injuries, was average-to-below last year, and average-to-above this year. You could take his divisional status as a sign that he’s got some framing upside in a normal season, or it could all be noise. Tyler Flowers, now a free agent, posted his lowest rate-basis framing marks in years. While any prospective decision on whether to bring Flowers back for a last hurrah probably hinges less on him than on the other catchers in Atlanta’s system, this whole 2020-framing-division-thing might further complicate the calculus. It’s not high-stakes by any measure, but still, I don’t envy the folks tasked with wading through this to figure out how to estimate framing value for 2021 and beyond. Maybe, like most of 2020, it’s all just best forgotten. (Except for the Mets, who are now stuck with James McCann.)

Sign up for the newsletter Sign up for the Battery Power Daily Roundup newsletter!

A daily roundup of Atlanta Braves news from Battery Power