2019 Atlanta Braves projections retrospective: pitchers

This is a hard post to write. The whole idea of a projections retrospective is that you take preseason projections, look at actual season performance, compare the two, bingo bango voila etc. That all sounds good in theory, but it really only works if the projection and the performance are more or less on the same basis. In past years, this hasn’t been much of an issue: while there was uncertainty or fluidity in the cast of characters the Braves could use to start and pitch the bulk of baseball games, the usage of said characters didn’t move around very much. In 2019, though, all of this went out the window.

The Braves used 11 starters in 2019. Of these 11, six had long stints, and five were fill-ins or quickly lost their chances to start additional games. Of the 11 starters, six also made at least one relief appearance. Basically, you can kinda/sorta group the 2019 Braves starters or potential starters this way:

• Projected as a starter, was a starter exclusively: Julio Teheran, Mike Soroka, Mike Foltynewicz, Kolby Allard (for another team)
• Projected as a starter, both started and relieved: Sean Newcomb, Touki Toussaint
• Projected as both a starter and reliever, both started and relieved: Max Fried (mostly started though), Bryse Wilson, Kevin Gausman (though all relief appearances came with not-the-Braves)
• Projected as both a starter and reliever, only relieved: Wes Parsons
• Enigma/no projections: Kyle Wright (very low track record in professional ball); Dallas Keuchel (joined team midseason)
• Got one courtesy start, actually a reliever for 2019: Josh Tomlin

That’s a big ol’ mess, and untangling it is pretty annoying.

Here’s the thing, though: in the end, despite not only the roles, but the projections themselves, being almost entirely all over the place and not particularly accurate on a player-by-player basis... in aggregate, everything worked out basically fine.

Before the season, the Atlanta Braves rotation was estimated to contribute between 9 and 11 fWAR under two different playing time paradigms, one which featured 936 total innings across 13 total starters, and another which featured 889 innings across six total starters. (The former featured Foltynewicz, Newcomb, Gausman, and Teheran with full-ish seasons, and then nine pitchers filling in the fifth starter spot and in case of injury; the latter featured those same four getting most starts but a fifth starter role split mostly between Soroka and Toussaint.) What actually happened was that the Braves got 873 innings out of their starters, totaling 11.3 fWAR. On a per-the-actual-873-innings basis, just for an apples-to-apples comparison, the projections were 8.5 or 8.6 (Steamer), 9.0 or 9.3 (ZiPS), and 10.0 or 10.3 (IWAG). The Braves finished at 11.3 instead, a swing of something like one to three wins, largely driven by Mike Soroka getting way more innings than forecasted. In other words, even with all of the variation described below, the rotation ended up more or less as expected, albeit a little better. Even dramatic changes like adding a whole new pitcher (Dallas Keuchel) didn’t move the needle too much.

But, the individual projections... oh boy. What a crazy mishmash. Basically nothing worked out as expected at the micro level, even though it was pretty much all fine at the macro level. Let’s take a look.

Mike Soroka

• Expectations: 90 FIP- to 101 FIP-; 2.4 to 3.6 fWAR/200; 64 to 106 IP
• Actuals: 79 FIP-; 4.6 fWAR/200; 174.7 IP

Mike Soroka posed two projection challenges before the season, at least for me and IWAG. The first was that his minor league track record was so good that the projections he was getting were initially nonsensical. I ended up having to tear out the guts of the way IWAG handled prospect pitchers and rebuild it to taper down wildly exciting outcomes with no precedent. The original IWAG prospect pitcher module saw Soroka pitching like a 5.2 fWAR/200 guy, because he was so dominant at every level. That didn’t really make sense as a central estimate (i.e., in 50 percent of iterations he was better than that) and it had to be tamped down accordingly. So, of course, what Soroka did in his rookie season was put up 4.6 fWAR/200, closer to the original “oh boy gotta deal with this no way it’s right” estimate than the “revised, more reasonable” IWAG estimate of 3.6 fWAR/200. Welp.

The second issue was a matter of health. At the time the projections were being compiled, Soroka was still questionable to be a major contributor due to concerns about his shoulder. As a result, the projection systems saw him pitching only a modest amount. Once again, though, Soroka shoved all of this aside and ended up tying Julio Teheran for the most innings hurled by a Brave in 2019. Only 40 pitchers threw more frames than he did in 2019, which makes the hand-wringing over just how little of a partial season he’d throw look silly in retrospect (though it was a real concern at the time). His superior performance, along with his ability to stay on the field, is the biggest reason why the Braves’ rotation ended up not just on par with, but a bit ahead, of its projected production totals.

As far as individual projections go, Steamer saw Soroka as a league-average arm (101 FIP-, 2.4/200), while ZiPS and better-calibrated-IWAG gave him 90 FIP-, 3.5ish/200 marks. Soroka, of course, was better than all of this. He cleared both his most optimistic fWAR and fWAR/200 projections by about a win.

IWAG distribution commentary: notably, IWAG saw that Soroka had something like a 20 percent chance (80th percentile outcome) of clocking his 79 FIP- and 4.6 fWAR/200. The real difference was that IWAG didn’t foresee him actually getting the innings to finish with 4 fWAR or anywhere particularly close. With a healthy season behind him, he could get some real-gaudy projections going forward.

Julio Teheran

• Expectations: 112 to 120 FIP-; 0.5 to 1.3 fWAR/200; 159 to 183 IP
• Actuals: 106 FIP-; 1.8 fWAR/200; 174.7 IP

The good news: Julio Teheran did way better than any projection system foresaw (though IWAG wasn’t that far off with 106 FIP- and 1.3/200 marks). The bad news: he still clocked in with a below-average FIP. Teheran’s improvement relative to projections was based almost entirely on his bewildering eight-start stretch in May and June where he allowed zero homers. After that stretch, his FIP- was 115; before it, it was 119. It was those eight starts with no homers, all bunched together, that made the difference.

It’s a notable commentary on the wildness of the projections relative to actuals that Teheran actually aligned more closely to his projections than the other Atlanta starters — his average error on fWAR/200 was a win while his average error across the innings he actually pitched was around 0.8 fWAR — compared to the other starters discussed here, that’s fairly small. Which, again, no excuses: the projections kinda ended up like poorly-thrown darts with respect to the Braves pitchers this year. Some were off the board and stuck on the wall to the left, some were off the board and stuck on the wall to the right, so on average, they hit the target. But they also ruined the wall in your den. Sorry.

IWAG distribution commentary: Teheran was far off from his point estimates, but IWAG didn’t see him pulling out the type of season he did as particularly strange. It was just something like a 70th percentile outcome. Again, even though he did better than his central estimate (and much better than forecasted by Steamer or ZiPS), he still wasn’t good.

Max Fried

• Expectations (as swingman, Steamer): 91 FIP-; 2.4 fWAR/200; 33 IP
• Expectations (as starter, ZiPS and IWAG): 111 to 117 FIP-; 0.9 to 1.3 fWAR/200; 105 to 111 IP
• Actuals: 30 starts, 3 relief appearances, 85 FIP-; 3.6 fWAR/200; 165.7 IP

Max Fried’s projection retrospective reminds me of the post-credits tag to Thor: Ragnarok. “And... uh... it’s a tie.” Steamer set forth with Fried getting only a few starts and relief appearances, but being super-good in them. ZiPS and IWAG saw him as a half-season starter, but with very limited effectiveness.

None of these things happened. Fried made only three relief appearances all season, working as a starter most of the year, and flat-out thriving. That he was better in a starting role than Steamer projected him in short stints says pretty much everything. It’s hard to belabor this point, though it’s very much worth belaboring: Max Fried kinda came outta nowhere to do what he did, but do it he did, and it was sweet, if very unexpected. I, for one, welcome our new Max Fried overlords.

IWAG distribution commentary: Again, not much to say... though it belies the ridiculousness of his performance. IWAG didn’t see this coming in any way, shape or form. To go from a guy who had trouble cracking a major league roster, with good but not great, fairly limited reps in the high minors, to a top-50 starter (and even better by many measures) in the majors is wild. IWAG did not see it coming, and gave basically no thought to the possibility.

Mike Foltynewicz

• Expectations: 94 to 100 FIP-; 2.6 to 3.2 fWAR/200; 166 to 192 IP
• Actuals: 114 FIP-, 1.4 fWAR/200; 117 IP

The first three pitchers discussed here did better than expected. Mike Foltynewicz was, given Soroka’s injury history, expected to be the Braves’ best pitcher on an aggregate basis. Instead, he had somewhat of a nightmare season and ended up undershooting his projections by as much as any Atlanta hurler in 2019.

At worst (ZiPS), Foltynewicz was projected to be a league-average starter, which over a projected three-fourths of a season meant 2-WAR production, and crept closer to a three-win mark with more playing time. Instead, the big right-hander combined a gruesome first half with a post-demotion rebound to pitch more like a fourth starter for half of the season instead. This deviation was part of the offset to the too-low whiffs by the projections on the other three rotation mainstays; had Foltynewicz not faltered, the Braves’ rotation would have been very strong, with three guys throwing three-win seasons in the mix.

IWAG distribution commentary: While 2019 was a disappointing decline for Foltynewicz, it wasn’t that unexpected, just unfortunate. Given his track record, there was always the possibility that he’d return to his pre-2018, fairly ineffective self; while the reasons for his stumbles were different in 2019 than previously, the end result wasn’t one he had avoided before. Sure, he finished as something like a 15th percentile outcome by IWAG, but that’s not too weird, especially not compared to some of the other pitchers here.

A note about Dallas Keuchel

Dallas Keuchel wasn’t technically projected as a Brave before the season started... because he wasn’t. Suffice to say, Keuchel was generally expected to provide at least average, if not better, rate-based production. After all, he was coming off a three-win season and hadn’t posted an FIP- above 92 since 2013.

Unfortunately for the Braves, the smaller-than-full-season sample and some HR/FB shenanigans tripped up their big midseason addition. Keuchel’s 91 xFIP- was the same in 2019 as it was in 2018, but he had by far the highest HR/FB rate of his career, which picked his FIP- to a worst-since-his-rookie-season 108. As a result, he too ended up pitching like a fourth starter, though perhaps undeservedly so. (On the flip side, Keuchel wasn’t super-unlucky on eventual homers, suggesting that his HR/FB was perhaps more than just random variation... which may have interesting implications for his market this offseason, and why the Braves gave Cole Hamels a one-year deal rather than retaining his services.)

Kevin Gausman

• Expectations: 99 to 100 FIP-; 2.6 to 2.7 fWAR/200; 170 to 184 IP
• Actuals: 17 starts, 14 relief appearances (none with the Braves); 3.1 fWAR/200; 102.3 IP

Kevin Gausman was supposed to be a steady, unexciting innings eater that existed in the rotation and added some marginal wins while the most interesting and variable talents around him thrived, worked through issues, or faltered and got replaced with other alternatives. He was supposed to be the sort of humdrum, but still productive pseudo-veteran that keeps the team viable without doing much to boost it on his own. 2019 was anything but that.

There are multiple Kevin Gausman realities. One of them is that he was something like one of the unluckiest pitchers of the past decade. Since the start of 2001, only nine starters with 80 or more innings pitched have had ERAs in excess of their FIPs by 2.00 or more. (One of those nine, Jordan Zimmermann, also achieved this feat during 2019, and had an even bigger gap than Gausman.) Not only was the gap large, it was very persistent: after his fifth start of the year, when he allowed five runs in five innings despite just seven baserunners (it was still a bad start, just not that bad), it basically not only just hung there, but got ridiculously worse due to three meltdowns in June.

But that’s not the only reality. The obvious other one: despite all this, the Braves rapidly cut ties with the right-hander, giving him the very short leash of just one additional blow-up before casting him to the wind. Even though he was unlucky, he was also getting hit hard, with a 31st-percentile xwOBA-against among starters, suggesting he was perhaps doing somewhat worse than his league-average FIP- as a starter (96) was signaling.

In the end, on a value basis, Gausman more or less did exactly what the projections were expecting, if not a bit better. The problem was that the value basis didn’t line up with the results basis, and the Braves decided that they had apparently had enough. As a result, he managed only about half of the innings he was projected for as a Brave, even though his performance in those innings was in line with expectations. Even where the projections lined up for a Braves pitcher, the universe somehow found a way for them to not. That’s the theme of this post, I think.

Fun fact: Gausman produced 0.4 fWAR and 0.2 RA9-WAR after moving to the Reds. (0.4 fWAR and 0.3 RA9-WAR as a reliever.) From the time the Braves let him to go to the end of the season, that RA9-WAR topped the marks managed by all but three of the Braves’ relievers (Luke Jackson, Mark Melancon, Shane Greene) and the fWAR again topped all but three relief arms (swap Shane Greene with Chris Martin in the above). I hope the Braves got good mileage out of whatever fractional salary bits they saved by cutting Gausman, because they probably could have actually improved their bullpen down the stretch by using him in lieu of, say, Anthony Swarzak or Chad Sobotka... or even Sean Newcomb, who really didn’t fare well to end the year.

IWAG distribution commentary: Nothing unexpected here, given the above. Per IWAG’s distribution curves, Gausman’s FIP-based value was a little better than expected (70th+ percentile FIP- and fWAR/200) but his lack of exposure hampered his overall production (fifth-percentile-ish fWAR, due to pitching only 102 innings instead of a full season).

The other not-starters

Here we have a quintet of arms who didn’t play anything like the expected role for the Braves. Interpreting and analyzing their projections is pretty hard, given this lack of alignment. Still, suffice to say, things didn’t really work out for the Braves with these guys, though not always for the same reason.

• Bryse Wilson was projected to be a league-average-ish, or slightly below, arm. Steamer expected him to do better as a swingman, ZiPS and IWAG saw somewhat below-average FIPs. Instead, Wilson imploded to the extent that he only got 20 IP all season, with a ghastly 145 FIP- in that stretch. That’s not much (or any) indictment of his skillset or potential, as he fared just fine in Triple-A; it’s more just a recording of the fact that a guy who was supposed to be fine in the majors (and probably could have been had the sample been expanded) wasn’t, and wasn’t given a chance to prove otherwise.
• Kolby Allard’s projections, meanwhile, ranged from fifth starter type (Steamer) to something somewhat better, up to a #3/#4 profile (ZiPS, 109 FIP-). Allard didn’t even get a chance with the Braves, but posted 45 innings with an 84 FIP- for the Rangers after being shipped away at the Trade Deadline. That’s a pretty sexy mark, good for 1.1 fWAR in a fourth of a season, and therefore an extreme 4.9 fWAR/200. But, anyone prepared to rend garments on account of Allard no longer being in the organization may want to peek at his 111 xFIP-, which is basically in line with preseason projections for his FIP-. That the Braves traded away actual-universe 1.1 fWAR in 45 innings is kind of annoying given what they got back; whether they’ll actually miss the potential fourth-or-fifth-starter-type that Allard’s xFIP suggests he might be with greater exposure seems like a less concerning query.
• Sean Newcomb was projected at worst (Steamer) to be something like a good #4 starter (1.8 fWAR/200). He got sent to the bullpen, where he pitched 68 innings of 97 FIP- ball, delivering arguably less to the team than if he had just repeated his 2018 or half-season 2017 with no personal progress made. He was an okay reliever, just like he was an okay starter, and unlike that weird commercial that plays during MLB.tv commercial breaks sometimes, just okay is definitely okay, sometimes.
• Touki Toussaint, I like to think, is the coin flip of Max Fried. Fried was projected for FIP-s of 91 to 117 and fWAR/200s of 0.9 to 2.4 (though again, Steamer had him as a low-exposure swingman). Toussaint was projected for FIP-s of 104 to 117 and fWAR/200s of 0.9 to 2.1. IWAG was the low system on both and didn’t see them as having notably different central outcomes. As noted, Fried punched all of his projections in the face and then sauntered away as helicopters exploded behind him in the background. Toussaint was more like one of the exploding helicopters, with his 114 FIP- and sub-replacement-level performance, even though he should have benefited from relief work. That’s not to say that Toussaint was always awful, as he had a few brilliant outings in 2019. But the bulk of his season was unfortunate. There but for the grace of Trout goes Max Fried, basically.
• Wes Parsons was a very trendy (and my favorite) pick for a dark horse, unheralded candidate to make an impact for the Braves in 2019. He did all the right things in Spring Training, which added some steam to his train. He made the team out of Spring Training, and for a few halcyon, chilly days in the spring, it looked like he would actually make a difference for his team. In his first nine outings, only one was bad, only two cost his team WPA, and a bunch were bacon-saving. Then he imploded, was quickly demoted, wasn’t any better when recalled, and eventually got cut by the team and claimed by the Rockies, where he was similarly terrible. (Somewhere on the ironic-to-annoying spectrum is the fact that after he was cut, his best outing as a Rockie came against the Braves.)

So, once again, we come to more or less the same conclusion as with position players, but more extreme: the projections were fine in aggregate, even if they were off horribly for individuals. The rotation was projected to be average-y, and average-y it was, though it did outperform expectations a bit.

---

The relievers

This is very much the “silly season” portion of any projections retrospective, and almost not worth its digital ink. But for the sake of completeness, let’s run through it.

Various projection systems and depth chart playing time assignments figured that somewhere between 12 and 17 pitchers would take the ball in relief for the Braves in 2019. Fangraphs’ Depth Charts playing time assignments and the player projections for 17 relievers yielded bullpen fWAR estimates of 2.8 (ZiPS), 3.2 (Steamer), and 4.2 (IWAG) over 530 innings. Using IWAG’s more simplified health-based model for playing time, the projections were 2.2 fWAR (Steamer), 2.6 (ZiPS), and 3.4 (IWAG) over 571 innings.

In actuality, the bullpen pitched 577 ²⁄₃ innings of 1.1 fWAR ball; it fared somewhat better by RA9-WAR (yay, sequencing and BABIP) and much better by WPA. Just imagine how painful the bullpen travails would have been had the RA9-WAR and WPA performance been even worse than the fWAR...

The end result: the bullpen was supposed to be average-y, maybe a little worse, maybe a little better, and instead, it was pretty bad. It ended up being the balancer for a position player group and rotation that both did slightly better than expected. The Braves were projected for 89 WAR-wins and something like 84-86 wins, they finished with 87 WAR-wins (and of course, 97 wins due to some really great game-level run sequencing). Thanks but no thanks, 2019 Braves bullpen. Collectively, the bullpen ended up at something around the fifth percentile estimated by IWAG, as compared to the position players and starters, who were both in the 70th percentile or above. You can kind of see why the Braves moved to shore up that part of the roster with such frantic alacrity during the offseason, though it’s unclear whether that is going to pay any dividends or is just an overreaction in the name of attempting to retain control, the way my wife cleans when she stressed even though the cleaning doesn’t actually address the source of the stress. We’ll see.

Anyway, here’s what happened with the projected pitchers, relative to their expectations, in brief:

Was pretty bad, as expected

• Shane Carle’s best projection was IWAG’s 0.3/65; Steamer and ZiPS both had him as replacement level. He was worse and somehow compiled -0.4 fWAR in just six games and 19 outs of work. And he was so fun early in 2018, too!
• Chad Sobotka’s best projection was Steamer’s 0.0/65. He ended up well below replacement level in limited playing time. I kind of want to make a crack that’s like, “Oh boy if only someone could have seen it coming,” but honestly, no one can see anything about relievers coming.

Was randomly, unexpectedly good!

• Luke Jackson! Luke Jackson, Luke Jackson, Luke Jackson! I remember being reasonably salty (Planters peanuts level, not Dead Sea level) that Jackson made the Opening Day roster over Dan Winkler. That episode has now made me completely apathetic about the qualities of individual relievers. Luke Jackson went from roster barnacle with middling projections (0.3/65 Steamer, 0.6/65 ZiPS, 0.0/65 IWAG) to one of the best relievers in baseball. Will it keep up? Who knows? Was it fun? I think, to the extent anything about relievers can be fun. Does ZiPS have some kind of biomechanical mindmeld with Luke Jackson? Probably!
• “Good” might be overselling Webb a bit, but he was certainly interesting when he wasn’t expected to be. Webb got replacement-level projections (0.0/65) from Steamer and IWAG, and ZiPS (-0.6/65) wanted to kill him with fire. Webb didn’t go nuts with fWAR (just 0.2 in 32 frames, or around 0.4/65), but he had some real nice xwOBA numbers and stranded a lot of runners.

Was supposed to be good, barely existed

• Arodys Vizcaino was projected as an average-ish-to-maybe-above reliever (0.3 fWAR per 65 IP to 0.6/65). He pitched four innings of sub-replacement-level ball, got hurt, got traded, and didn’t pitch in the majors again. Welp.
• Darren O’Day was at least effective during the five frames he actually worked, and even made the playoff roster. He was supposed to be pretty decent (0.6/65 to 0.7/65), and maybe he would have been, had he actually existed. 2020 is yet another chance for him to maybe provide a full month of active duty in a Braves uniform, I guess.
• Corbin Clouse had some nice (again, maybe not “good”) projections (0.2/65 to 0.5/65) but got hurt and didn’t even make the majors. Better luck next year, or maybe the one after that.

Was supposed to be good, was awful

• A. J. Minter had some of the better projections for a reliever in baseball coming into the year. He was generally getting top-40-ish reliever projections before the season, which isn’t elite, but it certainly isn’t the dramatically-gross mess he ended up being in 2019, either. The most conservative projections missed on Minter’s fWAR/65 by about a full win, which is a little bonkers. The Braves didn’t need yet another cautionary tale about reliever downfalls, what with Ian Krol et al. already in the fold, yet here we are. Which expected-to-be-good reliever will implode in 2020? Stay tuned and find out.
• Dan Winkler got a more tepid projection than Minter did from Steamer (0.3/65, compared to 0.8/65 from Steamer), but had similar projections to the left-hander by both ZiPS and IWAG. He flipped the script by being even more profoundly awful than Minter, and getting traded away at the Trade Deadline, where he was promptly cut by the Giants. Not great. Winkler was an awesome story, but his 2019 was a case study in why you can’t trust right-handed relievers either.

Was supposed to be okay, was awful

• Okay, ZiPS saw Jesse Biddle’s meltdown coming (-0.1/65), but Steamer and IWAG (0.5/65, 0.3/65, respectively) thought he’d be okay. He was definitely not okay, got traded, and ended up logging time in three different major league bullpens in 2019. He was somehow worse not in Atlanta than in Atlanta, which is ghastly.
• Sam Freeman went from 0.7 fWAR in 2017, to 0.3 fWAR in 2018, to getting cut by the Braves, to -0.1 fWAR in just one major league appearance (for the Angels) in 2019. Life comes at you fast. Freeman wasn’t expected to be of much interest (0.0/65 to 0.3/65) anyway, so this isn’t a huge miss.
• Grant Dayton got 0.4/65 projections from Steamer and ZiPS; IWAG saw him as replacement level. He was slightly below replacement level. So it goes.

Wharrgarbl

• Jonny Venters, the 2018 NL (even though he spent some of it in the AL) Comeback Player of the Year, had some really variable projections, from -0.2/65 (ZiPS) to 0.7/65 (IWAG). He managed to put up -0.4 fWAR as a Brave in just nine games, managing to get just 14 outs. He fared a little better in three games for the Nationals, but still, this was just a sad story all around.
• Thomas Burrows didn’t have good projections, but also didn’t get a shot. He also didn’t get claimed in the Rule 5 draft. Maybe we’ll see him in Atlanta eventually.

The above 14 players combined to pitch only 232 of the team’s 578 or so relief innings, or just about 40 percent. If that doesn’t tell you how silly it is to try and project bullpen-anything in earnest, I’m not sure what will. In lieu of these 14 players combining for the bulk of relief innings, the Braves relied on late additions (Josh Tomlin, who led the team in relief innings), converted or filling-in starters (Sean Newcomb and Touki Toussaint), and trade acquisitions (Anthony Swarzak and Jerry Blevins early, and then the better-performing relievers at the Trade Deadline). Notably, across any reliever not in this group of 14 who pitched 25 or more innings as a Brave, the sum total fWAR was -0.1 fWAR across 253 innings. So yeah, you read that right: the Braves got more innings from guys not even projected to be in the bullpen at the time the projection exercise was done in mid-March than actual projected team relievers, and all of those guys ended up being pretty terrible.

---

Now that that’s all done, my real hard work can begin — re-tinkering with IWAG to make an even better version for 2020. Say, anyone have any idea what the run environment is going to be next year?