The developers at Sony San Diego did something new and experimental on Monday by releasing a massive roster update that changed the ratings of many, many players in MLB The Show. It was a meaningful update for a variety of reasons that I will get into, but more than anything it was another attempt by a sports game developer to try and ensure its game is not shelved and forgotten. 

 

So after the jump, I’m going to talk about this update and how it fails and succeeds to hit the mark.

In a perfect world, this update comes out around the All-Star break as originally planned, changes a ton of ratings and makes everyone happy. However, it is not a perfect world, and even if the update had come out on time, some people would have been unhappy with the results. Either way, the first issue seems to be a classic argument about sample size. And to me, this argument about sample size informs every other positive and negative about this update.

 

Nick Livingston, a designer on MLB The Show, explained on the official Show forums that you should not worry too much about “overall” ratings, and you should focus more on “attribute” ratings. In addition, he pointed out that they go with a sabermetrics approach that focuses more on “a player’s performance independent of the team or stadium he is working with” over a three-year span. In other words, I will take that to mean the developers care more about things like FIP and xFIP for pitchers rather than ERA and wins; and for hitters I will assume that means the developers care more about things like line-drive percentage and BABIP rather than batting average and RBI. Both these things are positive in theory, but they become trickier in a video game with competing parties.

 

 

In short, there is the Diamond Dynasty crowd, and then there is the Franchise crowd. The Diamond Dynasty crowd has to worry about “overall” ratings because it impacts buying and selling prices. The Franchise crowd does not have this concern. The Diamond Dynasty crowd has been getting updates all season based on recent results, while this update more closely takes into account the three-year span once more. The Franchise crowd should be happy that recency bias plays less a part in this update as baseball statistics teach us -- like with most things -- that a “true” outcome becomes more apparent over a larger sample size. One player having a hot month should not make him a 90 overall in your franchise if you don’t actually think he’s a 90-rated player. Still, I hope it’s clear there’s a very apparent issue arising between these two audiences.

 

What Best Serves The Diamond Dynasty Audience?

 

You will not see me write a bad word about a developer using advanced statistics to judge players. I love that approach as it’s the more logical approach. The less we use ERA and RBI the better. Still, even with baseball’s advanced statistics there is the idea of what has already happened versus what will happen in the future. The easiest example of this is when a team projected to lose a lot ends up being 20-8 early in the season. This team was projected to lose a lot, and over a larger sample they probably still will lose a lot. Nevertheless, in the short term this team is maybe way better than expected because it already has those 20 wins under its belt. This is the epitome of Diamond Dynasty.

 

Diamond Dynasty survives and thrives based on regular and somewhat reactionary updates. The marketplace, in particular, remains active because people do play it like a stock market of sorts. Users are looking for people having hot or cold weeks, and then they buy and sell based on potential projections. Ratings players like this is anathema to advanced baseball statistics, but it’s probably healthy for the Diamond Dynasty mode.

 

I especially say that because the Diamond Dynasty roster is set in stone. New cards are not introduced, and because there is no salary cap or reason to avoid using the best players, only Legend, Gold and Silver cards are the ones that really matter. Now, perhaps it’s a design flaw that only so many cards have value, but regardless, the point is that change is probably healthy and necessary to keep Diamond Dynasty interesting.

 

 

However, this is where video game rule sets come into play. Sports video games are mostly based on rules. Diamond Dynasty has been operating under what appears to be an umbrella of logical “snap judgment” ratings changes. What I mean is that someone who has shown no power for five years does not automatically get a plus-20 boost to his power ratings if he hits five home runs in a week. But someone like Bryce Harper can get shot up to Diamond status because of his age, prospect status and previous sample sizes. 

 

Point being, this roster update changed the rules for Diamond Dynasty users as it seems to have mostly gone back to adjusting for the “three-year” sample size. Again, in theory that’s a good thing as you will likely have more accurate ratings in the long run. But remember my point about short-term sample sizes.

 

Brandon Crawford is an easy example to use and has been brought up a bunch by folks who think this update was not worth the wait. Crawford has hit 19 home runs in 427 at-bats. His previous high was 10 home runs in 564 at-bats during the 2014 season. He did not get a power boost in this update. Now do I think Brandon Crawford has turned into a 25 home-run player? Absolutely not. 

 

Breaking it down, Crawford’s line-drive percentage and fly-ball rate are both down, and his groundball rate is actually up year over year. In addition, he’s a hitting a home run on 18 percent of his fly balls, which is insane compared to his previous seasons. Now, he is hitting the ball a bit harder this year overall, but the point is this season seems to be an outlier. But the results are still what they are at this point in the season. Over a three-year span, it’s likely Crawford will not hit 25 home runs in a season again. But over the last 1.5 months of a season where a lot of results are already in the bag, is Crawford now a 25 home run player? I would be inclined to say for Diamond Dynasty purposes he is that player. It does not mean when next season rolls around he gets to keep the upgrades he has right now, but it certainly feels like he should be higher rated at this juncture. 

 

What Best Serves The Franchise Audience?

 

Franchise folks have a less complicated relationship with this update. First off, the roster update can be completely changed because it’s a living file for Franchise fans. Besides that, a lot of people won’t be restarting their franchise to use this update. On top of that, others might be using OSFM (Operation Sports Full Minors) rather than this update.

 

However, there is still an audience out there buying the game at a reduced price. Maybe that audience does not know about the glory of OSFM, or any of the other amazing work that the roster editors do on the site. Those folks are going to use the default rosters, and this update better serves them. Now, it’s a “modified” version of a year-to-year update according to the developers, which seems to mean stuff like defensive ratings and most speed/steal attributes were not touched (advanced defensive stats in particular are even more volatile from season to season so this seems fine to leave this way).

 

Moving Forward

 

So really what this update did is put the Diamond Dynasty audience in particular in a weird spot. It showed that perhaps Diamond Dynasty should be treated differently than other modes. Maybe there needs to be two different rosters. Maybe Diamond Dynasty needs to change a bit and make more players matter to better fit into the grand scheme of one universal roster. There’s a lot that could change to make things better.

 

That being said, it’s unfair to expect developers to have new tools at their disposal during a development cycle, so the folks at Sony San Diego were always going to have to deal with some negative backlash to this roster update. But the point is that a lot of the time you don’t figure these things out until after the fact. This is live play-testing of sorts. This is a developer experimenting and seeing what the reaction is to a large change. It’s the world we live in now where “live” updates are expected. It’s a world where developers want you playing their game all year, and hopefully maybe even putting some extra money into the game in the process. It’s a world everybody is still trying to figure out the best way to do all of these things.

 

 

So maybe the update did not leave everybody with warm and happy feelings, but it’s an experiment that I hope to see continue moving forward. It’s always going to be hard trying to avoid angering those who play your game all the time versus trying to do something you think will help the audience as a whole. Removing the Triple Play Challenge angered some, but others liked it because it seemed to unbalance the marketplace. Some liked the ratings changes, others hated that they got burned buying a bunch of players like Manny Machado because they thought they now understood how ratings updates worked. These are issues that will always crop up as you change a living, breathing product. 

 

The point is that hopefully the developers will not be afraid to experiment even if it enrages some folks along the way. The hope is that both developers and players can live with some “negative” results in pursuit of something better in the end.