Blogads

Google Ads


« The Yankee Years: It's Just Not That Bad | Main | Stain on the Sheets »

Wednesday, February 04, 2009

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

thanks for compiling all this stuff paul. it's much appreciated.

If you have the time and interest, I wonder how these systems did with 2008 relative to actual performance. Do they publish "error" terms so we know a range of how far they're usually off? I think they're going to be far off on Youkilis. He seems to be a late bloomer rather than a one-year wonder.

To nitpick, if Drew has the year they're projecting, that's a clear step down.

If the Sox get either the PECOTA or MARCEL projection for Lowrie, you'll be pleased. That's a top five shortstop in the AL.

I don't see the optimism for the pitching. I don't see how Smoltz puts up 100 innings. Either way, that means more Wakefield and Penny. Their projections aren't that good, especially the K:BB numbers (where Penny is really bad) in the AL Beast. All combined, that's a lot of innings to a lot of unknowns. Even if the top 3 don't regress (and hard to see how at least Dice-K doesn't and perhaps Lester too), it's tough to be very confident about the staff. I know I'd think more highly if they were giving a slot to a kid with more upside.

Great work Paul, you always seem to outdo yourself. I'm really looking forward to some goddamn baseball!

Fascinating. Thank you, Paul.

To nitpick, if Drew has the year they're projecting, that's a clear step down.

Absolutely, but Baldelli under these projections would be a clear step up from the production the Sox got out of Ellsbury and Kotsay when they were in right field last year.

As for last year's projections, the Hardball Times looked at some of this, though it didn't include Marcel. (ZiPS turns out pretty poorly, actually; maybe I should have used CHONE.)

Tango had this to say:

... [W]e can stop with the nonsense that one system is better than the other. As I’ve said in the past, it is a very dubious claim, akin to saying a team that wins 85 games is better than a team that wins 84 games. Marcel will win 83 or 84 games, the good forecasting systems will win 84 or 85, and the not so good will win 81 or 82.

Tango looked at CHONE, ZiPS, PECOTA and THT and found CHONE to be the best, with PECOTA and ZiPS roughly tied. But they're all pretty close anyway.

Cool, thanks. Rather than a ranking, I'm interested in what kind of variability we can expect like +/- AVG, +/- SLG, etc. If they're saying that that Drew's SLG will be .460 +/- .50, that's not saying much.

> I wonder how these systems did with 2008 relative to actual performance. Do they publish "error" terms so we know a range of how far they're usually off?

Averaged across all players in the league, the deviation from the expected mean will be minuscule. On a case-basis, of course there will be varying degrees of drift but I don't see the point in player-by-player analysis of errors in predictors other than how the true statistical gurus may find whether those drifts were the fault of some specific issue with the predictive analysis tool rather than "why we play the game."

I'm an idiot. That Times piece is pretty good.

Last year's error on OPS (not adjusted):

THT, .051
CHONE, .053
PECOTA, .055
ZiPS, .061

So within 50 and 60 points for all hitters. That's hardly small, but it is pretty good.


Last year's error on ERA (not adjusted):

PECOTA, .620
THT, .629
CHONE, .665
ZiPS, .676

So almost 2/3's of a run for all pitchers. Again, not insignificant, but pretty good.

Overall, it's a good not great picture. For a projected .850 OPS hitter, we could be looking at a .800 or .900 OPS hitter. For a projected 3.60 ERA pitcher, we could see a 3.00 or 4.20 ERA pitcher.

For a projected .850 OPS hitter, we could be looking at a .800 or .900 OPS hitter. For a projected 3.60 ERA pitcher, we could see a 3.00 or 4.20 ERA pitcher.

Which I ghope we'd all be kind of adjusting for in our heads anyway, even if we didn't have the exact numbers in mind. It's why I'm pretty careful to use the word "projection" and not "prediction."

> all hitters // all pitchers

Each, not all. I agree, it is pretty good.

all = across all. It's an average not a data point.

ex-sox/ex-yankee ramiro mendoza singed on with the brewers today. who knew he was still out there?

zzzzzzzzzzzzz

"A good friend of mine used to say, 'This is a very simple game. You throw the ball, you catch the ball, you hit the ball. Sometimes you win, sometimes you lose, sometimes it rains.' Think about that for a while."

Seriously? Someone takes the time to put together free and interesting content, during the most boring time of the baseball year, and folks are complaining that it's boring and irrelevant?

Show some basic manners or don't comment.

Well done. February would be unbearable without projections. Hope springs eternal. Thanks for putting this comparison together. Cam Martin

ag, as someone that actually understands volatility, error bars, standard deviations, and stuff, I disagree with you.

I would be interested though, to see if performance is normally distributed (or some variation like lognorm) since otherwise, the errors won't mean much..

> ag, as someone that actually understands volatility, error bars, standard deviations, and stuff, I disagree with you.

Point taken, Lar.

Actually, without going into a huge deal about it, one of the things that that irks me is that, generally, articles usually just give averages, which paints a picture, but perhaps a fuzzy one.

Without giving a math lesson (maybe that'll be an article!) volatility/variance cleans that picture up a little bit - it allows us to visualize the probability buckets, and maybe with systematic contributions (basically, error functions to the model) or player contributions (overperform/underperform).

I think the problem is that there's no real easy way to do it, other than maybe analyzing previous data/predictions. I've thought about this problem a bit. (say, someone who hits .300 every month, vs someone hitting .400 in October, both with the same average)

On a somewhat related note (but perhaps not), one observation is that "Game Score" by pitchers actually easily fits into this - you can easily compute the "Game Score" average, and thus the stdev/vol, and all the good stuff.

It turns out (not surprising to game theorists) that you can work out that you actually want a higher game score volatility if your average game score is not particularly high - I can elaborate, but this was actually referring to Josh Beckett's awful 5 ERA year. Go figure.

Maybe I should code this up, hahaha.

That lack of recognizing variance also really bothers me, esp. in the projections. But if they say that they're 95% confident that Wang will have an ERA between 3.34 and 4.66 (or Jeter will OPS between .750 and .850), you soon realize how much folly is involved. Still, for clarity, they absolutely should. It reminds me of the Netflix competition. The projection systems are incremental changes but the data is so inherently noisy we're just stuck happily watching the games.

rob- pretty sure dc's "zzzzzz..." was in response to the mendoza signing. besides, that's his standard operating procedure. rest assured he wasn't denigrating pauls fine effort.

I don't mean to denegrate Paul, either. Clearly his research is solid and relevant and worthy of our respect.
However, I do have moments when I'm painfully old fashioned, such as when I have to do some new-fangled cipherin'.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been posted. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Search YFSF




Sports Gambling

twitter

schedule & standings