tag:blogger.com,1999:blog-38600807.post3814381443512554616..comments2007-09-08T02:56:01.225+03:00Brian Burkenoreply@blogger.comBlogger10125tag:blogger.com,1999:blog-38600807.post-70757427988242976532007-09-08T02:56:00.000+03:002007-09-08T02:56:00.000+03:00

yeah -- sorry muddled projections wasn't very clear. FO projects the probability of wins by bucket. Some of the teams are very clearly projected to be very good or bad -- Philly ~90% in the 9-10 plus 11+ buckets; KC ~70% in the 0-4 plus 5-6 buckets. Some team's projections are ~equally spread through all buckets, like Minn -- 10% in the 0-4 and 11+ and ~25% in the others. If you throw out

mileshttp://www.blogger.com/profile/06893515504632890107noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-31715465093655222862007-09-06T05:10:00.000+03:002007-09-06T05:10:00.000+03:00

Thanks for running that analysis on the 8-win predictions. I didn't really feel like it to be honest. I'm surprised how little value the FO predictions add. But I think this is why it's important to publish your error rates and accuracy metrics. If you're trumpeting what you got right, you need to be forthright about what you got wrong. It's not good business (and clearly FO has built a

Derekhttp://www.blogger.com/profile/17941314072950152029noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-52824548444436763992007-09-06T01:29:00.000+03:002007-09-06T01:29:00.000+03:00

Miles-I saw that post, but I don't understand what you mean by muddled predictions.

Brian Burkehttp://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-82707913471854675812007-09-05T19:57:00.000+03:002007-09-05T19:57:00.000+03:00

Here's a similar discussion on the FO site: http://footballoutsiders.com/discussion/viewtopic.php?t=146I did find the FO predictions to be better (in 2006) when ignoring the teams that had muddled predictions, but only marginally better in 2005. Not a whole lot of data to use.

milesnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-57903490342756896462007-09-05T06:33:00.000+03:002007-09-05T06:33:00.000+03:00

Derek, I agree MAE is only one way to look at it. But I disagree that 17/32 within 2 games, and 10/32 within 1 game, is any good.I ran the same comparison for the "mindless" 8-win prediction method (which I guess should be called the "null prediction"). Over the past 2 years, it was within 2 games an average of 16.5 times, and within 1 game an average of 10 times.For 2006 it was within 2 games 16

Brian Burkehttp://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-45426420797098216892007-09-05T06:03:00.000+03:002007-09-05T06:03:00.000+03:00

I think there are other ways to look at it besides mean absolute error. By MAE, yeah, they're no good. Correlation coefficient is another way to look at it. I looked at it another way (as I know you've read) and found that 17 out of the 32 predictions were within 2 games of being correct, 10 of those being within a game. That's not bad actually.And maybe it's not the exact win totals that

Derekhttp://www.blogger.com/profile/17941314072950152029noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-30708835362180151982007-09-04T22:32:00.000+03:002007-09-04T22:32:00.000+03:00

Chip-By "last year's record" I mean a simple regression based on last year's record, not just using a team's previous wins as the projection for next year. See the end note for the formula.What this does is simply apply a regression to the mean to last year's wins. So a prediction for a 13-win team would be about 11 wins, and a prediction for a 2 win team would be about 5 wins.

Brian Burkehttp://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-34516268879575323222007-09-04T22:26:00.000+03:002007-09-04T22:26:00.000+03:00

FO's projections are not great, but not worthless either.I looked at the Pearson rank correlation for the FO's and Vegas' projections as well as last year's record: 2006 2005FO 43% 33%Vegas 45% 20%Last Year's 30% 20%

Chiphttp://www.blogger.com/profile/15848013553962095565noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-60389967342919340292007-09-04T21:38:00.000+03:002007-09-04T21:38:00.000+03:00

The only pattern I recognize is that expert picks tend to go out on a limb. They need to validate their existence, so if they pick teams to win within a range of 6-10 games, no one would bother paying attention. But the smartest prediction strategy is to go as short out on a limb as possible (like the 8-win strategy).I'd put money on teams that Vegas has as outliers: 5 or less wins and 10 or more

Brian Burkehttp://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-17140839204905668682007-09-04T19:00:00.000+03:002007-09-04T19:00:00.000+03:00

Is there any discernable tendency if we look at the predictions where Vegas/FO/8-8/regression to the mean disagree strongly with one another?Let me give an example. I have no intention of wagering on team wins. But if I were to place such a wager, I would look for teams where:- The Vegas over/under and the FO prediction differed by more than 2 games.- The Vegas over/under was far off from 8-8,

Tarrhttp://www.blogger.com/profile/14368810359650066790noreply@blogger.com