How Draft Experts Are Graded

It seems that everyone has a mock draft board these days. The object of this parlor game seems to be to predict which players will be picked by each team. So draft gurus tend to be judged by how many correct predictions they make. When you think about it, it's pretty ridiculous. A draft expert has to get four things right. He needs to not only evaluate players and team needs, but also evaluate all 32 teams' own perceptions of each player and its needs. And if one prediction at the top of the first round is off the mark, the house of cards collapses.

I think a much better way to evaluate draft experts is to wait several years, then see which players actually turned out to be more productive. Look at how they rated each player, not how well they read the minds of the league's GMs.

Besides, in my eyes the real value of a draft expert is be able to tell me in 30 seconds everything there is to know about that strong safety from Alcorn State my favorite team just picked up midway through the 4th round...without any notes or any hesitation...and has a full head of hair. And for that, there is only one man.

Drafting Wide Receivers

If Lions GM Matt Millen has contributed anything to the world of football player evaluation, it's that wasting a top pick on a wide receiver is very dumb idea. The fragile-egoed temperamental prima donnas that they are, WRs are known as a notoriously undependable lot. Worse, their productivity seems completely unpredictable. But it's not true.

Using data from the draft database at Pro-Football-Reference.com, all WR draft picks from 1980-2000 were evaluated by Pro Bowl selection, receiving yards per year, and years as a starter.

Pro Bowls

Receivers were classified as being selected one or more, two or more, or three or more Pro Bowls. The likelihood of each by round and by position draft order is illustrated in the two graphs below.



After the 4th WR taken in the average draft class, it becomes drastically less likely a team will find a star player. It looks like that the drop-off usually happens around halfway through the 2nd round. By the 5th WR taken and the 3rd round, WRs appear to be about equally as likely as much later picks of becoming an all-pro at about a 5-10% probability.

Receiving Yards Per Year

I normally prefer rate stats when comparing players, but comparing the records of receivers is an exception. Yards per reception is a helpful stat, but receivers create their own receptions by getting open, so yards per reception would not capture a very important part of playing WR. Yards per "target" might be better, but targets are very subjective and unstandardized statistic. For this comparison, I'll use career yards per year broken out by round and by position draft order.

We see a similar pattern as with Pro Bowl selection. After the 4th WR taken, prospects for finding a solid contributor drop significantly.

Scout Accuracy

How well to NFL team scouts and player evaluators predict the performance of WR draft picks? The table below lists the likelihood that a WR will end up having more Yds/Yr than the next WR taken in the draft. For example, the 1st WR taken ends up as better than the 2nd WR taken only 38% of the time.

WR Pick	Pr(Better)
1	0.38
2	0.67
3	0.48
4	0.81
5	0.33
6	0.62
7	0.62
8	0.38
9	0.52
10	0.52
11	0.57
12	0.57
13	0.48
14	0.43
15	0.52

It seems to be very difficult for teams to identify the better player between two closely matched potential draft picks, even for the #1 WR taken. But overall, they seem to have selected the better receivers in the top round. These results suggest a good strategy for picking a good WR with a very high draft pick is to trade down to pick the second or third WR available in the draft. Teams should have very little confidence that the highest rated WR will turn out any better than the second. On the other hand the #1 WRs tend to become the biggest stars, being selected to three or more Pro Bowls in one out of three cases.

Years as Starter

Another way to evaluate draft picks is to look at how many years they are starters for their teams. The tables below list the averages for each round and draft order.

Round	Yrs as Starter
1	5.7
2	3.3
3	2.1
4	1.3
5	0.7
6	0.5
7	0.6

WR Pick	Yrs as Starter
1	5.9
2	6.6
3	4.9
4	4.7
5	2.0
6	3.3
7	2.7
8	1.8
9	2.5
10	1.9
11	1.7
12	1.0
13	1.3
14	0.9
15	0.8

Conclusion

As temperamental as they may be, top WR draft picks really do turn out to be stars far more often than later picks. They seem to be a lot like QBs. There is a real scarcity of talent at both positions, and it is difficult to predict with much certainty which ones will pan out. A team's chances of finding a highly productive player are still better with a top pick.

Drafting Defensive Ends

Where can NFL teams find quality defensive ends in the draft? This post looks at DEs drafted in the first seven round from 1980-2000. Data comes from the great draft database at Pro-Football-Reference.com. Picks from each round and at each draft order are compared by Pro Bowl selection, sacks per year, and number of years as a primary starter.

Although sacks are obviously not the only contribution a DE makes, and some DEs benefit from certain defensive schemes more than others, I believe that it is still a valid measure. First, sacks are only used as one measure. Overall DE performance can be reflected in Pro Bowl selections and years as primary starter. Additionally, sack rate isn't used to rank individual players, just the aggregate from each round and at each spot in the draft order. Other DE contributions to the pass rush such as QB hits or hurries (or even soaking up a double team) are very difficult to quantify. However, sacks can be used as a proxy measure for the overall contribution to the pass rush. In other words, a player with a lot of sacks very probably also has a lot of hits and hurries. Across so many seasons and with so many players, the effects of scheme and other considerations largely wash out.

Unlike QBs and RBs, players who did not make a roster or make many game appearances did not pose a problem when scoring their contribution. If they didn't have any sacks, they simply got a zero.

Pro Bowl Selection

The graphs below show the likelihood that a drafted DE will be selected to the Pro Bowl. The first graph is broken out by draft round. There is a steady decline from the top rounds to the last.

The second graph is broken out by draft order, that is, which DE was the player taken as in his draft class. This graph is much more erratic.


To me, the erratic nature of the graph suggests that compared to QBs and RBs it is easier for later round picks to shine. My guess is that later pick DEs get far more opportunities than later pick QBs or RBs, partly because a team needs two DEs on the field. Plus, they're more likely to be platooned.

Sacks Per Year

Despite its limitations as a comprehensive measure of DE performance, there is a clear relationship between draft round and order and sacks. First round picks significantly outperform subsequent picks in particular.

Notice the similarity in shape from the sacks graph to the Pro Bowl graph. This suggests that DEs are being picked for the Pro Bowl on the basis of their sacks. No surprise there.

Scout Accuracy

How certain can scouts and GMs be that they're selecting the better DE compared to the next guy down the draft board? The tables below list the likelihood that the higher player selected will turn out to have a better sack rate than the next DE taken in the same draft.

DE Pick	Pr(Better)
1	0.48
2	0.67
3	0.38
4	0.71
5	0.62
6	0.33
7	0.71
8	0.57
9	0.43
10	0.67
11	0.50
12	0.47
13	0.32
14	0.58
15	0.50

Years as Primary Starter

Each DE's total years as one of their team's primary starter is a good indication of a draft pick's worth. The tables below break out years as primary starter by round and draft order.

Rnd	Yrs as Primary Starter
1	5.6
2	3.2
3	2.7
4	2.4
5	1.2
6	1.2
7	0.6

DE Pick	Yrs as Primary Starter
1	6.8
2	6.5
3	4.0
4	4.3
5	4.1
6	2.6
7	3.6
8	1.8
9	2.2
10	3.0
11	2.7
12	1.6
13	1.2
14	2.4
15	0.6

Conclusion

It appears that, on average, each draft provides two DEs in the first round who have a high probability of being an above-average player. But compared to QBs and RBs, DEs taken in later rounds may have a better opportunity to excel.

Drafting RBs

Can solid running backs really be found anywhere in the NFL draft? Years ago the conventional wisdom seemed to be that a team needed a superstar RB from the first round to win consistently. Now it seems that the conventional wisdom is that teams still need a star ball carrier, but one can be found deep in the draft. So which is it?

Data

The data consists of RB draft picks from the 1980 through 2000 drafts found at Pro-Football-Reference.com. Running back career performance was judged three ways. First, I averaged the likelihood a RB would be selected to one or more Pro Bowls by round and draft order. Second, I averaged career Yards Per Carry (YPC) by draft round and by draft order. (I also tried various ways of including receiving yards, but the variance in Yards Per Reception is very large and it distorted the data, particularly for players with relatively few receptions. Ultimately, simple YPC worked best and aligned closest with how most people see RBs. That is, Steve Sowell and Dave Meggit aren't ranked above Barry Sanders, Thurmond Thomas, or Emmit Smith.) And lastly, I averaged the number of years as the primary starter by round and by draft order.

Pro Bowl Selection

Although Pro Bowl selection is a flawed measure of career performance in many ways, it can indicate that a draft pick has "panned-out." If you sort the data by PB selection, it very quickly separates the generally productive RBs from the "three yards and a cloud of dust" guys. After looking at PBs for a number of positions now, it seems that 2 or more PB selections is a particularly good measure of career productivity, especially when judging top draft choices.

The graphs below illustrate the likelihood that a RB will be selected to one or more, two or more, and three or more PBs. The first graph is by draft round, and the second graph is by RB draft order (i.e. 1st RB taken, 2nd RB taken, etc.)




I wouldn't read too much into the spike at the 5th RB taken. It's likely just a statistical quirk, but it might be one reason why many experts believe that later round RBs are as good as early round picks.

[Edit: Some have asked why I brushed off the spike of Pro Bowls at the 5th RB taken as a quirk. If we analyze enough draft picks for various positions, as I'm in the process of doing, we're bound to see a significant bunching like this by chance once or twice. The graph is relatively continuous except in one place, where there is a depressed result at the 4th and 6th pick and the spike at the 5th. What is likely at work is that positive results in the 5th pick "bin" have randomly "stolen" positive results from the 4th and 6th bin. There were 20 RBs taken as the 5th RB in the data set, so it would only take 2 or 3 RBs who would otherwise have been the 4th or 6th pick to be bunched into the 5th pick to give us this result. Unless we had a reason to believe there is some special quality about the 5th RB taken before seeing the results, we should not interpret the data to say there is something magical about being the 5th RB taken.]

Yards Per Carry

There is probably no simpler and truer measure of running back performance than yards per carry. Of course, YPC does not belong to the RB alone. For any one RB's season, offensive line ability has a tremendous influence on his stats. But over 490 careers and over 24 years of data, the abilities of offensive lines will average itself out to a great degree, leaving career YPC a reliable estimate of true RB performance when grouped by round or draft order.

As with QBs, the biggest question is how to score draft picks with no carries or very few carries. RBs with fewer than 200 career carries tended to have extreme YPC stats. I assigned them the YPC of the 5th percentile qualifying RB, which was 3.58.

The two graphs below break out career YPC by draft round and by draft order.

The first round RBs, particularly the first couple taken, tend to significantly outperform later picks. By the 3rd or 4th round and the 7th RB taken, teams are likely getting sub-replacement level special teams fodder.

Also notice the nearly 1:1 relationship between career YPC and PB selections, including the spike at the 5th RB taken. This suggests that PB selection is merit-based and is a reasonable proxy for grading career performance.

Scouting Accuracy

How often do the scouts get it right? In other words, how often does the higher pick turn out to be better than the next pick? The two table below lists the likelihood that the higher pick will have a better career YPC than the next RB taken in the same draft. We shouldn't expect the scouts to be perfect, but this table tells us how difficult it is to predict the better player.

RB Pick	Pr(Better)
1	0.62
2	0.67
3	0.57
4	0.48
5	0.55
6	0.33
7	0.43
8	0.33
9	0.33
10	0.38

Years as Primary Starter

Another way of judging the value of draft picks is the number of years the pick has served has his team's primary starter. Top RB picks, especially the first two each year, tend to serve much longer as a primary starter.

Round	Yrs as Primary Starter
1	4.1
2	2.7
3	1.5
4	0.7
5	0.8
6	0.6
7	0.5

RB Pick	Yrs as Primary Starter
1	5.0
2	5.1
3	3.2
4	2.4
5	3.7
6	1.6
7	1.9
8	1.4
9	2.7
10	1.0
11-22	0.8

Conclusion

Top picks solidly outperform subsequent picks. The top two RBs taken tend to almost be in a class to themselves, then there is a steady decline in expected performance until the 8th RB taken, at which point there is very little to be expected from a pick.

So do teams need a superstar #1 pick RB to win, or can they find a premier runner deep in the draft? Which conventional wisdom was right? My theory is it's neither.

I think most people still grade RBs in terms of total yards, whether it's for a single game or for a season. Even though it should be well known now that winning leads to running, rather than vice versa, commentators and analysts continue to count 100 yard games or 1000 yard seasons as measures of RB effectiveness.

But even below-average RBs on winning teams with good passing games and good defenses will tend to accumulate large chunks of total yards due to frequent carries. Even a RB who was a 5th round pick on a great passing team will appear much better than he truly is. I believe that might explain the perception that solid RBs can be found anywhere in the draft.

The better RBs really do come from the top picks. It's just that they're not that important, or at least they're not as important as they were in the 1970s before the NFL became a passing league. Plus, our understanding of which RBs are truly the very good ones is distorted by analysts who insist on total yards as the best measure of RB performance.

Drafting QBs 2

My last post looked at how likely NFL quarterbacks are to become Pro Bowl selectees based on their draft order. This post will directly analyze QB performance by draft round and order based on passing stats. Ultimately, the passing stats will be converted into average expected team wins through linear regression. The result reveals how many more wins per year a 1st round pick could be expected to deliver than a 2nd round pick, or the 1st QB taken over the 2nd.

How QB Performance Was Measured

Passing performance is measured by career average yards per attempt and interception rate. These stats are combined into a single measure, commonly referred to as Adjusted Yards Per Attempt (AdjYPA). AdjYPA basically turns every interception into a -45 yard pass play. This is a commonly accepted equivalence based on research going back to a 1988 book called The Hidden Game of Football. It's also somewhat intuitive--An interception basically precludes the possibility of a punt, which is typically about 45 yds before the return. Additionally, 10 yards of passing is added for every touchdown thrown. This adjustment is intended to compensate for the truncation of the field and the added difficulty in the compressed field near the goal line.

The Data

Data were obtained from Pro-Football-Reference.com. Quarterbacks drafted from 1980 through 2000 in the first 7 rounds were studied. Players from round 8 or later were excluded because the current draft is limited to 7 rounds, and the point of this analysis is prescriptive. Supplemental picks were not included because the focus here is on draft day itself.

Adjustment for Era

Because the passing game in the NFL has evolved over the years (it's become steadily easier), an adjustment was made for year. Average AdjYPA has increased signficantly between the draft classes of 1980 and 2000. The 1980-1983 draft classes averaged 5.6 AdjYds per attempt compared to an average of 6.0 AdjYds per attempt for the 1998-2000 draft classes. The increase overall in the NFL was a fairly steady 0.13 yds per year. The midpoint of each QB's career was used to adjust all QB's stats as if they played in 2004.

Draftees Without Many Attempts

One big problem about this kind of draft analysis is how to score drafted QBs who never played. If they are assigned zero yards per attempt, an unrealistically low record, it would severely weigh down the averages of all but the very top picks. If they are excluded from the data, then the true expected value of QBs drafted in later rounds would be severely biased upward. QBs who played but did not have many pass attempts are also problematic. They can have erratic stats, some having an insanely high 15 yards per attempt or as low as -20 yds per attempt.

My solution to the problem of QBs with too few attempts was to assign them the stats of the 5th percentile qualifying QB. I chose 200 attempts as the qualifying level because it's where the the stats settle down to reasonably steady and apparently representative levels. The 5th percentile makes sense because it would be unfair to say every QB that didn't get their shot would be as bad the very worst to play. Although many of them would undoubtedly be pretty bad, some of them were simply bottlenecked behind slightly better QBs.

Thus we now have the Wuerffle Line, football's very own version of the Mendoza Line. The 5th percentile falls between Danny Wuerffle and Akili Smith. Even though I think Danny is a good guy, the "Smith-Line" just doesn't resonate.

Wins Added (per year)

Now we have a good measure of QB performance that considers passing efficiency, interceptions, and career era, but we're left with an abstract, indirect stat--AdjYPA. The bottom line for every NFL team is wins and not passing stats, so I converted the AdjYPA stat into expected team wins per season.

I ran a regression of all teams from the 2002-2006 seasons that weighs offensive and defensive passing stats, running efficiency, turnover rates, and penalty rates. The model was very similar to the one I ran here. AdjYPA was highly significant, the residuals were randomly distributed, and the model's overall r-squared was 0.72. By holding equal all the factors other than the passing stats, we can see how much passing contributes to team wins. For every 1 yd increase in AdjYPA by a QB, a team can expect to win an additional 1.4 wins each season.

A reference point is necessary when comparing QB to QB, so I borrowed an idea from baseball sabermetrics--the replacement player. But since there is no replacement QB across 21 years of draft picks, I used the 5th percentile again as the reference point. The number of wins added above the 5th percentile was calculated for each QB draft pick. Instead of "Wins Above Replacement," I termed this stat "Wins Above Basement (WAB)." The reference point itself is arbitrary, but it's needed to compare QBs from one round to another.

The first graph below illustrate how many Wins Above Basement a QB could be expected to have based on draft round. The average first round pick earns about 0.3 wins per year more than the average second round pick.
As you can see, the relationship between round and expected QB performance is very linear. But as the next graph shows, not all first round QBs are created equal. The next graph shows wins added by QB draft order. For example, the first QB taken in each draft earns about 2.4 wins per year (more than the worst QBs), and the second QB taken earns about 2.0 wins per year.
Again, we see a very linear relationship. But there may be something more. There are large drop offs in performance from the 1st QB taken to the 2nd, and from the 2nd to the 3rd. Then from there until the 9th or 10th QB taken, it's pretty random. It appears that if your team doesn't get one of the first two QB picks, it might as well take a chance on a later pick. Chances fall off quickly after the first two QBs that a team will find a franchise player.

Overconfidence

Teams often jockey their draft position, even by just a few spots, to ensure they can pick a particular player. Part of the reason they do this may lie in their overconfidence in their ability to identify the better player, a point made in the Massey-Thaler paper. Sometimes players taken later in the draft turn out to be superior to the players taken early.

The table below lists the probability that each QB taken will end up better than the next QB taken in the draft (based on AdjYPA). For example, the 1st QB taken in the draft has turned out to be better than the second QB taken 81% of the time, and the 2nd QB taken has been better than the 3rd 38% of the time. (Note that some of the lower probabilities are due to the high number of consecutive QBs in the later rounds who did not have enough qualifying attempts. They are considers "ties.")

QB Pick	Pr(Better)
1	0.81
2	0.38
3	0.52
4	0.38
5	0.50
6	0.28
7	0.31
8	0.54

It appears that after the first QB taken, there isn't much certainty among GMs in predicting who will turn out to be the better passer. Teams are jockeying draft position, and paying a price to do so, for very small marginal probability of picking the better player. Perhaps there are times when the quality level drops off sharply between a QB and the subsequent one on the draft board, and trading up makes some sense. But this table should give GMs pause when considering whether to trade away next year's second round pick to move up 7 picks in the first round.

The data really tells two stories. Overall, when taking all 21 years of picks into account, the higher picks tend to have significantly better performance compared to subsequent picks. But when you look at the data in any one year, the differences aren't so clear, especially after the second QB taken.

Years as Primary Starter

Another way to grade draft picks, aside from Pro Bowl appearances or actual performance stats, is their years as their team's primary starter. The table below breaks out the average number of years a QB draft pick will serve as their team's starter by draft round.

Round	Yrs as Starter
1	5.6
2	4.0
3	1.8
4	1.0
5	1.7
6	1.6
7	0.2

Here is the same data, this time broken out by draft order.

QB Pick	Yrs as Starter
1	6.4
2	3.9
3	2.3
4	2.6
5	2.0
6	1.2
7	1.4
8	1.8
9	1.2
10	0.2
11	0.1
12	0.0
13	0.5
14	0.0

Years as starter may be a flawed comparison, however. Opportunity is everything, especially for QBs in the NFL. Top picks would certainly benefit from disproportionate opportunities. The years as starter figures for 1st round picks may be inflated because there are nearly no zeros. Teams might also stick with poorly performing QBs who were top picks longer than they deserved because of sunk costs. I think the better measure is what players do once they get their chance on the field. (Now that I think about it a little more, a better idea may have been to compare "years as starter beyond the first year.")

Drafting QBs

If a team is in the market for a young quarterback, should it trade up and grab someone a spot or two higher on their draft board? Or should they stand pat and take the best QB that falls to their own pick? There are many considerations to weigh, including:

• What will they have to trade to move up in the draft?
• How much cap space will they have?
• How likely is it that a higher rated QB will turn out to be better than another further down the draft board?

Perhaps the last question is the hardest to quantify. As fans, we're frequently reminded by commentators how Tom Brady was a 6th round pick and how Ryan Leaf was rated as highly as Peyton Manning. Ok, but what about the dozens of low round quarterbacks who stunk up the league, or the many first round picks who are now in the Hall of Fame?

Quantifying career-long performance is more difficult than I expected. For starters, QBs playing over different NFL eras need an adjustment to their stats. Second, how do we measure a QB's total performance? Career totals? Yards per attempt and interception rates? Both methods give different results. If we use career totals, a slightly above average QB with a long career might outshine a great QB who retired sooner. But then again, longevity should count for something. If we use rate stats, there are a number of QBs with outlier performance stats simply due to small numbers of pass attempts.

We'd need to use a minimum number of qualifying attempts. But then that defeats the purpose of the analysis. QBs who aren't even good enough to win a starting job should be counted when we're comparing draft outcomes. If they're excluded we'd have a severe selection-bias in the data because only the "diamonds in the rough" of later rounds will remain, obscuring the true worth of a late-round QB.

I'll address those issues in a later post, but for now I'm taking a cheap and easy solution--I'll rate QB draft picks by how many Pro Bowls they've been selected to. This method avoids the problems of normalizing stats across eras and figuring out how to handle QBs without a significant number of pass attempts.

With data from Pro-Football-Reference.com (which has a great draft database) I compiled a list of all QB draft picks from 1980-2000. I chose those years to limit the QBs to the modern era with the post-1978 passing rules, and to give enough time for the class of 2000 to develop and be assessed. I counted how many times each QB had been to at least one Pro Bowl (PB), at least two PBs, and at least three PBs. One PB might indicate a flash-in-the-pan guy, such as Gus Frerotte, but two PBs probably indicates sustained excellence. Three or more PBs would make any GM happy.

The table and graph below show the likelihood that a QB drafted in each round will pan-out and be selected to at least 1 Pro Bowl (1+ PB), at least 2 PBs (+2 PB), and at least 3 PBs (+3 PB). The data set is not large (only 193 QBs total), so the graph contains smoothed lines for a more realistic estimate of future expected performance from each round.

Round	1+ PB	2+ PB	3+ PB
1	0.38	0.28	0.23
2	0.32	0.21	0.16
3	0.13	0.04	0.00
4	0.06	0.03	0.03
5	0.10	0.10	0.10
6	0.14	0.11	0.06
7	0.03	0.00	0.00

Historically, moving up from the second round to the first round for a QB buys you a 16% to 23% improvement in the chance he'll go to at least three PBs. It would also buy you a reduction from a 68% to 62% chance he'll never go to a single PB.

Looking at draft picks by round may not be the best approach. What round someone is selected in may have more to do with team needs and other factors aside from how much potential a prospective QB has. Looking at QBs in terms of which QB pick they were in their draft class might be helpful. For example, although Tom Brady was taken in the 6th round of 1999 and Brock Huard was taken in the 3rd round of 2000, they were both the 7th QB taken in their respective years.

The table and graph below list the same information as above--how often a drafted QB is selected to a PB. This time the QBs are broken out by which QB pick they were in their draft year. Again, smoothed lines are added.

QB Pick	1+ PB	2+ PB	3+ PB
1	0.38	0.38	0.33
2	0.29	0.14	0.05
3	0.14	0.10	0.10
4	0.19	0.10	0.10
5	0.19	0.10	0.00
6	0.11	0.06	0.06
7	0.17	0.17	0.17
8	0.19	0.06	0.00
9	0	0	0
10	0	0	0
11	0	0	0
12	0	0	0
13	0	0	0
14	0	0	0

There appears to be a very large drop-off in expected performance from the first QB taken to the second. A third of the first QBs taken went to three or more PBs, but only 5%-10% of the second through sixth QBs taken went to three or more.

I interpret these results as an indication that unless a team can get the very first QB or at least the second taken in a given year, it shouldn't expect to find a franchise player. It does happen, but it's very rare. From the 3rd QB taken through the 7th, there's no apparent difference. If a team doesn't get one of the first two, it might as well wait until later rounds and take a chance on a later pick or set it sights on a free agent. Meanwhile, it can fill other needs.

Of course, Pro Bowl selection is not the best measure of a QB's performance--it's subjective, somewhat arbitrary, and it's an all or nothing measure. But it does help answer a the question all GMs ask when drafting a QB--how likely is it this guy going to pan out? Next, I'll attempt to quantify just how much better top picks are in terms of team wins.

How to Beat the Point Spread

Last season I received several comments asking me to make game predictions against the spread. I tend to resist doing spread predictions. I'm not interested in gambling myself--I just don't have that gene. Plus, I'm more interested in analyzing the sport of football for its own sake rather than exploiting it for money. But I will oblige some of my readers (and my own curiosity) in this post.

As a basis for my investigation of office pool strategies, I've recently developed a database of games that includes spread data, so I really couldn't help seeing if I could beat the spread with any regularity.

Additionally, this post at Sabermetric Research blog suggested that most NFL games have obvious favorites, so there are only a few dozen games per year that experts could reasonably disagree about the likely winner. Picking winners straight up may not be the best way to test football expertise because it would take several years of games for true expertise to overcome random luck. (But I'm not convinced that picking against the spread is any better).

I experimented with a linear regression that predicted actual point spreads. I also tried a logistic regression (binary outcome regression) of whether a team would beat the spread. Neither worked very well. The method I ultimately stumbled upon predicted correctly against the spread for about 60% of the 2007 regular season, which seems to be pretty good. According to this site that tracks over 50 handicappers and systems, my method would come out on top. It's only one season of results, but there is a solid theoretical foundation to why it works.

One of my recent posts discussed how defensive interceptions appear to be totally random. Teams cannot sustain high or low interception rates throughout the season. I thought that if handicappers and bettors are factoring in defensive interceptions in their spreads, than you'd do well to bet against teams with high to-date interception rates. Unfortunately, that theory was a bust. It beat the spread only 50% of the time.

Then I remembered a comment from a reader early in the season last year about how teams that win outright also beat the spread over 80% of the time. (It's actually 82.7% over the past six years.) So a system that picks SU winners 70% of the time could beat the spread over 70%*80%=56+% of the time. It turns out that my SU model is actually a pretty good ATS system too.

(This applies to the 2007 regular season weeks 4 through 16. I needed a few weeks of data for reasonable predictions, and I excluded week 17 because of its notorious unpredictability due to playoff teams resting their starters. (There's no way Troy Smith leads the 2007 Ravens over the Steelers in any other week.))

The table lists the number of games and the accuracy of my SU model in predicting the ATS winner for each spread size. For example, in 2007 there were 31 games when the spread was either +3 or -3. My SU model was correct in predicting the ATS winner in 61% (19) of those games. The cumulative number of games and ATS accuracy is also listed.

Spread	Accuracy	Count	Cm. Count	Cm. Accuracy
0	0.50	4	4	0.50
1	0.83	6	10	0.70
1.5	0.71	7	17	0.71
2	1.00	3	20	0.75
2.5	0.25	8	28	0.61
3	0.61	31	59	0.61
3.5	0.56	27	86	0.59
4	0.83	6	92	0.61
4.5	0.50	6	98	0.60
5	0.00	1	99	0.60
5.5	0.80	5	104	0.61
6	0.33	9	113	0.58
6.5	0.80	5	118	0.59
7	0.56	9	127	0.59
7.5	0.75	4	131	0.60
8	0.50	6	137	0.59
8.5	0.50	4	141	0.59
9	0.80	5	146	0.60
9.5	0.80	10	156	0.61
10	0.33	6	162	0.60
10.5	0.33	9	171	0.58
11	0.00	1	172	0.58
11.5	1.00	2	174	0.59
12.5	1.00	2	176	0.59
13	1.00	1	177	0.59
13.5	1.00	1	178	0.60
14	1.00	1	179	0.60
14.5	0.00	1	180	0.59
16	0.75	4	184	0.60
16.5	1.00	1	185	0.60
17+	0.00	4	189	0.59

Overall, the SU model is 59% against the spread. Although that's pretty good as is, if we cut off predictions at spreads of 5 or 5.5 points, the model is 60 to 61% accurate ATS. We could go out to 9.5 points and still have 60% accuracy. That includes 156 games, hopefully enough 'action' for those gambling enthusiasts out there.

Keep in mind I just stumbled on this, so the model is not optimized for ATS predictions. By tweaking it here and there, or better refining which games have the highest probability of beating the spread, the ATS accuracy could be further improved.

I realize there are thousands of handicappers out there who claim higher accuracy rates. But personally, I don't believe them. Statistical slight of hand is not hard, especially when you control access to the data. I'd guess that most of them are cherry-picking results after the fact. Here's one example:

1. A handicapper always has 3 sets of picks, say his "guaranteed plays," his "locks of the week," and "his gold specials." He doesn't necessarily advertise which picks are in which category.
2. As the season goes on, naturally one of the three sets will outperform the others. Even if his picks overall are worse than 50%, there is a very good chance one set of picks will be better than 50%.
3. On his website, the handicapper advertises that his "guaranteed plays" are 21-13...over 60% accurate!

Another factor is Darwinism. There are new handicappers that hang up their shingle all the time. Naturally, just by luck alone, half will be better than 50% and half will be worse than 50%. The unlucky ones will tend to close up shop, leaving the luckier handicappers in business and claiming to be geniuses. (This likely also applies to fund managers on Wall Street, which is one reason I prefer index funds.)

This result is only based on one year of games, but I'm very confident it is repeatable. For 2006, I ran a very similar model and it beat the spread as a SU predictor by several percentage points. That's really all you need. If you can significantly beat the spread as a SU predictor, you'll have about a 60% accuracy ATS.

Well, you commented, and I listened. There's my ATS post. I've focused on gambling enough for a while. I feel like I need a shower. Like I said, it could probably be even more accurate with some tweaks, but my attention over the next few weeks will be on the upcoming draft.

Announcement: NFL Stats PREMIUM

I've really been overwhelmed by all the attention this site has received. I appreciate all the fan mail, offers to intern, and marriage proposals. So to better be able to serve my readers, I've decided to create a premium zone for NFL Stats. Premium subscribers will have unlimited access to proprietary statistical analysis found nowhere else. Subscribers will also receive my "Lock of the Week" which is guaranteed to beat the spread*. But don't worry, regular members will still have access to past research, plus free** tours of the nerve-center of the NFL Stats Institute for Advanced Football Research (shown at left) here in Reston, Virginia.

Become a member today. If you subscribe now, you'll receive a 10% discount and a free NFL Stats tote bag. All this for a low rate of only $99 per month. Subscribe today, operators are standing by.

(*Note: Guarantee not an actual guarantee. **Parking is $39.)