Crunching the Numbers on the Quarterbacks

If you remember a while back, I proposed a formula for evaluating quarterbacks. It was:

Rating = (yards passing + yards running -yards lost by sack + 20*Touchdowns -30*Turnovers)/(# of designed passing plays).

To recap, the main features of my rating are that it:

• cuts down the credit given for completing passes for short gains,
• rewards passers who avoid sacks and get positive yardage on the ground,
• penalizes turnovers more than does the conventional passer rating, and
  
• expresses the result in an easily comprehensible yards-per-passing-play format rather than a completely incomprehensible format.
  

Well, there's a mild problem with this formula. The statistics available to me do not differentiate between sacks, scrambles, and designed runs. They're all counted as "carries", which is incredibly dumb, especially for sacks, which are clearly and obviously yards lost in the passing game. I also have no statistics on fumbles, so I just have to skip those. I wanted to evaluate the 2006 SEC quarterbacks, so I evaluated them without differentiating between the types of runs. The results are below. I only included quarterbacks who started. Because I couldn't figure out how to make a usable table out of this, I'll just list the results:

Jamarcus Russell, LSU, 9.16
Blake Mitchell, S. Car., 8.26
Erik Ainge, Tenn., 8.07
Andre Woodson, UK, 7.79
Chris Leak, Fla. 7.02
Syvelle newton, S. Car. 7.00
Casey Dick, Ark., 6.99
Chris Nickson, Vandy, 6.55
Brandon Cox, Aub., 6.37
John Parker Wilson, Bama, 6.14
Omarr Conner, MSU, 6.13
Michael Henig, MSU, 5.82
Matthew Stafford, UGA, 5.78
Mitch Mustain, Ark. 5.68
Jonathan Crompton, Tenn., 5.35
Brent Schaeffer, Ole Miss, 4.27

I can only represent to you that I devised the formula and wrote the previous post without knowing the results. I did not know that my formula would result in Jamarcus Russell being in a class by himself. I thought he might be, because I think he really WAS in a class by himself. The thing is, even if you use conventional passer ratings, he was still in a class by himself. He was the highest rated, with a 167 rating. Woodson was second with a 154 rating. By either measure, Jamarcus was clearly the most effective quarterback in the SEC.

It's not an artifact or an accident. He was second in the league in yards and in touchdowns, behind Andre Woodson in both, who had almost 80 more pass attempts. Speaking of Woodson, Kentucky put the ball in his hands a LOT. If he hadn't been sacked so many times, he would have been fairly close to Jamarcus. On the other hand, one of the strengths of Jamarcus's game is the fact that he is so hard to sack. As a result, he finished almost 1 full yard/play ahead of every other starting quarterback.

If Kentucky could run the ball, they'd be dangerous.

Who got hurt the most by my changes to the formula? Brandon Cox, Andre Woodson, and John Parker Wilson. Cox and Wilson lost a lot of yards to sacks, and Wilson had a relatively poor touchdown to interception ratio, which is underpenalized in the conventional formula. Cox, by the way, was the least effective veteran starter in the SEC. No one who had been a regular starter in 2005 performed worse than Cox. Who got helped the most? Probably Chris Nickson of Vandy, whose 9 touchdown runs really helped his rating. However, he also gave up a lot of points by throwing a lot of interceptions.

I also think it's interesting that under either measure, Brent Schaeffer was the least effective quarterback in the SEC last year. I'm careful in my wording here. I don't say "worst". I say "least effective". Obviously, nobody on the football field operates by himself. A quarterback is dependent upon his blockers and his receivers to give him the opportunity to be successful. Therefore, no single statistic can measure how "good" a quarterback is, independent of the players around him. We can only measure how "effective" he is, which is dependent on the other personnel. This isn't to say that ratings are independent of skill. Clearly they are not independent, but we have to recognize the limitations inherent in this formulation.

Notice also where the true freshmen are: very near the bottom. This goes to show something I will touch on again. It's never a good idea to start a true freshman at quarterback.

What does all this mean for next year? Well, the top returning starters are Tim Tebow (who isn't included in my chart, but he was very effective in his time on the field, but will have to show it every down) and Blake Mitchell.

Blake Mitchell? I have to admit that was a surprise. Wasn't he benched? He has a very solid yards/attempt number and doesn't lose a lot of yards in the running game. His TD/interception ration is weak, but he made up for it in other areas. Even under the conventional formula, he's solidly mid-pack among SEC starters (higher than Leak, for example). When you consider that both Woodson and Ainge lost yards running and Mitchell didn't, Mitchell starts to look a lot better.

Blake Mitchell was statistically the second most effective starting QB in the SEC in the time he was in the game, and got benched in favor of Syvelle Newton, who incidentally wasn't nearly as effective. I'm willing to accept the proposition that Mitchell's stats are inflated by the fact that he had Sidney Rice, but that doesn't explain why Newton was less effective.

Players to watch for next year: Ainge, Mitchell, and Woodson. If Cox can move his legs, and if Auburn has any receivers, he should be very effective next year.

I'll definitely be keeping up with this during the season. If I keep up with it game-by-game, I may even be able to refine the numbers by getting better statistics on quarterback fumbles. It'll be interesting to see how young QBs like Casey Dick and John Parker Wilson progress.

Passer ratings?

Most football fans pay attention to statistics, at least a little bit. Some statistics are very basic and understandable even by a novice. Others are a lot more complex and obscure. Some positions (offensive line) have no real statistics to gauge performance. Most of the stats you see are of the simple variety. It's pretty easy to know what "yards" means when talking about a running back. It's pretty easy to know what "completion percentage" means when talking about a quarterback. The "passer rating" falls into the opposite category.

How many times have you seen "passer rating" or "quarterback rating" discussed or mentioned? You've probably heard it quite a lot. How many times has anyone in the media actually explained what it is? Personally, I can't swear I've ever heard a television talking head actually discuss how passer rating is formulated. I don't understand why this is so, but I am here to educate you about it.

College and pro football each have their own ratings, and you can't really compare the numbers to each other. I'll focus on college. For college football, here is the formula for passer rating:

(100*Completions/attempts) + (8.4*yards/attempts) + (330*Touchdowns/attempts) - (200*Interceptions/attempt) = Passer rating

That clears it all up for you, doesn't it? Well, let's do some algebra and go a little deeper. If you do some factoring, here's what it comes down to:

8.4*(Yards passing + 11.9*Completions + 39.3*Touchdowns - 23.8*Interceptions)/(Number of attempts) = Passer Rating

I think this is a much more useful way to show it. It shows the rating for what it is. It's a measure of yards per pass attempt, with certain bonuses and penalties. There is a 39.3 yard bonus for throwing a touchdown, a 23.8 yard penalty for throwing an interception, plus a weird 11.9 yard bonus every time the ball doesn't fall incomplete. All of that is then multiplied by 8.4, apparently for no other reason than to multiply by some arbitrary number.

Part of this makes a lot of sense. Yards per attempt has a pretty clear meaning to a quarterback, and I think everyone can grasp its importance in evaluating a player. Likewise, the number of touchdowns and the number of interceptions a player throws are also extremely important.

There are a number of weaknesses in this statistic, however. The most important weakness is that it does not reward a quarterback for escaping the rush and gaining yards on the ground, a key part of many modern quarterbacks' games. Consider the following scenario:

Quarterback drops back to pass. The defense gets a good rush and the quarterback escapes out of the pocket. He continues looking for his receivers, but the receivers aren't getting open and the QB will be sacked if he doesn't do something. Should he a) stand there and take a sack, or b) try to get out of the pocket and run for positive yardage? Assume that playclock effects are not important at this stage of the game.

Any good quarterback will try to run for it and get whatever he can get. Some will be better at it than others. However, the passer rating will not reflect these differences. As far as the passer rating is concerned, the quarterback who runs for an electrifying 50 yard touchdown will get the same result as the one who trips over his own feet in the backfield.

This problem can and should be fixed by taking out the "yards per attempt" and using instead a "yards per designed pass play", which would include passing yards as well as yards gained or lost on the ground, with the number of attempts changed to include all times the quarterback drops back to pass. Go ahead and count rushing touchdowns among the number of touchdowns as well.

Another weakness is that second term. I don't understand the second term, which seems to reward a quarterback simply for completing a pass, even if the pass made no yardage, or even lost yardage. Consider the above scenario, but changed in this way:

Quarterback drops back to pass. The pocket collapses before the receivers break open. The quarterback is surrounded, unable to escape the pocket. Should he a) stand there and take a sack, or b) throw the ball away? Assume that playclock effects are not important at this stage of the game.

While no quarterback will simply take the sack in this situation if he has a choice, his Passer Rating will suffer for his decision to throw it away, because it will increase his number of pass attempts. In that scenario, your passer rating will stay the same if you simply take the sack. In fact, you can actually improve your passer rating by throwing a completed pass that loses yards, so long as it loses less than 11.9 yards.

I'm not suggesting that a quarterback, mindful of his "rating", will deliberately take an action that will hurt his team. I am merely saying that in this common situation, where a player has two choices, the primary statistic that gauges his performance will actually suffer if he makes the correct choice. That, to me, is a serious deficiency in the quarterback rating, and renders it almost unusable.

I would recommend removing this term and instead have a term that gives a quarterback a bonus not for simply completing a pass, but for gaining a first down. After all, it's a lot better to get 11 yards on 3rd-and-10 than it is to get 9, but your passer rating will hardly notice the difference.

Lastly, I don't like the magnitudes of the bonuses. Is a touchdown really worth the same as a 39 yard pass? Is an interception the equivalent of 24 yards lost? I don't buy it. For one, it means that a quarterback who throws almost twice as many interceptions as he does touchdowns is about as good a quarterback as one who throws none of either. Of course, you'll be hard-pressed to find a quarterback who throws neither touchdowns nor interceptions, but it implies a quarterback who throws 1.65 interceptions for every touchdown pass is statistically "neutral". And here I was thinking that Rex Grossman's 23 touchdowns and 20 interceptions last year was bad. Shows you what I know; his passer rating was helped more by the touchdowns than it was hurt by the interceptions. A lot more.

When you hear about Passer Ratings, realize that it is a measure of yards per pass attempt, with a 40 yard bonus for touchdowns and a 24 yard penalty for interceptions, plus another factor that makes little sense, all multiplied by an arbitrary number. Realize also that it fails to reward quarterbacks who help their teams through running out of trouble. Realize further that it actually penalizes a quarterback for throwing the ball away to avoid a sack, and artificially puffs up a quarterback who throws a lot of short completions. I don't like it as a measure of quarterbacks.

I prefer giving the quarterback a 20 yard bonus for a touchdown, and a 30 yard penalty for an interception. It means that a QB who has a 1.5:1 touchdown to interception ratio is statistically neutral in that regard, and I think it more accurately reflects the field position lost be a typical interception. And while we're at it, let's count the times a quarterback loses a fumble the same way we'd count an interception. It just makes sense.

Here's the formula I would use (I'll take out the 8.4 factor just on principle):

Passer Rating = (yards passing + yards rushing - yards lost by sack + 5*First down completions and runs + 20*Number of Touchdowns - 30*Number of Turnovers)/(Number of called pass plays).

My formula ha the features I talked about, plus it gives a 5 yard bonus for making a first down. After all, a pass that goes for 11 yards on 3rd-and-10 is much better than a pass that goes for 9 yards, but the conventional passer rating will hardly notice the difference.

It all makes so much more sense to me to use this measure than it does to use the traditional Passer Rating, but it isn't going to happen. This formula isn't even particularly useful to me because they don't keep track of 1st down throws (and runs). I can't compile these numbers even for my own benefit. I can, however, use it if I take out the 1st-down bonus, leaving the following formula:

Passer Rating = (yards passing + yards rushing - yards lost by sack + 20*Number of Touchdowns - 30*Number of Turnovers)/(Number of called pass plays).

Next time, we start compiling some numbers.