Excellent article on the QB Rating system

[Darkiller]

this is pretty good and interesting to read :



The leading rusher in the National Football League is the player who carries the ball for the most yards. Last year it was Edgerrin James, with 1,709. The league's receiving champion is the guy who catches the most passes. Marvin Harrison and Mushin Muhammad tied at 102.

To learn that Denver's Brian Griese was the NFL's top passer last season, it helps if you didn't leave college early for the draft. Here's the league's official rating formula that made Griese number one:

QB rating = {(C/A – 0.3)/0.2 + (Y/A – 3)/4 + (T/A)/0.05 + [0.095-(1/A)]/0.04}x(100/6)

As you can see, Griese's rating was precisely 102.9. So of course he was the top quarterback. 102.9! The guy's on fire! In fact, give the boy some ibuprofen, it looks like he's running a fever!

What the hell is the meaning of 102.9? Most sports numbers tell a story. Twelve under par. A first round knockout. A $50,000 fine for calling David Stern a wiener. Batting .300 means you got three hits every ten at bats. But 102.9? Isn't that the FM station that plays Steely Dan all day? Why do we measure the most important player on the football field using a quadratic equation so puzzling it's actually used as the opening problem in the math textbook College Algebra (8th Edition, Addison Wesley, 2000)?

"Other than one attorney in our office, I am unaware of a single human being who has the capacity to figure a quarterback rating," says agent Leigh Steinberg, who has negotiated multimillion-dollar contracts for NFL quarterbacks for 26 years.

"I know interceptions kill your rating, but if you asked me to compute it, I'd have no idea," admits Steve Young, who besides having history's highest career (96.9) and single season (112.8) passer ratings is also lawyer and the chairman of a Silicon Valley software company.

Does it help to know that, in Fermat's theorem above, A is pass attempts, C completions, Y passing yardage, T touchdown passes, and I interceptions? Uh, a little. Does it help to know that this is technically only meant to be a passing-efficiency statistic, not really a quarterback rating? It doesn't try to account for other useful quarterbacking activities, like running -- or winning.

Does it add perspective to understand that the "average" rating was designed, according to circa 1970 standards, to be precisely 66.7? Or that the maximum, "perfect" rating is a freakish 158.2 -- and that eight quarterbacks have tossed perfect games, the latest being Kurt Warner last October when he didn't actually complete every pass he threw? Does that help give meaning to 102.9? Yes, we thought so: a really, really little bit.

Few topics have done more to brings jocks and nerds together than mutual fear and disbelief of the passer rating formula.

"Their main problem was that no one tried to figure out mathematically how the formula worked," says Pete Palmer, a sports statistician whose book The Hidden Game of Football. blasts the calculus of the NFL formula. "You get what I think are unreasonable results."

"I pay attention to my rating on third down and in the red zone," says Trent Dilfer, who despite his wobbly 76.6 rating last season quarterbacked the Ravens to a Super Bowl victory. "Otherwise, it's most useful for fantasy football people who are more concerned with numbers than good old-fashioned winning."

So how did we get here?

"They asked if I had any ideas about rating passers. I did," says Don Smith, who invented the formula. "I'm the guilty guy."

It's time to go behind the jockstrap and examine the glory that is the NFL passer rating system.

The year was 1971, and the NFL had merged a year earlier with the 10-team hippie upstart American Football League. One of the issues facing commissioner Pete Rozelle -- albeit not the most burning one-- was standardizing official statistics. The leagues, and even individual teams, had been issuing a hodgepodge of stats. Grooviest sideburns? Not under the new management.

The NFL had particularly struggled with how to crown a passing king. In the mid-1930s, when the league began keeping individual player stats, the passing leader was the quarterback with the most passing yardage. From 1938 to 1940, the passer with the highest completion percentage was number one. For 1941 they invented a system ranking the league's quarterbacks in each of six separate categories -- touchdown passes, yards, interception percentage, etc. -- and if you were first-place in touchdowns you'd get one point, and if you were 10th place in yardage you'd get 10 points, and the guy with the lowest total for the six categories was the top passer. Then over the next thirty years the criteria waffled back and forth, reverting occasionally to single categories, like average-gain-per-pass-attempt, that were interesting but really didn't tell the whole story. By the time Woodstock brought the era of free love to a close, the league had returned to a rotisserie-style point system where quarterbacks received ranking numbers relative to their peers' performance in four different passing categories, and the one with lowest total got the kitty.

It was an imperfect system that made it impossible to tabulate any quarterback's standing until all the other quarterbacks were done with their Sunday business. Rozelle asked the league's statistical committee to fix it. And they called upon Smith, an executive at the Pro Football Hall of Fame who was known as a statistical whiz.

Smith's goal was to build a system where each quarterbacking performance could get a fixed rating that wouldn't depend on how other quarterbacks did. "A rusher's record isn't affected by what anybody else does. Why should a passer's?" he asked. He knew that if he came up with a good formula, it could be applied to all past and future stats, allowing the world, someday, to compare Norm Van Brocklin's career passing proficiency (75.1) with, say, Steve Beuerlein's (74.3).

Smith liked the numerical cocktail, mixing what were essentially the only the passing stats tracked -- completion percentage, passing yardage, touchdowns and interceptions. He was in a plane flying over Kansas when the eureka moment hit. What if "average" performance in each of those four categories would score one point? And record-level performance would score two points. And for playing really poorly, you'd get zero. Simple! There would be a sliding scale in between. With data supplied by Elias Sports Bureau, the league's statistician, Smith determined that the average pass completion rate for the 1970 season was around 50 percent. Equaling that would be good for one point toward a passer's rating total. The record completion rate for a season was just over 70 percent, so from then on anybody hitting at a 70 percent rate would get two points in that category. Completing 30 percent (or less) of your passes would score a big zero.

.Here comes the college algebra. What's the magical formula that turns 50 into 1, 70 into 2, and 30 into zero? You subtract 30 and divide by 20 . That's the first leg of the formula above.

Smith then got league averages for yards-per-pass-attempt (7), percentage of passes scoring touchdowns (5%), and percentage of passes intercepted (5.5%). For each of these he devised a conversion formula to give a player one point for working at that average rate, two points for a record level, and as low as zero points for really eating it. Since records are made to be broken, a score above 2 in any category -- up to 2.375 -- is possible. Every quarterback in history would receive ratings relative to those 1970 standards.

So how in the name of Garo Yepremian does this get Brian Griese 102.9? Well, Griese actually got 1.71 points for completing 216 passes in 336 attempts, 1.25 points for his 2,688 passing yards, 1.13 points for throwing 19 touchdowns, and a remarkable 2.01 points for tossing just 4 interceptions all season. "Using those figures, you add it up," Smith explains (thereby giving Griese a raw total of 6.174) "and get something that means absolutely nothing."

Smith thought it would be more meaningful if an excellent score came to around 100, just like in school. "I think our attitude was that 100 was an A," he recalls. "And anything above 100, that was an A-plus." So, in a move that made sense at the time and has had everyone else confused for three decades, he multiplied the raw total by 100 and divided by 6, turning a statistically average performance -- 1s across the board -- into 66.7. It also made the maximum rating a ridiculous 158.2.

The NFL statistical squad applied this formula to every quarterback's data since the 1930s, and the results didn't seem ludicrous. They showed the proposed Passer Rating System to Rozelle, and he said, more or less, "If this is what you guys want to recommend, if you think it'll work, let's do it," according to Don Weiss, who headed the statistical committee.

"It could have been that it was so complicated, people didn't understand it, so they accepted it," Weiss says. It went official for the 1973 season.

On its face, it has worked. Only the excellent have had ratings over 100 for a full season. Last year only Griese and Trent Green did. In 1999, only Warner did - his 109.2 was the fifth best passing season ever, topped by:

Steve Young 1994 112.8
Joe Montana 1989 112.4
Milt Plum 1960 110.4
Sammy Baugh 1945 109.9

And nobody with more than 1,500 pass attempts has maintained a rating above 100. History's leader, Young, retired at 96.8. Behind him are Montana, Dan Marino, Bret Favre and Peyton Manning. You can look at that list and ask: how screwed up can the formula be? It's not like you feed in the data, and -- surprise -- out of the cake pops Neil Lomax as number one QB of all time (actually, he's 10th).

But what gives with the 49ers? Sure, Young and Montana were studs -- but come on. Here's what gives. It started in the late 1970s, when the NFL began bending the rules to favor passers.
The "illegal chuck" rule forced defenders to back off receivers, and refs lightened up on holding calls against offensive linemen. By 1979 the league completion average hit 54.1 percent, and it has never been below that since. Then came the "clearly in the grasp rule" to protect quarterbacks from brutality by would-be sackers. And receivers started wearing sticky gloves. And wind-free indoor stadiums with fast fake turf turned teams like the St. Louis Rams into passing factories. There's much more pass offense today, and that explains why the all-time top five looks like a list made by somebody born after John Belushi died.

But most influential of all was Bill Walsh, the QB-nurturing 49ers coach. The West Coast Offense he pioneered in the 1980s turned precisely timed, high-percentage short passes into catch-and-run long gains. You couldn't invent a better scheme to juice QB rating numbers. Because, it turns out, the formula mathematically whacks guys who try to throw long.

Imagine two quarterbacks -- Super Joe and Broadway Joe -- who both drive their teams 30 yards to a touchdown in three plays. Super Joe does it with three 10-yard passes. His completion percentage is 100, and for the drive his rating is 147.9. Broadway Joe throws two incomplete passes, then on a clutch third and long he finds a receiver in the end zone -- touchdown! For the exact same result, his rating is 111.1.

Young and Montana -- and Griese last year -- surely benefited from quarterbacking in West Coast schemes. The authors of The Hidden Game of Football calculate that even complete passes that lose yardage can, in some weirdball situations, boost a quarterback's rating. "There's something wrong with that," says co-author and football historian Bob Carroll. His book proposes a New Improved Rating System, which ignores completion percentage but counts sacks. (Still, when the book applies this even more complicated formula to 1997 data, Young comes out number one anyway.)

"I guess I'm a little defensive when people talk that way about the West Coast offense," says Young, who, just for the record, happened to run the ball like hell too. "Because in the end the West Coast works. It wins games. The truth is, if you're playing decent football, your rating's high. I never got the sense that I could take a game and manipulate my ratings. I don't think you can go out and dink and dunk and beat the system."

Is there a perfect way to enumerate the performance of one man on the field with 21 others? What about perfectly thrown, dropped passes? What about John Elway, who ended his career with a saggy rating of 79.9, even though he had the most career wins by a quarterback and a record 41 fourth-quarter game-saving drives. What about Troy Aikman (81.6), who had Emmitt Smith running behind him, so he didn't throw a lot of four-yard touchdown passes, "even if he made the pass that got the ball to the four," says his agent, Leigh Steinberg.

Sonny Jurgensen said the real measure of a quarterback's greatness is how he does on third and long, when everybody and his bookie knows a pass is coming. Many consider Jurgensen the best pure passer of all time, but his rating of 82.6 would embarrass Jeff Garcia. What about Donovan McNabb and Michael Vick and the new generation of quarterbacks who use the threat of a pass to open up rushing lanes for themselves?

Football is a team game, a game of drives and momentum. Individual numbers struggle to describe it. Still, numbers are all we have. Statistics are what separate sports from just playing around in the yard. Coach Lombardi said it: if you're not keeping score, you're just practicing. So you calculate what you can. If you wanted to keep it really simple, you could just do what golf does and list who gets the most money (Warner, $11.8 million, Manning $11 million, Vick $10.3 million). Or you could be like boxing and figure skating, where unless somebody gets knocked out it's two guys in bow-ties and a lady from the New Jersey State Athletic Commission making up numbers for how you did.

It has flaws, yeah, but the passer rating system achieves its goal: it establishes a standard, so we can compare today's quarterbacks with one another easily, and with yesterday's to see how the game has changed. The key is using the same yardstick for everyone, never moving the goalposts. "If a guy has a lifetime passing rating of 89.9, that's what it is. It's not gonna change a decade from now," Smith says.

Of course, the NFL did move the goal posts in 1974, screwing up field goal statistics forever. But who the hell cares about kickers?

[3ric]

Good find. I've always wondered how they came up with that formula.

[fantastic flying froggies]

Interesting read.

[MIJB#19]

Yeah, it's good to know why "they" set the perfect performance numbers at the places where the are.
77.5% completion percentage (like in the article, 30% is worth nothing)
12.5 yards per attempt (less then 3 is worth nothing)
19 touchdowns per 160 attempts (no score is worth nothing)
0 interceptions (19 per 200 attempts is worth nothing)

Still, there's a good point about having a stat that nobody understands the value of, even after the explaination, it doesn't tell all its secrets.

And freaking why is 19 so important in the formula?

[Samdari]

Very good article, albeit three years old. It does bring up an important point:

Brian Griese led the league passer rating??????

Probably a bigger indictment of the formula than anything else I have seen.

[Ksyrup]

Originally posted by Samdari
Very good article, albeit three years old. It does bring up an important point:

Brian Griese led the league passer rating??????

Probably a bigger indictment of the formula than anything else I have seen.

Why? He had a great year, and that's what the measurement reflected. That's like arguing that batting average doesn't accurately measure hitting ability simply because Terry Pendleton won a batting title one year.

[Noop]

•Drop quickness and balance
•Set up quickness and positioning
•Release quickness
•Accuracy: short, medium and long-direction, positioning, velocity and catchability of the ball.
•Judgement and decision making
•Field Vision: quick,decisive and accurate reads. Going through progressive read options corectly.
•Poise: leadership skills, handling pressure and moving the team.
•Ball handling: selling the play action fake, smooth effort with eyes positioned downfield.
•Timing: throwing on rhythm and to pre-determined spots on time.
•Delivery: quickness and positioning of release.
•Follow through: does he finish across his body?
•Avoiding the rush: making defenders miss in the pocket.
•Escape: moving the pocket effectively to avoid pressure.
•Scrambling: ability to run and pick up positive yardage.
•Rollout passing to the right and left: can he throw accurately and with velocity without having to set his feet. Does he have arm and hip flexability to throw the ball across his body if he is a left handed quarterback?
•Arm strength: ability to throw all routes and the ability to throw with just his arm without having to muscle the ball with his body.
•Zip: velocity and tightness of the spiral.
•Touch, timing and positioning: on screen passes and swing passes
•Effectiveness on: short out routes, deep out routes, go routes, post routes and corner routes





noop

[Darkiller]

If I recall corretly, Jim Harbaugh won the 1995 NFL Passing Title...

(in the midst of Steve Young winning 6 titles in 7 years, those of 1991-92-93-94-96-97 !!!)

[damnMikeBrown]

Nice list Noop. I'd love a football sim to incorporate scouting reports on just those things.

[QuikSand]

So, what should we conclude from all this generic criticism of the QB rating system? That it's too "mathy" for regular people, so it should be abandoned? Come on...

Just because Leigh Steinberg can't calculate it in his head doesn't mean a damned thing, other than a snicker-worthy comment. If you've got something meaningful to say about the system (like the intimations that it might bestow too favorable a rating on a dink-and-dunk system QB) fine, but this bullshit that it's bad because it's math is just that.

Wh is the "best" quarterback or passer, then? The opening salvo in the article suggests that the title ought to simply go to the guy who threw for the most yards on the season. Okay... somebody find me that list, and tell me how often in the last 20 years has that guy been anywhere near being the consensus best QB in the league. Five or six our of twenty, perhaps?

Just because nitwits are afraid of complicated measurements doesn't mean that we should abandon them. There's plenty of room to criticize the formula-- but I don't think it's fair to generically criticize all formulas by implication.

[Samdari]

Originally posted by QuikSand
So, what should we conclude from all this generic criticism of the QB rating system? That it's too "mathy" for regular people, so it should be abandoned? Come on...

I am as mathy as it gets, and I hate this formula. My biggest beef, as has been detailed (and detailed, and detailed...) in other discussions is that it rewards the passer for things that he does not make contributions to.

[MIJB#19]

Personally, I decided to copy the formula in an excel spreadsheet so I could actually get to understand the meaning of it.

It's a horrible formula to work with, though there is no easier rating system (as of yet.)

On the other end, what does the example of Edgerrin James say?
- He has a great blocking FB in front of him?
- He had a great OL?
- His team basically runs a rush friendly game plan?
- He can run for 15 yards on 3rd and 20?
- He was the only RB in the NFL not missing a game (or significant parts of games) due to injury?
- The rest of the team (passing game) keeps the chains moving so James can keep running?
- James' team had a schedule with a lot of opponents lacking the DL and LB force to stop James?
- Edgerrin James was the best running back?

Same case could be made for Muhammed or Harrison.

[dawgfan]

We've discussed this at length in various forms in past threads, but statistical measurements of football players will always be less precise in determining value than statistics for baseball players and probably even basketball players, due to the extreme team nature of the game.

I don't have any problem with using a statistical formula to judge the performance of a QB, so long as we understand that it is what it is - part of the picture, but not the whole picture.

If I were to redesign it, I would have the system be edited to be relative to each season. Rather than continuing to live with the standards based on the 1970 season, I would change the formula so that the "averages" that comprise the median values were based on that seasons overall averages. So, rather than having 50% considered an average pass completion percentage, it would be 55% (or whatever the average is currently) - same with the yards per pass attempt, td pct and int pct. That way passers are judged in the context of the era in which they play.

Some might complain that this unfairly ignores the likelihood that today's modern athletes are superior to those of yore. Well, the gross stats don't change - Dan Marino's 5,000 yard season will still be much higher than the 1,500 yard seasons of Otto Graham. There's still the evidence of superior performance.

However, I think we lose something by not judging something like a passing-efficiency rating based on the context in which that performance was conceived. For example, Neil Lomax was a pretty good QB, but the fact he recently was as high as 10th all-time in pass-efficiency rating seems wrong.

Perhaps some more enlightened NFL statistics books will start offering such a contextualized version of the passing rating in addition to the normal one, much like sabermatricians in baseball have introduced concepts like park-effects and era-adjustments.

[vtbub]

Any serious look at QB rating needs to be done with at least an era adjustment and probably taking YAC(yards after catch) into the equation as well.

[Darkiller]

I won't say it's anything near perfect but I, for one, like the passer rating formula and really believe it is a measuring stick to judge a QB's performance.

Once again, it might not be the perfect thing, but I trust it for what it is worth and I think it is indeed a very good means to evaluate a QB's efficiency.

[vtbub]

You can't compare it across era's anymore. At least tweak it some.

[dawgfan]

Originally posted by Darkiller
I won't say it's anything near perfect but I, for one, like the passer rating formula and really believe it is a measuring stick to judge a QB's performance.

Once again, it might not be the perfect thing, but I trust it for what it is worth and I think it is indeed a very good means to evaluate a QB's efficiency.

I think it's less effective in judging the performance of a QB then it is in judging the effectiveness of a team's passing attack. Do we really want to give credit to a QB for yards gained after the catch by a receiver (which favors teams that are successful with crossing patterns and receivers that are good runners)? Do we penalize a QB because his offensive coordinator favors more risky downfield passing which leads to more interceptions and lower completion percentage?

The rating has some value in judging a QB's performance, but is only part of the whole picture. Adjust it to reflect contemporary averages in QB performance and we get a little clearer picture in comparing QB's of different eras. As a means of judging a team's passing game efficiency, it's quite useful.

The game of football has changed so much over the years that to compare the ratings of QB's in different eras using a formula based on the average performance of QB's in one particular year is basically pointless.

[Darkiller]

Originally posted by dawgfan

The rating has some value in judging a QB's performance, but is only part of the whole picture.

true

[/i]
The game of football has changed so much over the years that to compare the ratings of QB's in different eras using a formula based on the average performance of QB's in one particular year is basically pointless. [/quote]

I agree.

[Samdari]

Originally posted by dawgfan
I think it's less effective in judging the performance of a QB then it is in judging the effectiveness of a team's passing attack.

clapclapclap

[larrymcg421]

I don't like the system because it relies too heavily on TD's. That may seem like a weird thing to say, but consider these examples:

Lets say QB A can't complete a thing. His running back carries it 90 yards down to the 1. Then the QB completes a 1 yard TD to a wide open receiver. Same things happen two more times. These three 1 yard TD passes are his only completions out of say, 10 attempts.

Now let's say QB B is passing the ball much better, but when his team gets down to the 1, the coach calls a handoff the to the fullback who pounds it in.

QB A's line is 3 completions in 10 attempts for 3 yards, 3 TD's, and 0 INT's
QB B's line is 5 completions in 10 attempts for 80 yards, 0 TD's, and 0 INT's

QB A's rating is 79.2
QB B's rating is 77.1

Another example. Let's consider Marino's 1984 season.

He was 362 for 564, for 5084 yards, 48 TD's, and 17 INT's. This gave him a QB rating of 108.9

Pete Johnson was a short yardage specialist for the Dolphins that year. He ran for 9 TD's. Lets say instead Marino throws those 9 short TD's. His QB rating jumps to 114.3

Marino would've had to throw for 5,800 yards to make up that difference without the TD's. And that that's without any additional pass attempts/completions.

[Darkiller]

Originally posted by larrymcg421


QB A's line is 3 completions in 10 attempts for 3 yards, 3 TD's, and 0 INT's
QB B's line is 5 completions in 10 attempts for 80 yards, 0 TD's, and 0 INT's

QB A's rating is 79.2
QB B's rating is 77.1

I am not shocked at all by these two ratings.
there is just a two-point difference between the two and while one has thrown for FAR more yards, QBa has 3 TDs while QBb has zero.

Some will argue that football is about winning games, thus scoring points so this reward the TDs that brought the points to the team.

Plus,
QBb has just completed 2 more passes that QBa .. out of 10 attempts so all in all :

one has much more yards, the other has much more TDs.
thoses ratings are quite fair I would say.

[larrymcg421]

Maybe it's just me, but I'd rather have QB B.

[fantastic flying froggies]

And another bump for Skydog to link to the FAQ.

Yeah, it 's a slow day at the office today, got nothing better to do...

[Wasabiak]

Originally posted by larrymcg421
Maybe it's just me, but I'd rather have QB B.

Exactly, because chances are QBb would have made those same passes had the same scenarios and plays been called. But then, if QBb is the hot hand, and the team is on the one yd line, would the defense be looking for a pass play because of QBb? Maybe the defense sold out on the run because QBa was in the game, thus leaving the intended WR wide open, where as that WR might be covered if QBb is in the game.

It's all very thought provoking.

[John Galt]

I think the QB rating has come to be like the Save in baseball - both are stats that are helpful in understanding what is happening, but they have foolishly come to change the way the game is being played. In baseball, you save your best relief pitcher for the 9th inning to protect lead, rather than using them in higher leverage situations earlier (or with a tie in the 9th). That is ridiculous.

In football, the QB rating is used to judge QB's and thus rewards West Coast style QB's. I think the Joe Broadway v. Montana example in the article highlights my biggest problem with the rating. A QB who goes 1/3 in a 99 yd passing drive is just as good as a QB that goes 3/3 in a 99 yd passing drive. The QB rating system doesn't think so. You may say that the 3/3 QB is better to have over time (because he is more accurate), but that is better judged over larger sample sizes and it will be obvious if the 1/3 QB doesn't repeat his performance. The deep ball has died in football and I think the QB rating is part of the problem. A QB that can reliably through it down the field and create big plays is probably more effective over the long run than a weak-armed high percentage guy.

[Francis_Cole]

anyone know how many games/yards a qb has to have before this rating is recognised? Chad "The Kid" Pennington currently has a higher rating than all the qb's the nfl has on its qb ratings table, but chad isn' t on it (presumbly cos he hasn't played enough)

[Darkiller]

a QB needs 1500 pass attempts for his rating to be compiled.

[Huckleberry]

The reason football stats are meaningless compared to baseball stats isn't, IMO, related to anything "extreme" about the game. It's the same reason that most basketball stats are meaningless.

Baseball is a game of individual actions. The batter is on his own against the pitcher. When the ball is put in play, the fielder is on his own making a play. At least to the point where he throws the ball. Baseball is a game played in series. Football and basketball are games played in parallel. It is incredibly easy to identify the value of each item in a series. It's much harder to identify the value of each item of 11 in parallel.