My Computer Ratings (NCAA and NFL)

[Huckleberry]

It was my goal at first to devise computer ratings that accounted for more than just points, such as yards gained and allowed. However, that quickly changed when I decided that I would want to be able to apply my system to the high school ranks and publish an open-source ratings set for Texas high school football.

With that in mind, I decided to combine aspects of both the Massey and Colley formulas, as explained below:

I've gone with a matrix system using the mathematical basis used by Colley's rankings as explained in this PDF file. This is basically a method to solve the system of equations created by attempting to solve for each team's rating based on each game played.

However, Colley uses a simple game scoring system of 1 and -1. For a win you get 1 and a loss you get -1. He's 100% opposed to using margin of victory in any way, and is actually quite feisty about it. I don't agree and am including it because it clearly indicates team strength when Team A beats Team C by 40 points while Team B beats them by 3 points.

So how to score games? There certainly should be a difference between winning by 7 points and winning by 21 points. But the bonus for winning by 54 instead of 40 shouldn't be quite so high. My game scoring method is based on the Game Outcome Function used by Massey and explained here:

Game Outcome Function (GOF)

Given the score of a game, GOF(pA,pB) assigns a number between 0 and 1 that estimates the probability that team A would win a rematch under the same conditions. Based on previous experience, it seems reasonable to distinguish between a 10-0 win and a 50-40 win. A close high scoring game is likely to have more variance, and less likely to be dominated by either team. While a low scoring game may indicate a defensive struggle, or poor game conditions. In which case, a small deficit is more difficult to overcome.

For my initial ratings, I developed a GOF that closely approximates Massey's example listed on that page. Here's a comparison:

A's points (pA)	B's points (pB)	Massey's GOF	Huck's GOF
30	29	0.527	0.5308
10	9	0.5359	0.5318
27	24	0.5836	0.5900
27	20	0.6924	0.6975
50	40	0.7292	0.7503
10	0	0.8548	0.8567
30	14	0.8786	0.8721
45	21	0.9433	0.9353
45	14	0.9823	0.9740
30	0	0.992	0.9882
56	3	0.9998	0.9991

That was my attempt to mimic Massey's GOF. Now, for my ratings I modified the function so that the minimum GOF for a winning team is 2/3 . While that's a popular number around here, I'm actually using it because I wanted a certain level of credit for winning the game. I'm actually using 3/4 for the NFL and the straight GOF above for high school. It's my opinion that the higher the level, the more confidence you can have that the winning team is the better team.

I took the GOF and multiplied it by 2 and subtracted 1 to arrive at the game score for each team in each game. This results in an infinite blowout being a score of 1 for the winning team and a score of -1 for the losing team.

Here are the results of this rating system for the season-to-date in college football:

Rank	Team	Rating	Record
1	Oklahoma	0.909	10-0
2	Southern Cal	0.889	10-0
3	Auburn	0.870	10-0
4	Utah	0.834	10-0
5	California	0.832	8-1
6	Texas	0.812	9-1
7	Arizona St.	0.777	8-2
8	Miami (FL)	0.760	7-2
9	Louisville	0.758	7-1
10	Florida St.	0.752	8-2
11	Georgia	0.746	8-2
12	Michigan	0.737	9-1
13	Boise St.	0.728	9-0
14	Texas A&M	0.724	7-3
15	Virginia	0.722	7-2
16	Tennessee	0.711	7-2
17	LSU	0.708	7-2
18	Oklahoma St.	0.703	7-3
19	Wisconsin	0.701	9-1
20	Iowa	0.694	8-2
21	Boston College	0.664	7-2
22	Purdue	0.663	6-4
23	Virginia Tech	0.662	7-2
24	Florida	0.659	6-4
25	Notre Dame	0.647	6-4
26	Texas Tech	0.639	6-4
27	UTEP	0.627	7-2
28	UCLA	0.620	6-4
29	West Virginia	0.619	8-2
30	Fresno St.	0.615	6-3
31	Arkansas	0.613	4-5
32	Bowling Green	0.611	8-2
33	Ohio St.	0.610	6-4
34	Oregon St.	0.604	5-5
35	South Carolina	0.602	6-4
36	Colorado	0.591	6-4
37	Michigan St.	0.577	5-5
38	Minnesota	0.576	6-5
39	Georgia Tech	0.574	6-3
40	Alabama	0.573	6-4
41	UAB	0.567	6-3
42	North Carolina	0.557	5-5
43	New Mexico	0.556	6-4
44	Oregon	0.553	5-5
45	Memphis	0.552	6-3
46	Stanford	0.549	4-6
47	Pittsburgh	0.547	6-3
48	BYU	0.547	5-5
49	Cincinnati	0.521	5-4
50	Northwestern	0.508	5-5
51	Missouri	0.508	4-5
52	Miami (OH)	0.507	7-3
53	Iowa St.	0.507	5-4
54	Kansas St.	0.505	4-6
55	Clemson	0.504	5-5
56	Nebraska	0.504	5-5
57	Southern Miss	0.503	5-3
58	No. Illinois	0.501	7-3
59	Toledo	0.498	7-3
60	Maryland	0.497	4-5
61	Kansas	0.494	3-7
62	Troy	0.489	6-4
63	Syracuse	0.487	5-5
64	Wake Forest	0.485	4-5
65	North Texas	0.482	6-4
66	Louisiana Tech	0.481	5-5
67	Washington St.	0.478	4-6
68	No. Carolina St.	0.478	4-6
69	Wyoming	0.477	6-4
70	Marshall	0.471	5-5
71	Navy	0.467	7-2
72	Colorado St.	0.464	4-6
73	Rutgers	0.457	4-5
74	Connecticut	0.455	5-4
75	South Florida	0.439	4-4
76	Hawaii	0.438	4-5
77	Penn St.	0.436	3-7
78	Mississippi St.	0.433	3-6
79	TCU	0.428	4-5
80	Middle Tenn. St.	0.427	5-5
81	Air Force	0.412	4-6
82	Mississippi	0.410	3-6
83	Akron	0.403	6-4
84	New Mexico St.	0.397	5-5
85	Houston	0.387	3-7
86	Indiana	0.386	3-7
87	Arizona	0.377	2-8
88	San Diego St.	0.373	3-7
89	Baylor	0.370	3-7
90	Tulane	0.356	4-5
91	Kent St.	0.350	4-6
92	Illinois	0.347	3-7
93	Rice	0.336	3-7
94	Vanderbilt	0.333	2-8
95	UNLV	0.327	2-8
96	Kentucky	0.316	2-8
97	La.-Lafayette	0.301	4-6
98	Duke	0.298	1-9
99	Washington	0.295	1-9
100	Nevada	0.291	5-5
101	SMU	0.287	3-7
102	Army	0.284	2-7
103	Arkansas St.	0.276	3-7
104	E. Michigan	0.272	4-6
105	East Carolina	0.270	2-7
106	Ohio	0.266	4-7
107	Temple	0.264	2-8
108	La.-Monroe	0.263	4-6
109	Tulsa	0.260	2-8
110	Ball St.	0.249	2-8
111	Idaho	0.239	3-8
112	Utah St.	0.227	2-8
113	San Jose St.	0.204	2-7
114	Cen. Michigan	0.191	3-7
115	Buffalo	0.156	2-8
116	UCF	0.139	0-10
117	W. Michigan	0.131	1-9

Initial review shows that this system is heavily points dependent, as it should be based on the GOF used. A team that wins by a single point gets a game score of ~.333 while the losing team gets a game score of ~-.333 as compared to the 1 and -1 from a system that doesn't use points at all. Logically this can be taken to mean that, based on this game, this GOF is saying that we can be 66.7% sure that the winning team is better than the losing team whereas the no-points system states that, based only on this game, we are 100% sure that the winning team is a better team.

Anyway, here's the NFL results for this year so far:

Rank	Team	Rating	Record
1	New England	0.756	8-1
2	Pittsburgh	0.753	8-1
3	Philadelphia	0.752	8-1
4	Indianapolis	0.645	6-3
5	Baltimore	0.640	6-3
6	San Diego	0.614	6-3
7	Atlanta	0.599	7-2
8	NY Jets	0.588	6-3
9	Denver	0.585	6-3
10	Jacksonville	0.567	6-3
11	NY Giants	0.543	5-4
12	Minnesota	0.538	5-4
13	Green Bay	0.508	5-4
14	Seattle	0.481	5-4
15	Cleveland	0.478	3-6
16	Detroit	0.477	4-5
17	Houston	0.475	4-5
18	St. Louis	0.472	5-4
19	Cincinnati	0.468	4-5
20	Buffalo	0.454	3-6
21	Chicago	0.445	4-5
22	Arizona	0.439	4-5
23	Kansas City	0.433	3-6
24	Tennessee	0.410	3-6
25	Dallas	0.393	3-6
26	Washington	0.388	3-6
27	New Orleans	0.377	4-5
28	Oakland	0.376	3-6
29	Tampa Bay	0.374	3-6
30	Carolina	0.341	2-7
31	Miami	0.312	1-8
32	San Francisco	0.230	1-8

Texas high school results will be forthcoming. But that's a lot of data to enter.

Thoughts?

[albionmoonlight]

Originally Posted by Huckleberry

Thoughts?

It passes the smell test in that the final rankings approximate how I think that the teams should be ordered in my subjective opinion.

Personally, I don't agree with using margin of victory simply because we tell teams that winning is the only thing, and then rate them on how much they win by. But that's just a difference of opinion.

Overall, I like the results.

[ISiddiqui]

Not a bad ranking system at all. There are a few quibbles, such as albionmoonlight refered to, that margin of victory can be deceptive sometimes. Some teams will run up the score, while others won't and that does have an effect, no matter how slight.

[DanGarion]

Originally Posted by ISiddiqui

Not a bad ranking system at all. There are a few quibbles, such as albionmoonlight refered to, that margin of victory can be deceptive sometimes. Some teams will run up the score, while others won't and that does have an effect, no matter how slight.

Which is why in my opinion anything that has "margin of victory" as a factor should have a cap or as the margin of victory gets higher the points count less.

such as it starts at .333 and it for the first 10 points and works down from there or something.

[Huckleberry]

The diminishing returns shown above is the same thing as a cap. And because it doesn't use direct margin of victory, the cap you speak of can't be implemented.

Calling a 63-42 win the same thing as a 41-0 win is crazy, IMO. (A 21 point MOV cap)

[Huckleberry]

Rank	Team	Rating	Record
1	Oklahoma	0.904	11-0
2	Southern Cal	0.890	10-0
3	Auburn	0.867	11-0
4	Utah	0.853	11-0
5	California	0.849	9-1
6	Texas	0.814	9-1
7	Arizona St.	0.782	8-2
8	Miami (FL)	0.782	8-2
9	Louisville	0.773	8-1
10	Boise St.	0.753	10-0
11	Virginia	0.747	8-2
12	Georgia	0.741	8-2
13	Iowa	0.732	9-2
14	Texas A&M	0.730	7-3
15	Florida St.	0.729	8-3
16	Oklahoma St.	0.707	7-3
17	LSU	0.703	8-2
18	Michigan	0.702	9-2
19	Tennessee	0.692	8-2
20	Virginia Tech	0.690	8-2
21	Purdue	0.680	7-4
22	Florida	0.677	7-4
23	Wisconsin	0.669	9-2
24	Boston College	0.662	8-2
25	Ohio St.	0.652	7-4
26	Texas Tech	0.644	6-4
27	Notre Dame	0.641	6-4
28	UTEP	0.640	8-2
29	Oregon St.	0.637	6-5
30	Fresno St.	0.631	7-3
31	UCLA	0.620	6-4
32	West Virginia	0.616	8-2
33	Arkansas	0.610	5-5
34	Bowling Green	0.610	8-2
35	Colorado	0.591	6-4
36	North Carolina	0.577	6-5
37	New Mexico	0.573	7-4
38	Minnesota	0.568	6-5
39	UAB	0.568	7-3
40	Alabama	0.563	6-5
41	Georgia Tech	0.559	6-4
42	South Carolina	0.559	6-5
43	Cincinnati	0.553	6-4
44	Iowa St.	0.548	6-4
45	Clemson	0.545	6-5
46	Pittsburgh	0.543	6-3
47	Memphis	0.542	7-3
48	BYU	0.535	5-6
49	Stanford	0.532	4-7
50	Michigan St.	0.530	5-6
51	Kansas	0.528	4-7
52	Miami (OH)	0.523	8-3
53	Oregon	0.520	5-6
54	Navy	0.518	8-2
55	Northwestern	0.516	6-5
56	No. Illinois	0.516	8-3
57	Troy	0.503	7-4
58	Toledo	0.501	7-3
59	Nebraska	0.500	5-5
60	North Texas	0.495	7-4
61	Syracuse	0.487	5-5
62	No. Carolina St.	0.485	4-6
63	Penn St.	0.481	4-7
64	Washington St.	0.479	5-6
65	Kansas St.	0.476	4-7
66	Maryland	0.474	4-6
67	Marshall	0.473	6-5
68	Wake Forest	0.472	4-5
69	TCU	0.471	5-5
70	Missouri	0.470	4-6
71	Connecticut	0.468	6-4
72	Louisiana Tech	0.467	5-6
73	Hawaii	0.466	5-5
74	Wyoming	0.463	6-5
75	Air Force	0.460	5-6
76	Southern Miss	0.456	5-4
77	Mississippi St.	0.430	3-7
78	Colorado St.	0.424	4-7
79	Rutgers	0.414	4-6
80	Mississippi	0.407	3-7
81	South Florida	0.404	4-5
82	San Diego St.	0.402	4-7
83	Middle Tenn. St.	0.393	5-6
84	Akron	0.388	6-5
85	Houston	0.381	3-8
86	Arizona	0.377	2-8
87	Baylor	0.376	3-8
88	Indiana	0.369	3-8
89	New Mexico St.	0.356	5-6
90	Tulane	0.355	4-5
91	Kent St.	0.347	4-6
92	Rice	0.344	3-7
93	Illinois	0.341	3-8
94	Vanderbilt	0.340	2-9
95	Kentucky	0.312	2-8
96	UNLV	0.297	2-9
97	Army	0.294	2-8
98	Washington	0.292	1-10
99	Tulsa	0.291	3-8
100	Duke	0.290	1-10
101	SMU	0.285	3-8
102	Nevada	0.280	5-6
103	La.-Monroe	0.278	5-6
104	East Carolina	0.271	2-8
105	La.-Lafayette	0.267	5-6
106	Ohio	0.264	4-7
107	Temple	0.260	2-9
108	Utah St.	0.260	3-8
109	E. Michigan	0.254	4-7
110	Arkansas St.	0.250	3-8
111	Ball St.	0.230	2-9
112	Idaho	0.218	3-9
113	Cen. Michigan	0.201	4-7
114	San Jose St.	0.186	2-8
115	Buffalo	0.141	2-9
116	UCF	0.138	0-10
117	W. Michigan	0.134	1-10

[sooner333]

What about the differnence between a 63-42 win vs. a 21-0 win. I didn't read all too in-depth about your margin system, but woudln't you think that a 21-0 win should get more than a 63-42 win?

[The_herd]

Very nice looking rankings this week.

[Huckleberry]

sooner333 -

In this system, the 21-0 gets more credit.

[Huckleberry]

Rank	Team	Rating	Record
1	New England	0.760	9-1
2	Philadelphia	0.758	9-1
3	Pittsburgh	0.753	9-1
4	Indianapolis	0.669	7-3
5	Baltimore	0.658	7-3
6	San Diego	0.628	7-3
7	Atlanta	0.614	8-2
8	Denver	0.610	7-3
9	NY Jets	0.605	7-3
10	Minnesota	0.553	6-4
11	Jacksonville	0.544	6-4
12	Green Bay	0.530	6-4
13	NY Giants	0.516	5-5
14	Buffalo	0.488	4-6
15	Seattle	0.482	6-4
16	Cincinnati	0.469	4-6
17	Cleveland	0.464	3-7
18	Houston	0.462	4-6
19	Detroit	0.454	4-6
20	Tennessee	0.445	4-6
21	Kansas City	0.439	3-7
22	St. Louis	0.429	5-5
23	Chicago	0.423	4-6
24	Tampa Bay	0.397	4-6
25	Carolina	0.389	3-7
26	Arizona	0.388	4-6
27	Oakland	0.380	3-7
28	Washington	0.376	3-7
29	Dallas	0.374	3-7
30	New Orleans	0.353	4-6
31	Miami	0.299	1-9
32	San Francisco	0.198	1-9

[digamma]

The rankings seem to be about as spot on as any I've seen.

However, I'm curious about this statement from your first post...

That was my attempt to mimic Massey's GOF. Now, for my ratings I modified the function so that the minimum GOF for a winning team is 2/3 . While that's a popular number around here, I'm actually using it because I wanted a certain level of credit for winning the game. I'm actually using 3/4 for the NFL and the straight GOF above for high school. It's my opinion that the higher the level, the more confidence you can have that the winning team is the better team.

I think I'm probably of the opposite opinion. My thought would be that the higher the level, the closer the two teams probably are in talent, the more impact coaching and game planning can have and the greater chance of an upset.

It probably doesn't change your college ratings because they certainly lie in between the NFL and high school, no matter which way you go, but I'm wondering how much it skews the NFL ratings if you were to go closer to the straight GOF.

[Huckleberry]

First my logic. Professional athletes are more likely to perform at their ability level. They have more practice time, more experience, etc. High school kids, and to a lesser extent college players, will have more variation in their performance, therefore our confidence that they played at their ability level is lower for each game. Anyway, here are the rankings using the straight GOF (~.53 instead of .75 for a close win):

Rank	Team	Rating	Record
1	New England	0.698	9-1
2	Philadelphia	0.696	9-1
3	Pittsburgh	0.678	9-1
4	Indianapolis	0.658	7-3
5	Baltimore	0.638	7-3
6	San Diego	0.628	7-3
7	Denver	0.590	7-3
8	NY Jets	0.588	7-3
9	Atlanta	0.553	8-2
10	NY Giants	0.530	5-5
11	Minnesota	0.526	6-4
12	Green Bay	0.517	6-4
13	Buffalo	0.508	4-6
14	Seattle	0.496	6-4
15	Jacksonville	0.494	6-4
16	Cleveland	0.493	3-7
17	Kansas City	0.487	3-7
18	Cincinnati	0.484	4-6
19	Houston	0.481	4-6
20	Detroit	0.464	4-6
21	Tennessee	0.459	4-6
22	Chicago	0.442	4-6
23	Tampa Bay	0.438	4-6
24	Carolina	0.437	3-7
25	St. Louis	0.416	5-5
26	Washington	0.412	3-7
27	Arizona	0.406	4-6
28	Oakland	0.395	3-7
29	Miami	0.376	1-9
30	Dallas	0.363	3-7
31	New Orleans	0.344	4-6
32	San Francisco	0.262	1-9

[digamma]

Interesting. Thanks. I understand your logic (though still not sure if I agree with it), but I do think the ratings look better with the modified GOF.

[Huckleberry]

Rank	Team	Rating	Record
1	Southern Cal	0.912	11-0
2	Oklahoma	0.902	11-0
3	Auburn	0.869	11-0
4	Utah	0.858	11-0
5	California	0.852	9-1
6	Texas	0.837	10-1
7	Louisville	0.790	9-1
8	Miami (FL)	0.781	8-2
9	Boise St.	0.758	11-0
10	Georgia	0.750	9-2
11	LSU	0.733	9-2
12	Arizona St.	0.729	8-3
13	Florida St.	0.729	8-3
14	Virginia Tech	0.728	9-2
15	Iowa	0.723	9-2
16	Virginia	0.719	8-3
17	Texas A&M	0.708	7-4
18	Michigan	0.695	9-2
19	Tennessee	0.687	9-2
20	Florida	0.676	7-4
21	Oklahoma St.	0.676	7-4
22	Texas Tech	0.676	7-4
23	Purdue	0.675	7-4
24	Wisconsin	0.666	9-2
25	Ohio St.	0.646	7-4
26	Oregon St.	0.643	6-5
27	Fresno St.	0.635	8-3
28	Notre Dame	0.620	6-5
29	UCLA	0.618	6-4
30	Boston College	0.607	8-3
31	Colorado	0.603	7-4
32	UTEP	0.598	8-3
33	West Virginia	0.594	8-3
34	Arkansas	0.585	5-6
35	North Carolina	0.580	6-5
36	Bowling Green	0.577	8-3
37	New Mexico	0.577	7-4
38	Alabama	0.567	6-5
39	Memphis	0.565	8-3
40	Minnesota	0.565	6-5
41	Pittsburgh	0.562	7-3
42	South Carolina	0.559	6-5
43	Georgia Tech	0.558	6-5
44	Clemson	0.543	6-5
45	BYU	0.536	5-6
46	UAB	0.536	7-4
47	Kansas	0.535	4-7
48	Cincinnati	0.530	6-5
49	Stanford	0.529	4-7
50	Toledo	0.529	8-3
51	Syracuse	0.527	6-5
52	Iowa St.	0.524	6-5
53	Michigan St.	0.523	5-6
54	Miami (OH)	0.523	8-3
55	Navy	0.521	8-2
56	Oregon	0.521	5-6
57	No. Illinois	0.513	8-3
58	No. Carolina St.	0.509	5-6
59	Troy	0.509	7-4
60	North Texas	0.499	7-4
61	Hawaii	0.494	6-5
62	Maryland	0.494	5-6
63	Nebraska	0.492	5-6
64	Missouri	0.488	5-6
65	Northwestern	0.484	6-6
66	Washington St.	0.483	5-6
67	Connecticut	0.483	7-4
68	Southern Miss	0.478	6-4
69	Kansas St.	0.477	4-7
70	Marshall	0.470	6-5
71	Louisiana Tech	0.470	5-6
72	Penn St.	0.470	4-7
73	Wyoming	0.466	6-5
74	Air Force	0.462	5-6
75	Wake Forest	0.450	4-6
76	Mississippi	0.443	4-7
77	TCU	0.442	5-6
78	Arizona	0.434	3-8
79	Colorado St.	0.428	4-7
80	San Diego St.	0.403	4-7
81	Rutgers	0.397	4-7
82	Mississippi St.	0.395	3-8
83	Middle Tenn. St.	0.394	5-6
84	Akron	0.384	6-5
85	Houston	0.382	3-8
86	Tulane	0.380	5-5
87	South Florida	0.376	4-6
88	Baylor	0.372	3-8
89	Indiana	0.363	3-8
90	Kent St.	0.360	5-6
91	New Mexico St.	0.352	5-6
92	Rice	0.350	3-7
93	Vanderbilt	0.342	2-9
94	Illinois	0.335	3-8
95	Tulsa	0.332	4-8
96	Kentucky	0.320	2-9
97	UNLV	0.299	2-9
98	Washington	0.298	1-10
99	Duke	0.292	1-10
100	Army	0.288	2-8
101	Nevada	0.286	5-7
102	SMU	0.285	3-8
103	La.-Monroe	0.278	5-6
104	La.-Lafayette	0.269	5-6
105	Ohio	0.266	4-7
106	Utah St.	0.262	3-8
107	Arkansas St.	0.260	3-8
108	Temple	0.257	2-9
109	E. Michigan	0.255	4-7
110	East Carolina	0.251	2-9
111	Ball St.	0.225	2-9
112	Idaho	0.223	3-9
113	Cen. Michigan	0.201	4-7
114	San Jose St.	0.187	2-9
115	Buffalo	0.145	2-9
116	W. Michigan	0.136	1-10
117	UCF	0.120	0-11

[Huckleberry]

I'm not sure if anyone still cares about this, but I've added automation to the process. Now all I have to do is type in the teams and scores for each game and let the program I wrote do the rest.

Here's a link to the output page from the program for college ball. In case you weren't aware, it's pretty clearly based on David Rothman's program. I had to change it up, but the code includes references to all source material.

http://www.hornfans.com/recruiting/h...rry/D1RATE.TXT

[Cuckoo]

I haven't said so yet, but I definitely care about this. Nice job, Huck.

[duckman]

Pretty good for a Longhorns fan.

[kparker15]

could you send me a copy of you program. I have dabled in trying different ranking systems and have even made an elaborate spreadsheet in excel. Can your matrix be input into excel? I have read massey's articles but, my pathetic math knowledge limits me in many ways.

[kparker15]

could you send me a copy of your program. I have dabled in trying different ranking systems and have even made an elaborate spreadsheet in excel. Can your matrix be input into excel? I have read massey's articles but, my pathetic math knowledge limits me in many ways.

[Huckleberry]

Actually, my rankings set started out in Excel because it's matrix based. However, my Excel (Office 97) has issues inverting matrices larger than 52 x 52, so I had to get Quattro Pro to do that part.

The matrix I have is Colley's Matrix but the game scoring system is different. PM me with your e-mail address and I can send you what I have.