In any College Football ranking system, Strength of Schedule, is a crucial component. The BCS's method of calculating SoS relies solely on Win-Loss Records (Just like everything else in the BCS). This is a less than ideal situation because a team's record is less an indicator of their strength than their performance in those games. Consider two teams with 5-5 records, Team A won it's five games handily while the losses were close and hard fought. Team B, on the other hand was blown out in their five losses and just barely scraped through the games it won. We all know Team A is better than Team B, but to the BCS they are equally strong opponents. Because of this I use the Team Rating (performance) to determine how tough an opponent is. In my ranking system, a team's Strength of Schedule is simply the average Team Rating of their Div 1-A opponents. I don't think that this is the best system, but it's easy to balance and produces expected results. Below, I'll discuss more about SoS and why a simple average is not always reflective of a schedule's actual difficulty.
What makes a schedule hard?
Is the hardest schedule the one with the greatest average strength of opponents, or is it the one that's hardest to get through undefeated? What's the difference?
Consider two schedules, one with 5 strong teams and 5 weak teams, and a second with 10 average teams. Both have an identical average strength, despite being very different. Given a top tier team, or any above average team, the schedule with 5 strong teams would probably result in a worse record than the one with 10 average teams. As you get to the below average teams, the first schedule becomes the easier one, due to the fact that a below average team has a better chance against the 5 weak teams than against any of the average teams. What we see, is that how hard a schedule is, varies from team to team and that the average difficulty of opponents does not always reflect a true strength of schedule.
A Possible Solution
First, a true SoS solution should be based on performance as opposed to W/L records. Second it should realize that there is a very substantial difference between average teams and the top 20 or so teams. Third, it should rank a schedule with 5 strong teams and 5 weak teams as more difficult than a schedule of 10 average teams.
Strength of Schedule
Rank all teams according to their Team Rating score.
Each opponent ranked in the top 10 is worth 3 points.
Each opponent ranked 11-30 is worth 2 points.
Each opponent ranked 31-70 is worth 1 point.
Each opponent ranked 71-100 is worth 0.5 points.
Each opponent ranked 101-120 is worth 0 points.
Each 1-AA opponent is worth 0 points.
Average a team's total points against the number of games played to find SoS.
This system satisfies the three criteria listed above. We'll see how this pans out over the course of the season and see if any adjustments or revisions need to be made.
Thursday, August 23, 2007
The Factor
Since we've already seen the performance-based aspect of the rankings it's time to unveil the achievement-based component. I call it Win/Loss Factor.
What is Win/Loss Factor?
Win/Loss Factor is based on a team's win percentage but includes bonuses for beating strong teams (Quality Wins) and penalties assessed for losing to weak teams (Embarrassing Losses). Each win is worth 1 point and each loss is worth 0 points. Quality Win and Embarrassing Loss points are given on a sliding scale based on the opponent's Team Rating. You can earn Quality Win points by beating a team with a rating greater than 0.65, and you will be assessed an Embarrassing Loss penalty by losing to a team with a rating lower than 0.4. (A rating greater than 0.65 usually places a team in the top 15 while a rating below 0.4 corresponds to the bottom 30 teams.) Here's how the scales are calculated:
QW Bonus = 0.5 + (2.5 * ([Opponent's Team Rating] - 0.65))
EL Penalty = -0.5 + (2.5 * ([Opponent's Team Rating] - 0.40))
For those not interested in doing the math, the range of bonus values for a QW is between 0.5 and 1.375 (since Team Rating is never greater than 1). In reality, teams almost never have ratings higher than 0.85 or 0.9 so the effective maximum bonus is between 1 and 1.1. The effect of this bonus is that a win against a very good team is worth at least 1.5 (as opposed to just 1), while beating the top team in the nation counts for about two wins.
Similarly, the penalty for an EL is between -0.5 and -1.5 which has the effect of taking away wins. A loss to a 1-AA team is considered to be a loss to a team with a rating of 0 and is worth the maximum penalty of -1.5.
What is Win/Loss Factor?
Win/Loss Factor is based on a team's win percentage but includes bonuses for beating strong teams (Quality Wins) and penalties assessed for losing to weak teams (Embarrassing Losses). Each win is worth 1 point and each loss is worth 0 points. Quality Win and Embarrassing Loss points are given on a sliding scale based on the opponent's Team Rating. You can earn Quality Win points by beating a team with a rating greater than 0.65, and you will be assessed an Embarrassing Loss penalty by losing to a team with a rating lower than 0.4. (A rating greater than 0.65 usually places a team in the top 15 while a rating below 0.4 corresponds to the bottom 30 teams.) Here's how the scales are calculated:
QW Bonus = 0.5 + (2.5 * ([Opponent's Team Rating] - 0.65))
EL Penalty = -0.5 + (2.5 * ([Opponent's Team Rating] - 0.40))
For those not interested in doing the math, the range of bonus values for a QW is between 0.5 and 1.375 (since Team Rating is never greater than 1). In reality, teams almost never have ratings higher than 0.85 or 0.9 so the effective maximum bonus is between 1 and 1.1. The effect of this bonus is that a win against a very good team is worth at least 1.5 (as opposed to just 1), while beating the top team in the nation counts for about two wins.
Similarly, the penalty for an EL is between -0.5 and -1.5 which has the effect of taking away wins. A loss to a 1-AA team is considered to be a loss to a team with a rating of 0 and is worth the maximum penalty of -1.5.
Tuesday, August 21, 2007
Team Rating = (Power + Efficiency)/2
One of the most important parts of my ranking system is the Team Rating. Team Rating is a value based solely on performance and independent of achievement. What's the difference? In football achievement is measured in wins, losses, and championships. Performance is how a team played, how many yards did they gain, how many points did they score, etc. Very often a team's achievement is not reflected in their performance which is why both should be considered when ranking teams. This is in stark contrast to the pollsters who tend to focus on a team's W/L record above all else. This leads to situations where weak teams luck into high rankings. Case in point, UCLA 2005. The Bruins finished 10-2 with huge blowout losses to USC and craptacular Arizona (3-8). UCLA played four close conference games and needed serious 4th quarter rallies to win or get into overtime. Three of those games were against the worst teams in the Pac-10 (Stanford [5-6], Washington St [4-7], Washington [2-9]). Yet somehow they ended up ranked 13th in the coaches poll. If you look at their Pythagorean Win Percentage, it's 57.9%, which suggests they performed at a level consistent with a 7-5 record, hardly top 15 material. This is precisely why I include a performance based component to my rankings.
What is Team Rating?
As the title of this post suggests, it's the average of a team's Power and Efficiency, which begs the question, What is Power and Efficiency?
Power
A team's Power is an average of a team's normalized Offensive and Defensive moduluses (moduli?). What does Offensive Modulus tell us? It's based on the Yards per Game statistic so, OMod is how good a team is at moving the football, adjusted for the defensive performance of it's opponents. Similarly, Defensive Modulus is based on Yards Given Up per Game and tells us how good a team is at holding the opponent to as few yards as possible. Why use yards and not points? After all, it's points that win the game. Well, the range of possible values a team's score can take goes from about 0 to 70, while a team can gain anywhere from 0 to 600 yards. A lucky break (a fumble recovered for a touchdown) has a much more significant impact (10% of the maximum value) on a team's score than it's total yardage, so yards gained will have less error than points scored.
Efficiency
Efficiency is an average of a team's normalized Yards Per Point and Yards Per Point Given Up (or as I like to call them, Offensive and Defensive Efficiency). These values tell us how good a team is at scoring and keeping their opponent from scoring. Also, as I explained in a previous post they contain other information about a team, such as special teams play and the effect of turnovers.
So by combining, in equal parts, a team's Offensive Modulus, Defensive Modulus, Offensive Efficiency, and Defensive Efficiency we get a value that contains information on a team's ability to gain yards, score points, stop it's opponents from gaining yards and scoring points, and a little bit about special teams and turnovers. Team Rating is a good composite of performance-based metrics and is completely independent of a team's W/L record.
Addendum - Why Normalize?
In order to combine two opposing statistics (one where higher values are better and one where lower values are better) in a meaningful way you have to get them both on the same scale. This is especially true if there is no limit to the possible range of values the stats can take. To normalize the values in a set, you subtract the lowest value from each member and then divide each member by the highest value. The result is a range of values from 0 to 1 where the ordering and relative distance between values is preserved. So if you normalize the Offensive Modulus, the team with the highest score has a value of 1 and the worst team has a value of 0. For Defensive modulus the opposite is true. However, since they're now scaled the same (between 1 and 0), you can average the two by subtracting the normalized defensive modulus values from 1.
What is Team Rating?
As the title of this post suggests, it's the average of a team's Power and Efficiency, which begs the question, What is Power and Efficiency?
Power
A team's Power is an average of a team's normalized Offensive and Defensive moduluses (moduli?). What does Offensive Modulus tell us? It's based on the Yards per Game statistic so, OMod is how good a team is at moving the football, adjusted for the defensive performance of it's opponents. Similarly, Defensive Modulus is based on Yards Given Up per Game and tells us how good a team is at holding the opponent to as few yards as possible. Why use yards and not points? After all, it's points that win the game. Well, the range of possible values a team's score can take goes from about 0 to 70, while a team can gain anywhere from 0 to 600 yards. A lucky break (a fumble recovered for a touchdown) has a much more significant impact (10% of the maximum value) on a team's score than it's total yardage, so yards gained will have less error than points scored.
Efficiency
Efficiency is an average of a team's normalized Yards Per Point and Yards Per Point Given Up (or as I like to call them, Offensive and Defensive Efficiency). These values tell us how good a team is at scoring and keeping their opponent from scoring. Also, as I explained in a previous post they contain other information about a team, such as special teams play and the effect of turnovers.
So by combining, in equal parts, a team's Offensive Modulus, Defensive Modulus, Offensive Efficiency, and Defensive Efficiency we get a value that contains information on a team's ability to gain yards, score points, stop it's opponents from gaining yards and scoring points, and a little bit about special teams and turnovers. Team Rating is a good composite of performance-based metrics and is completely independent of a team's W/L record.
Addendum - Why Normalize?
In order to combine two opposing statistics (one where higher values are better and one where lower values are better) in a meaningful way you have to get them both on the same scale. This is especially true if there is no limit to the possible range of values the stats can take. To normalize the values in a set, you subtract the lowest value from each member and then divide each member by the highest value. The result is a range of values from 0 to 1 where the ordering and relative distance between values is preserved. So if you normalize the Offensive Modulus, the team with the highest score has a value of 1 and the worst team has a value of 0. For Defensive modulus the opposite is true. However, since they're now scaled the same (between 1 and 0), you can average the two by subtracting the normalized defensive modulus values from 1.
YPP, YPPGU or maybe just Efficiency
This post is about a stat that is frequently overlooked but can tell us a good deal about a team. I'm talking about Yards/Point and Yards/Point Given Up. To find a team's YPP, you just divide the total number of yards they've gained on the season by the total number of points they scored. The same works with the defense, YPPGU is just the total number of yards given up divided by the number of points given up. For YPP, lower numbers are better (you're scoring lots of points) and for YPPGU you want higher numbers (you're giving up very few points).
So what can we learn from these numbers?
Yards Per Point:
A team can get a low Yards Per Point number by scoring more touchdowns as opposed to field goals and getting positive field position due to good special teams play and recovering turnovers. Any points scored off of a turnover or on a return do not add any yards to the offense's totals and boost a team's YPP that much more. Just the opposite, a team that is always playing a long field, turning over the ball, and settling for field goals is going to have a very high YPP, and they probably won't win very many games either.
Yards Per Point Give Up:
A team can end up with a high Yards Per Point Given Up number by having a "bend but not break" defense that can force punts and limit the opponent to field goals. Forcing turnovers and pinning your opponent in their own territory with strong return coverage will also drive your YPPGU up. Because of these factors, I like to think of YPP and YPPGU as Offensive and Defensive Efficiency, respectively. I think it sounds better too.
So wouldn't it be nice if I could give a real world example whereby we can pierce the veil of obfuscation created by conventional statistics? Well you're in luck! We need look no further than the 2004 NC State Wolfpack Defense.
I'm confused, is my defense good or bad?
So what can we learn from these numbers?
Yards Per Point:
A team can get a low Yards Per Point number by scoring more touchdowns as opposed to field goals and getting positive field position due to good special teams play and recovering turnovers. Any points scored off of a turnover or on a return do not add any yards to the offense's totals and boost a team's YPP that much more. Just the opposite, a team that is always playing a long field, turning over the ball, and settling for field goals is going to have a very high YPP, and they probably won't win very many games either.
Yards Per Point Give Up:
A team can end up with a high Yards Per Point Given Up number by having a "bend but not break" defense that can force punts and limit the opponent to field goals. Forcing turnovers and pinning your opponent in their own territory with strong return coverage will also drive your YPPGU up. Because of these factors, I like to think of YPP and YPPGU as Offensive and Defensive Efficiency, respectively. I think it sounds better too.
So wouldn't it be nice if I could give a real world example whereby we can pierce the veil of obfuscation created by conventional statistics? Well you're in luck! We need look no further than the 2004 NC State Wolfpack Defense.
I'm confused, is my defense good or bad?In 2004 the Wolfpack lead the nation in total defense, allowing an incredible 226 yards per game. Somehow, they only managed to win 5 of their eleven games, including a 42-0 beat down on 1-AA patsy Richmond. What explains this? Maybe they had a crappy offense? Well their offense was below average (61st), but they still finished in the black by about 120 yds/game and were never outgained by an opponent all season. Well if we take a look at their Defensive Efficiency (YPPGU), they were the second WORST in the nation allowing opponents a point for every 10.4 yards. This means they were giving their opponents a very short field. If we dig deeper into the box scores, we'll see that NC State gave up 31 turnovers (15 fumbles, 16 INTs), a -19 margin! It also didn't help the defense that NC State was 103rd in return coverage, allowing 12.4 yards per return. So yes 2004-Chuck, your defense is very good. You just need to teach the rest of the team how to hold on to the ball!
Thursday, August 16, 2007
Modulus
It's a classic debate... Are Pac-10 teams' offenses strong because they play against weak defenses? Are SEC teams' defenses so strong because they play against stodgy old-school offenses? Given the enormous differences in quality between the best and worst teams in Division 1-A, there needs to be a statistic that gives more information about offensive and defensive strength than the classic Yards and Points per game metrics. There needs to be a statistic that factors in the strength of a team's opponent. Laying half-a-hundred on Temple is a lot less impressive than doing the same to a team like LSU or Ohio State.
Enter the Modulus...
Modulus - A quantity that expresses the degree to which a substance possesses a property.
Modulus Example: Offensive Scoring Modulus or OSMod
The OSMod is intended to replace the Points/Game metric and will give us an idea of how good a team is at putting points on the board.
Here's how it's done:
In three of LSU's games from 2006 the Tigers beat Miss St 48-17, lost to Florida 23-10, and smashed Kentucky 49-0. To find the Tigers' OSMod for these three games we need to divide the number of points LSU scored in each game against the average number of points given up by the opposing defense.
Game 1: Miss St gave up an average 27.8 ppg in 2006. So we divide 48 by 27.8 and get 1.73.
Game 2: Florida gave up 14.5 ppg so 10/14.5 = 0.69
Game 3: Kentucky gave up 30.2 ppg, 49/30.2 = 1.62
To get the total OSMod for these three games we just average the three scores.
(1.73 + 0.69 + 1.62)/3 = 1.35
So what does this mean? It means that LSU typically scores 1.35 times as many points as it's opponents allow on average. A much more telling number than simply 35.7 ppg.
You can, of course, do this for any basic statistic although it might not be worth the effort to compute a team's modulus of penalization (or whatever you might call it).
On a week-to-week basis I keep track of the following moduli (moduluses?):
Offensive Modulus (Yards Per Game)
Defensive Modulus (Yards Given Up Per Game)
Offensive Scoring Modulus (Points Per Game)
Defensive Scoring Modulus (Points Given Up Per Game)
Passing Modulus (Passing Yards Per Game)
Pass Defense Modulus (Pass Yards Given Up Per Game)
Rushing Modulus (Rushing Yards Per Game)
Rush Defense Modulus (Rushing Yards Given Up Per Game)
--
So, How much difference does it make?
Take a look at the average points per game vs the offensive scoring modulus for each of the 11 conferences and independents:
That's a pretty significant shake-up. It also tells us that when it comes to giving up points, the Big East is significantly stingier than the Big XII. Suprised anyone?
Also, only the six BCS conferences managed to crack the 1.0 mark which is the modulus version of breaking even. Another neat thing you can do with this table is divide the Points Per Game by Offensive Scoring Modulus and find out what the average number of points each conference's opponents gave up. For the SEC, it's a mere 21 ppg, for the WAC it's almost 28!
Enter the Modulus...
Modulus - A quantity that expresses the degree to which a substance possesses a property.
Modulus Example: Offensive Scoring Modulus or OSMod
The OSMod is intended to replace the Points/Game metric and will give us an idea of how good a team is at putting points on the board.
Here's how it's done:
In three of LSU's games from 2006 the Tigers beat Miss St 48-17, lost to Florida 23-10, and smashed Kentucky 49-0. To find the Tigers' OSMod for these three games we need to divide the number of points LSU scored in each game against the average number of points given up by the opposing defense.
Game 1: Miss St gave up an average 27.8 ppg in 2006. So we divide 48 by 27.8 and get 1.73.
Game 2: Florida gave up 14.5 ppg so 10/14.5 = 0.69
Game 3: Kentucky gave up 30.2 ppg, 49/30.2 = 1.62
To get the total OSMod for these three games we just average the three scores.
(1.73 + 0.69 + 1.62)/3 = 1.35
So what does this mean? It means that LSU typically scores 1.35 times as many points as it's opponents allow on average. A much more telling number than simply 35.7 ppg.
You can, of course, do this for any basic statistic although it might not be worth the effort to compute a team's modulus of penalization (or whatever you might call it).
On a week-to-week basis I keep track of the following moduli (moduluses?):
Offensive Modulus (Yards Per Game)
Defensive Modulus (Yards Given Up Per Game)
Offensive Scoring Modulus (Points Per Game)
Defensive Scoring Modulus (Points Given Up Per Game)
Passing Modulus (Passing Yards Per Game)
Pass Defense Modulus (Pass Yards Given Up Per Game)
Rushing Modulus (Rushing Yards Per Game)
Rush Defense Modulus (Rushing Yards Given Up Per Game)
--
So, How much difference does it make?
Take a look at the average points per game vs the offensive scoring modulus for each of the 11 conferences and independents:
| POINTS PER GAME (Conference Averages) | |
|---|---|
| Big XII | 26.593 |
| WAC | 26.556 |
| Big East | 26.363 |
| Big Ten | 24.715 |
| SEC | 24.176 |
| Conference USA | 23.747 |
| Pac-10 | 23.712 |
| Mountain West | 22.841 |
| Independents | 21.551 |
| ACC | 21.438 |
| MAC | 21.027 |
| Sun Belt | 16.681 |
| OFFENSIVE SCORING MODULUS (Conference Averages) | |
|---|---|
| Big East | 1.18272 |
| SEC | 1.14331 |
| Big XII | 1.08517 |
| Big Ten | 1.06165 |
| Pac-10 | 1.03700 |
| ACC | 1.01066 |
| Mountain West | 0.97037 |
| WAC | 0.95246 |
| Conference USA | 0.92564 |
| Independents | 0.90664 |
| MAC | 0.85022 |
| Sun Belt | 0.66869 |
That's a pretty significant shake-up. It also tells us that when it comes to giving up points, the Big East is significantly stingier than the Big XII. Suprised anyone?
Also, only the six BCS conferences managed to crack the 1.0 mark which is the modulus version of breaking even. Another neat thing you can do with this table is divide the Points Per Game by Offensive Scoring Modulus and find out what the average number of points each conference's opponents gave up. For the SEC, it's a mere 21 ppg, for the WAC it's almost 28!
The Numbers Game
The reason I started this blog was to give myself a place to post the results of my College Football Ranking system. After the 4th or 5th week of the season I'll begin posting the results and any necessary analysis. Until then I'll post the results from last year along with some explanations about how the rankings work and the reasoning behind them. Comments and criticisms are welcome. I'm not a statistician so I'd like to know if I'm pushing bad science. And, just so everyone knows where I stand I'm a Georgia Tech alumnus who grew up in Baton Rouge so...
Go Jackets!
Geaux Tigers!
Go Jackets!
Geaux Tigers!
Subscribe to:
Posts (Atom)