On Polling

So I just posted this week's Lebowski Ratings. In case you've never seen them before, the reason why I started them was to create a list based on purely objective rationales that are simple and easily understood for all programs prior to beginning the season and which attempts to eliminate from the equation factors outside of the control of teams. The goal was to combat what I viewed as the flaws in the current system of opinion-based polling: that the pollsters do not have enough information or receive biased information; polling inertia; overreliance on ill-formed preseason predictions; etc. The result is a system where if you win, you move up; if you lose, you move down. If there is a tie, the tie goes to strength of schedule - the one aspect outside the control of players.


So that's what's good (in my view) about those Lebowski Ratings - minimal subjectivity, teams know the rules going in of what they have to do. But I'd be a fool not to admit that there aren't flaws. First among those flaws is that there's no subjectivity, and therefore the result is only what the data provide - it doesn't tell us who is good, or even who is better, only whose results have been the best. Adding subjectivity would allow us to answer those questions.

Traditional opinion polls, like the AP, the Coaches' Poll or even the BlogPoll do use subjectivity. They're opinion polls, so in a way, they're all subjective. Naturally, in any of those polls, a ballot that has no connection to the on-field results would be mocked and possibly spiked, removed from the aggregate poll. That's a good thing. But for the most point, as long as there's some logic behind an opinion, a voter can have whatever opinion he thinks. Team X loses on the field to Team Y, but many voters still think X is better, and it's reasonable to rank X higher. Team H has no losses playing a joke of a schedule - pollsters can vote Team L ahead of them because even though they have a loss, their difficult schedule makes them a better team. Subjectivity can produce better results than simply looking at the numbers - context can be very important.

Here's where I take issue: too many pollsters aren't using enough subjectivity, aren't putting enough things in proper context.

Subjective opinion is a tremendous power, and in many instances, it goes underused, or perhaps used only situationally. Here are a few situations and I'll develop my ideas a bit more.

Situation I - Strength of Schedule vs. Won/Loss Record

This is a common situation, most often seen early in the year. A team starts off winning a series of games against weak opponents. Other teams play significantly tougher opponents, but end up losing one or two. A pollster could be on solid ground by keeping the team that has lost a game or two against difficult opponents to the exclusion of the team that hasn't lost but hasn't played anyone good either. Now, using that logical position is fine, but it carries with it some pitfalls.

A. Teams with built-in support in polls receive the benefit of the doubt when playing weak opponents and winning. It's one thing if Ohio State opens the season by playing Youngstown State and Akron and winning both. It's another entirely if Kansas opens with Central Michigan and Southeastern Louisiana and wins both. The opening two weeks of the season for both teams were virtually identical (one 1-AA/FCS opponent, one MAC opponent). But Ohio State has played in several BCS games this decade and has won national titles. Kansas is a traditional doormat. Pollsters are far more likely to continue to rank highly a team like Ohio State, despite the lack of quality opponents, than they would be to rank highly a team like Kansas, who would be viewed with a "let's wait and see how they fare when they actually play someone" approach. That's a fundamental flaw. If you view one team as unproven until they face a strong opponent, voters should view all teams that way, regardless of how many championship banners are flying. It's possible that the team without the historic pedigree can continue winning, while the historically great team could end up falling apart. The point is that if "playing top competition" is a prerequisite a voter has for any team, that voter should apply such rule to all teams.

B. The flip side: later on in the year, if playing a strong schedule is enough to boost some teams, it should boost all teams similarly situated, and if playing a weak schedule is enough to mark down some teams, it should be enough to mark down all teams similarly situated. Over the last few weeks some online writers have focused on Hawaii's terrible schedule as a reason why voters should not rank the Warriors highly solely on the basis of their null set in the loss column. This makes great sense. However, Hawaii is not the only team ranked highly in many polls that has a schedule leaving much to be desired. If the Warriors are to be discredited for their weak schedule, so should Kansas, Boise State, Missouri, Cincinati, Southern California, and a few others, all of whom have played schedules ranked in the bottom third of the country. If a team like LSU is getting credit for playing a tremendously difficult schedule, then teams down the list should also get credit - like Florida, South Florida, Alabama, Florida State, Oregon, etc. And in some instances that is the case (such as Florida with 3 losses ahead in this week's polls of several teams with fewer losses) - but we should also be careful that all teams who play weak or strong schedules are treated with the same skepticism/praise.

The point is that if strength of schedule matters at all, it should matter to all. It should not simply be a prop one relies on when it suits already existing opinions.

Situation II - Wins and Losses in Proper Context

The greatest advantage human opinion polls have over computer polls or an objective listing like the Lebowski Rankings is the ability to take numbers and put them in proper context.

When Virginia ekes by its 5th opponent by 2 points or less, a computer or an objective ratings system merely adds another number to the wins column. The margin of the win, and whether any luck or externalities played a role in the result do not matter. Well, they can matter to human voters - but, and here's where I go against most of what I write, they don't matter enough.


If we're going to have subjective voting, go full out with it and don't be afraid of the objective getting in the way. Use context.

For example, let's look at two teams and how they've been presented this year. Lousiville and Connecticut. Louisville is considered one of the great disappointments of the season, currently 5-4. UConn is considered one of the great stories of the season, currently 8-1.

But let's look at the context. Louisville's strength of schedule is significantly more difficult than UConn, primarily because of two difficult OOC opponents (Utah and Kentucky), while UConn only has played one difficult OOC opponent (Virginia). Both teams played a 1-AA/FBS opponent, but UConn's remaining OOC is among the weakest in the nation (Duke, Akron, Temple), while Louisville played NC State and Middle Tennessee in addition to Utah and Kentucky. Louisville's third toughest OOC opponent just beat UConn's toughest opponent. Next let's look at exactly how these teams won/lost. In each of Louisville's losses, the end result has been in question late in the game. Against Kentucky, Louisville lost on a TD with 28 seconds to go. Against Syracuse, an onside kick recovered by Syracuse in the last minute secured the game (though, yes, it shouldn't have been that close). Against Utah, a FG with a minute to go put the game away, but that FG was the result of a Ute recovery on an onside kick. As for the UConn loss, I'll mention that in a second. The point is that Louisville has been able to compete relatively closely in each of their games, and if a single bounce on an onside kick or an incomplete pass had gone their way, it's possible a different result could've come out.

Then there's Connecticut. They lost to Virginia on a chip shot field goal with three minutes to go. But in two of their wins, the proper result was probably not achieved. In the Temple game, the Owls scored a TD to take the lead with 40 seconds to go, but the officiating crew incorrectly ruled the player out of bounds, which was inexplicably upheld on replay review. An error by the officials gifted UConn a win over Temple. The Louisville game was equally egregious. Seven points were incorrectly awarded to UConn after Larry Taylor illegally signalled for a fair catch on a punt and then proceeded to carry the ball 72 yards for a touchdown. What should have been a five yard delay of game penalty became 7 illegally obtained points. When the clock ran out, the scoreboard showed UConn ahead by 4 points.

The point is that Louisville is 3 bounces and a terrible official's call away from being undefeated. UConn is two atrocious officiating mistakes (both on their home field) away from being 6-3. In fact, if you just corrected the egregious officiating errors in UConn's record, both of these teams should be 6-3, with Louisville winning the head-to-head game.

And that's where I see the flaw: since poll voters can use their faculties to provide context to wins and losses, they should do it. Last year, when Oregon and Pac-10 officials stole a win against Oklahoma, I thought immediately of one thing: It's a great thing that we have subjective voting in college football because now the voters can correct the mistakes of the officials. A call gets blown and it costs a team a win - the voters don't have to accept the result on the field. They can say, with flawless logic, that Oregon did not beat Oklahoma by their own efforts, and therefore the voter doesn't have to accept the additional loss for the Sooners and the additional win by the Ducks. Alas, the next day the voters didn't do that. Instead the AP voters moved Oregon ahead of Oklahoma.

Similarly, voters can use their subjectivity to view UConn's ill-gotten wins in proper context - as in, not as wins at all. Is UConn deserving of a current #16 ranking? Maybe. But if they had a 6-3 record instead of 8-1 would they be ranked that high? Voters don't have to accept that 8-1. They can correct in their ballots the errors on the field. Use the power of that subjectivity (and of course, use it fairly and equally).




I've been writing for 3 years about how subjectivity in college football is a serious problem, mainly focusing on incomplete or unfair media coverage and inadequacies of the polling system. I still believe that a purely objective system of determining champions in college football should be the goal (though one I readily admit is nowhere near us). But I recognize that there are certainly some benefits of having subjectivity in college football - it allows us to provide context to wins and losses. The problem is that voters aren't using that subjectivity well enough - because they don't have adequate information in a broad and national sport where detailed coverage remains regional; because they still rely on quick persuals of scoreboards and glimpses of highlights instead of reading detailed recaps or watching complete games; because particular rules of construction when filling out ballots don't always apply to all teams equally (like how some teams require a higher standard of competition to be beaten than others before getting ranked).

Subjectivity can be incredibly useful in answering the question of who is the best. But it needs to be used properly, fairly, and with all its force. I don't think voters today are doing a good enough job of that.