Monday, November 05, 2007

On Polling

So I just posted this week's Lebowski Ratings. In case you've never seen them before, the reason why I started them was to create a list based on purely objective rationales that are simple and easily understood for all programs prior to beginning the season and which attempts to eliminate from the equation factors outside of the control of teams. The goal was to combat what I viewed as the flaws in the current system of opinion-based polling: that the pollsters do not have enough information or receive biased information; polling inertia; overreliance on ill-formed preseason predictions; etc. The result is a system where if you win, you move up; if you lose, you move down. If there is a tie, the tie goes to strength of schedule - the one aspect outside the control of players.

So that's what's good (in my view) about those Lebowski Ratings - minimal subjectivity, teams know the rules going in of what they have to do. But I'd be a fool not to admit that there aren't flaws. First among those flaws is that there's no subjectivity, and therefore the result is only what the data provide - it doesn't tell us who is good, or even who is better, only whose results have been the best. Adding subjectivity would allow us to answer those questions.

Traditional opinion polls, like the AP, the Coaches' Poll or even the BlogPoll do use subjectivity. They're opinion polls, so in a way, they're all subjective. Naturally, in any of those polls, a ballot that has no connection to the on-field results would be mocked and possibly spiked, removed from the aggregate poll. That's a good thing. But for the most point, as long as there's some logic behind an opinion, a voter can have whatever opinion he thinks. Team X loses on the field to Team Y, but many voters still think X is better, and it's reasonable to rank X higher. Team H has no losses playing a joke of a schedule - pollsters can vote Team L ahead of them because even though they have a loss, their difficult schedule makes them a better team. Subjectivity can produce better results than simply looking at the numbers - context can be very important.

Here's where I take issue: too many pollsters aren't using enough subjectivity, aren't putting enough things in proper context.

Subjective opinion is a tremendous power, and in many instances, it goes underused, or perhaps used only situationally. Here are a few situations and I'll develop my ideas a bit more.

Situation I - Strength of Schedule vs. Won/Loss Record

This is a common situation, most often seen early in the year. A team starts off winning a series of games against weak opponents. Other teams play significantly tougher opponents, but end up losing one or two. A pollster could be on solid ground by keeping the team that has lost a game or two against difficult opponents to the exclusion of the team that hasn't lost but hasn't played anyone good either. Now, using that logical position is fine, but it carries with it some pitfalls.

A. Teams with built-in support in polls receive the benefit of the doubt when playing weak opponents and winning. It's one thing if Ohio State opens the season by playing Youngstown State and Akron and winning both. It's another entirely if Kansas opens with Central Michigan and Southeastern Louisiana and wins both. The opening two weeks of the season for both teams were virtually identical (one 1-AA/FCS opponent, one MAC opponent). But Ohio State has played in several BCS games this decade and has won national titles. Kansas is a traditional doormat. Pollsters are far more likely to continue to rank highly a team like Ohio State, despite the lack of quality opponents, than they would be to rank highly a team like Kansas, who would be viewed with a "let's wait and see how they fare when they actually play someone" approach. That's a fundamental flaw. If you view one team as unproven until they face a strong opponent, voters should view all teams that way, regardless of how many championship banners are flying. It's possible that the team without the historic pedigree can continue winning, while the historically great team could end up falling apart. The point is that if "playing top competition" is a prerequisite a voter has for any team, that voter should apply such rule to all teams.

B. The flip side: later on in the year, if playing a strong schedule is enough to boost some teams, it should boost all teams similarly situated, and if playing a weak schedule is enough to mark down some teams, it should be enough to mark down all teams similarly situated. Over the last few weeks some online writers have focused on Hawaii's terrible schedule as a reason why voters should not rank the Warriors highly solely on the basis of their null set in the loss column. This makes great sense. However, Hawaii is not the only team ranked highly in many polls that has a schedule leaving much to be desired. If the Warriors are to be discredited for their weak schedule, so should Kansas, Boise State, Missouri, Cincinati, Southern California, and a few others, all of whom have played schedules ranked in the bottom third of the country. If a team like LSU is getting credit for playing a tremendously difficult schedule, then teams down the list should also get credit - like Florida, South Florida, Alabama, Florida State, Oregon, etc. And in some instances that is the case (such as Florida with 3 losses ahead in this week's polls of several teams with fewer losses) - but we should also be careful that all teams who play weak or strong schedules are treated with the same skepticism/praise.

The point is that if strength of schedule matters at all, it should matter to all. It should not simply be a prop one relies on when it suits already existing opinions.

Situation II - Wins and Losses in Proper Context

The greatest advantage human opinion polls have over computer polls or an objective listing like the Lebowski Rankings is the ability to take numbers and put them in proper context.

When Virginia ekes by its 5th opponent by 2 points or less, a computer or an objective ratings system merely adds another number to the wins column. The margin of the win, and whether any luck or externalities played a role in the result do not matter. Well, they can matter to human voters - but, and here's where I go against most of what I write, they don't matter enough.

If we're going to have subjective voting, go full out with it and don't be afraid of the objective getting in the way. Use context.

For example, let's look at two teams and how they've been presented this year. Lousiville and Connecticut. Louisville is considered one of the great disappointments of the season, currently 5-4. UConn is considered one of the great stories of the season, currently 8-1.

But let's look at the context. Louisville's strength of schedule is significantly more difficult than UConn, primarily because of two difficult OOC opponents (Utah and Kentucky), while UConn only has played one difficult OOC opponent (Virginia). Both teams played a 1-AA/FBS opponent, but UConn's remaining OOC is among the weakest in the nation (Duke, Akron, Temple), while Louisville played NC State and Middle Tennessee in addition to Utah and Kentucky. Louisville's third toughest OOC opponent just beat UConn's toughest opponent. Next let's look at exactly how these teams won/lost. In each of Louisville's losses, the end result has been in question late in the game. Against Kentucky, Louisville lost on a TD with 28 seconds to go. Against Syracuse, an onside kick recovered by Syracuse in the last minute secured the game (though, yes, it shouldn't have been that close). Against Utah, a FG with a minute to go put the game away, but that FG was the result of a Ute recovery on an onside kick. As for the UConn loss, I'll mention that in a second. The point is that Louisville has been able to compete relatively closely in each of their games, and if a single bounce on an onside kick or an incomplete pass had gone their way, it's possible a different result could've come out.

Then there's Connecticut. They lost to Virginia on a chip shot field goal with three minutes to go. But in two of their wins, the proper result was probably not achieved. In the Temple game, the Owls scored a TD to take the lead with 40 seconds to go, but the officiating crew incorrectly ruled the player out of bounds, which was inexplicably upheld on replay review. An error by the officials gifted UConn a win over Temple. The Louisville game was equally egregious. Seven points were incorrectly awarded to UConn after Larry Taylor illegally signalled for a fair catch on a punt and then proceeded to carry the ball 72 yards for a touchdown. What should have been a five yard delay of game penalty became 7 illegally obtained points. When the clock ran out, the scoreboard showed UConn ahead by 4 points.

The point is that Louisville is 3 bounces and a terrible official's call away from being undefeated. UConn is two atrocious officiating mistakes (both on their home field) away from being 6-3. In fact, if you just corrected the egregious officiating errors in UConn's record, both of these teams should be 6-3, with Louisville winning the head-to-head game.

And that's where I see the flaw: since poll voters can use their faculties to provide context to wins and losses, they should do it. Last year, when Oregon and Pac-10 officials stole a win against Oklahoma, I thought immediately of one thing: It's a great thing that we have subjective voting in college football because now the voters can correct the mistakes of the officials. A call gets blown and it costs a team a win - the voters don't have to accept the result on the field. They can say, with flawless logic, that Oregon did not beat Oklahoma by their own efforts, and therefore the voter doesn't have to accept the additional loss for the Sooners and the additional win by the Ducks. Alas, the next day the voters didn't do that. Instead the AP voters moved Oregon ahead of Oklahoma.

Similarly, voters can use their subjectivity to view UConn's ill-gotten wins in proper context - as in, not as wins at all. Is UConn deserving of a current #16 ranking? Maybe. But if they had a 6-3 record instead of 8-1 would they be ranked that high? Voters don't have to accept that 8-1. They can correct in their ballots the errors on the field. Use the power of that subjectivity (and of course, use it fairly and equally).

I've been writing for 3 years about how subjectivity in college football is a serious problem, mainly focusing on incomplete or unfair media coverage and inadequacies of the polling system. I still believe that a purely objective system of determining champions in college football should be the goal (though one I readily admit is nowhere near us). But I recognize that there are certainly some benefits of having subjectivity in college football - it allows us to provide context to wins and losses. The problem is that voters aren't using that subjectivity well enough - because they don't have adequate information in a broad and national sport where detailed coverage remains regional; because they still rely on quick persuals of scoreboards and glimpses of highlights instead of reading detailed recaps or watching complete games; because particular rules of construction when filling out ballots don't always apply to all teams equally (like how some teams require a higher standard of competition to be beaten than others before getting ranked).

Subjectivity can be incredibly useful in answering the question of who is the best. But it needs to be used properly, fairly, and with all its force. I don't think voters today are doing a good enough job of that.


Anonymous said...

So, when your own team was undefeated and underrated, you created these standings because you thought your team was getting jobbed. Now that teams you don't like are ahead of teams you do like, you have to change your ranking system because sometimes subjectivity is good when it lets you put LSU above Ohio State. Nice.

peacedog said...

LD, computers can do what you ask - evaluate those 5 2point or less wins in a certain way. They just have to be told how to do so. And, it could be difficult to have the computer cover every desired scenario, particularly since every scenario won't be envisioned up front.

We could give a comptuer based ranking system an algorithm for determing quality wins, quality losses, bad wins, bad losses. We would still need to agree on what criteria count, and how exactly these things should factor into the final rankings.

LD said...

Anonymous, leave a name. It doesn't cost anything.

I created the standings 3 years ago because in my view, sport should have no subjectivity - no judging, no style points. Sport should be about scoring more points, winning and losing, finishing the race first - objectively quantifiable results, which are readily known to participants before the season/game starts. College football doesn't work that way. It relies on predetermined viewpoints, the appearance of style, and a system of judges which have open or latent biases. I created the rankings because I'd like college football to be less like figure skating.

And I haven't changed a thing about those rankings. They use a specific formula. I didn't devise the formula because I thought my team was getting jobbed (if you can even tell me what "my team" is and how that would work, please go right ahead). And I haven't changed the formula because of results I disagreed with. Last year Boise State topped the rankings at the end of the year. Didn't mean I thought they'd beat Florida on a neutral field. Did mean that they topped the ratings. There's nothing outcome-driven about the ratings. And I haven't changed a thing about them.

Now, as for this post, which I'm not sure you really read... Just because I don't like the fact that subjectivity plays a role in college football at all doesn't mean it's changing. College football has and will continue to rely on subjective opinions to crown champions and aid in bowl selections. I don't think it's right conceptually, but it's going to be the case realistically for the next decade or more. So if we're stuck with the use of subjective analysis, my position is that voters should actually use that subjective analysis - fairly, equally and with maximum information.

If the question is Ohio State vs. LSU, let me be clear. From the perspective of the rankings I developed and from my personal view, Ohio State should be #1 because nobody has beaten them, period. It's not fair to impute a loss onto Ohio State because of schedule or some other reason when no team has been able to accomplish that on the field. That said, "should be #1" does not necessarily mean "is the best team". That's a question I personally don't care much about. That's my view. Now, from the perspective of the real world, voters don't vote solely on "should" or "deserves", they vote on who they think is the best team. It's a different question. And if they're using subjective analysis, go all the way. If a voter thinks LSU is the best team in the country and better than Ohio State, but votes Ohio State number one simply because their record is better, I think that voter isn't answering the question fairly.

It's a value judgment. If we want objectivity to rule the day, don't rely on opinions at all. If we want subjectivity to rule the day, rely only on opinions. My problem is when in an opinion poll, voters sacrifice what they actually think about teams because of objective results - for two reasons: (a) because an opinion poll should simply measure opinions; and (b) because opinions are often dreadfully wrong, it's my belief that if pollsters only relied on their actual opinions the pollsters would be exposed as the ill-informed dreck that they often are, and then maybe we'd start moving away from allowing opinion polling to control things.

9:12 AM

LD said...

Peacedog, I get what you're saying and I often view computers as a better source of actual info than the flawed human polling.

That said, just creating another algorithm that puts games in proper context won't change the current system. Human polls rule (the AP champ claims a title, the BCS is dominated by human polls). Human polls are flawed. That's the big problem, and another computer system, even if perfectly accurate, won't change things.

peacedog said...

I completely agree - no change to computer polls is going to fix the system. I was just throwing it out there.

Anonymous said...

I did not know if the names were free, or I had to sign up for something. Your own team is the Georgia Bulldogs, who were an undefeated but lower ranked team in 2005.
Much of my post was factitious. I was just ribbing you after many comments regarding the Big Ten in general and the Buckeyes in specific. I read your site often and was wondering when it would bother you that Ohio State was over LSU despite the strength of schedule difference. There is very little I disagree with you in the actual meat of the post. If I'm sorry about anything, it's oversimplifying a well thought out post to make a joke. Mah Bahd Dude.

LD said...

I see.

This post probably needed editing anyway - way too long.

I guess my main points are:

1) I prefer objective ranking.
2) If pollsters are using subjectivity, they should be subjective all the way:
(A) If they think LSU (or Oregon for that matter) is better than OSU, they shouldn't let the 0 in the loss column for OSU stop them from putting LSU (or Oregon) ahead of the Buckeyes.
(B) If they think a team stinks but has a good record (like Hawaii or Connecticut), they shouldn't just let the good record be reason enough to rank them highly when there are significant reasons not to do so.

I guess it's just a pleading to have voters actually say what they feel.

peacedog said...

No, UGA wasn't "undefeated but lower ranked" in 2005. Regular season record was 9-2.