S&P+ rankings: Overperformance or overcorrection?

[satyanash 14276]

Wanted to bring this up as a curious footnote to our 6-1 start to the season so far.

We've been plummeting like a rock in the S&P+ rankings week by week. For whatever reason (possibly because they take into account MOV), the S&P+ ratings has been hating us more and more with each successive game. Using S&P+ ratings to calculate win probabilities for individual games paints an even more dire picture.

The current S&P+ ratings have us #43 in the country. Not only are we behind Oklahoma, A&M, and USC, we're also behind teams like North Texas, Purdue, and Missouri. S&P+ ratings. This week, we dropped 5 spots after beating Baylor (#59). Last week, we dropped 8 spots after beating Oklahoma 48-45, while Oklahoma somehow stayed put at #6. That was the same week Bill Connolly wrote a piece claiming we weren't back, and that we wouldn't be until we started beating lower-ranked teams by bigger margins.

So here's the thing. S&P+ ratings are meant to be predictive of future success. In other words, they're not bashing our resume so much as they're saying our W-L record is not at all indicative of how good we are as a team, and that a course correction would involve us losing several games in the back half of our schedule. So is this a fault of the model itself? Or are we due to come back to Earth against Oklahoma State and the rest of our schedule?

[Magus Ossis 2773]

A model that drops the winner and sustains the loser of a game needs adjusting.

[Drew80 15]

I think it's a bit of both. We're winning, but not convincingly, and we keep finding ways to let our opponents stay in the game. Blowing a 21-point lead, missing two field goals, etc etc. We're much improved over last season, and with all our guys playing full throttle we look very good, but we're young and inconsistent.

That said, the model is clearly broken putting us at #43. Anyone who thinks North Texas would compete with us is stupid, but anyone who thinks we're a top 5 team is equally stupid.

Edited October 14, 2018 by Drew80

[Bevo14 8260]

Nobody is more amazing at beating up cupcakes and then folding like pussies against decent competition than OU. Cracks me up how they fool people every year

Edited October 14, 2018 by Bevo14

[Vito Andolini 9086]

Remember when sweater vest used to win all those games by slim margins up in Ohio? A W is a W, championships count the same.

I’d love to Alabama everyone, but we’re learning to win, and after the last 8 years, that matters more to me than anything else.

[longhornsftmfw 129]

Wasn't there a tOSU team back in the early 00s that barely won a bunch of games but then made the NC game? Was that the year Miami beat them? I wonder what their S&P+ looked like throughout that year.

[longhornsftmfw 129]

Just now, Vito Andolini said:

Remember when sweater vest used to win all those games by slim margins up in Ohio? A W is a W, championships count the same.

I’d love to Alabama everyone, but we’re learning to win, and after the last 8 years, that matters more to me than anything else.

Haha, nice. Beat me to it.

[GabrielsHorn 2142]

The model is flawed as it only accounts for end of game stats instead of what happens during the game. 

Either way, we are a showcase of a limit of its model as I don't know how we're behind USC and TCU despite beating them by multiple scores.

[Zavala 1384]

Shit algorithm, shit results. Takes ZERO game plan or weather conditions or play styles into effect. Rewards teams for running up the score on shitty teams. 

We drop spots after beating a #6 team on a neutral field, and they stay at 6? Sure thing. Sports writers who bring up this stupid shit should be ignored forever.

[Vito Andolini 9086]

3 minutes ago, longhornsftmfw said:

Haha, nice. Beat me to it.

Great minds... and the Buckeyes actually won that game, the infamous pass interference call. 

[horn_of_the_morning 978]

S&P+ values efficiency and explosiveness. We are horrible at both. Per Bill Connelly it is "the opponent adjusted and tempo adjusted look at college football efficiency".

[GabrielsHorn 2142]

Also some fun results, Oregon stays the same after beating Washington, Michigan State dropped after beating Penn State, Tech drops after beating TCU, UVA drops after beating Miami, and Tennessee drops after beating Auburn. 

Edited October 14, 2018 by GabrielsHorn

[Zavala 1384]

6 minutes ago, Vito Andolini said:

Remember when sweater vest used to win all those games by slim margins up in Ohio? A W is a W, championships count the same.

I’d love to Alabama everyone, but we’re learning to win, and after the last 8 years, that matters more to me than anything else.

Yes.

Plus, as fucking stressful as it is, we are getting more and more clutch experience for our guys. Can you compete when your back is against the wall? Well, yeah done it all season.

[longhornsftmfw 129]

2 minutes ago, Vito Andolini said:

Great minds... and the Buckeyes actually won that game, the infamous pass interference call. 

Yesss, crazy game.

Dropping after beating a highly ranked rival at a neutral site, no matter the performance is pretty dumb. I don't think we're 2001 or 2002 tOSU by any means, but dropping after beating OU in Dallas shows this ranking is devoid of any real-world intangibles, imo.

[satyanash 14276]

The thing is, S&P+ isn't necessarily broken. @Huckleberryuses it in his models, I think. 

And Sagarin's ratings, which also use MOV, have us at #16 in the country. So it clearly isn't just that we're winning by close margins.

[TheYoungHorn 57]

It's fun that Texas is actually winning games so us fans can now join the complaining against computers debate again.

 

[TheYoungHorn 57]

Quote

A reminder: S&P+ is intended to be predictive and forward looking.

Good predictive ratings are not résumé ratings, and they don’t give you bonus points for wins and losses. They simply compare expected output to actual output and adjust accordingly. That’s how a given team can win but plummet or lose and move up.

Through seven weeks, the S&P+ rankings are performing well, hitting 54 percent against the spread and 53 percent on the over/under point totals for the year.

 

[horn_of_the_morning 978]

8 minutes ago, satyanash said:

The thing is, S&P+ isn't necessarily broken. @Huckleberryuses it in his models, I think. 

And Sagarin's ratings, which also use MOV, have us at #16 in the country. So it clearly isn't just that we're winning by close margins.

It penalizes us because we are not explosive.

[ClubWhatever 12085]

I feel like the only reason we are winning these games is because we are not losing the turnover battle.  Great teams can beat people even when they turn the ball over.  We can’t.  

[Bookman 4610]

We're 6-1 but I honestly don't know how good we are. But winning is sure nice.

[Vic Mackey 46919]

Who the fuck cares about this nerdy shit. We are 6-1 and are about to be ranked near the top 5 in the only polls that actually matter, not this junk.

[longhornsftmfw 129]

1 minute ago, Bookman said:

We're 6-1 but I honestly don't know how good we are. But winning is sure nice.

Yep, second half of the season will be very interesting to watch. Can't wait to see how we do. This season has given us a good data point in proving that Herman can improve the team year-over-year, and I kinda feel like we're playing with house money at this point. I had us winning 9 at the beginning of the season, to be 2/3 of the way there with half the season left is encouraging. If we close how we started (save MD, of course), the improvement in the W/L column will only serve to inflate the expectations within the team of what they can accomplish next season, as will the close wins that would have been losses just last year. Confidence for this team is a good thing.

[horn_of_the_morning 978]

11 minutes ago, ClubWhatever said:

I feel like the only reason we are winning these games is because we are not losing the turnover battle.  Great teams can beat people even when they turn the ball over.  We can’t.  

Winning the turnover battle is one of  the strongest predictor of who will win a football game. It is not a coincidence that it is one of the major points of emphasis in Herman's program. And therefore it is also not a coincidence  that we are winning the turnover battle. In Herman's program ball security and creating turnovers is paramount and will get you becnhed if you are lacking in those departments, no matter how talented you are. Best example is Toneil Carter. He would probably be getting 10-15 snaps in say Charlie's program if ony speed and talent determined who saw action.

Edited October 14, 2018 by horn_of_the_morning

[Trey3216 15325]

39 minutes ago, Vito Andolini said:

Great minds... and the Buckeyes actually won that game, the infamous pass interference call. 

I call it the “Collective Gasp” game after our collective reaction to Willis McGahee’s knee doing things it wasn’t supposed to 

[Machinator 32097]

 

[taybo20 2815]

We are breaking that algorithm. I’ve been hoping we could win grind it out games. Basically when everything is on the line, we hold the line. Last year we pretty much did the opposite. This year, when we’ve needed to, we have pretty much come through every time. It’s very nice to see. 

 

[Hozz 1012]

This is some baseball level nerd shit.  I don't know how it works but when you beat #6 and they stay put while you drop 8 spots, that is clearly retarded.  538 should hire the S&P+ guy, he'd fit right in.

[Huckleberry 32685]

1 hour ago, satyanash said:

The thing is, S&P+ isn't necessarily broken. @Huckleberryuses it in his models, I think. 

And Sagarin's ratings, which also use MOV, have us at #16 in the country. So it clearly isn't just that we're winning by close margins.

I don't use S&P+ at all, mine are completely independent.

I've got us as #11 but #14 in my power rating. We are a Top 15 team in terms of team strength. S&P+ is "incorrect" when it comes to our team so far this year. The interesting thing is that we are about to come up on a stretch of 4 games which are basically coin flips according to the numbers.

[Valmy77 13092]

We need to get Cumberland College on the schedule.

[Treefidy 1463]

1 hour ago, TheYoungHorn said:

 

Performing well, slightly better than flipping a coin.  

Take that to vegas and you will eventually go broke on the juice

[Huckleberry 32685]

As for Oklahoma staying put while we dropped when we beat them, that's actually easier to explain than our current rating. Advanced ratings take a look at all games played by every team, so the same week we played Oklahoma, the following games happened:

Tulsa 26, Houston 41 (we had played Tulsa)
Kansas St. 34, Baylor 37 (we had played KSU, they had played Baylor)
Iowa St. 48, Oklahoma St. 42 (they had played ISU)
Washington 31, UCLA 24 (they had played UCLA who overperformed their previous games in this one)
Florida Atlantic 52, Old Dominion 33 (they had played FAU and Old Dominion had shocked VT, now there's more data on them)
Maryland 21, Michigan 42 (we had played and lost to Maryland)

So one result between the two teams should have moved us up and Oklahoma down. 6 other results should have moved us down and/or Oklahoma up.

Edited October 14, 2018 by Huckleberry

[Huckleberry 32685]

5 minutes ago, Treefidy said:

Performing well, slightly better than flipping a coin.  

Take that to vegas and you will eventually go broke on the juice

You only need to hit 52.4% of your bets at -110 juice to break even.

Edited October 14, 2018 by Huckleberry

[taybo20 2815]

@Huckleberry what’s your website with your rankings?

[Huckleberry 32685]

http://www.adjustedstats.com/

[OU Sucks 8744]

1 hour ago, satyanash said:

Not only are we behind Oklahoma, A&M, and USC, we're also behind teams like North Texas, Purdue, and Missouri. S&P+ ratings.

Advantage aggy

[TheYoungHorn 57]

11 minutes ago, Treefidy said:

Performing well, slightly better than flipping a coin.  

Take that to vegas and you will eventually go broke on the juice

How is your model performing this season? 

[TheYoungHorn 57]

1 hour ago, horn_of_the_morning said:

It penalizes us because we are not explosive.

Or efficient. 

Or consistent. 

[Pato del Muerto 52322]

6 minutes ago, TheYoungHorn said:

Or efficient. 

Or consistent. 

We are doing one thing consistently. Winning. 

[gmr548 16463]

1 hour ago, horn_of_the_morning said:

S&P+ values efficiency and explosiveness. We are horrible at both. Per Bill Connelly it is "the opponent adjusted and tempo adjusted look at college football efficiency".

That description actually of makes the lower ranking make sense. We have fucked around with three bad to mediocre teams and lost to another. We let Oklahoma, our signature win, back into the game to the point that statistically the game was essentially a draw. We beat USC and TCU soundly, but TCU is cratering and USC has their share of issues as well. These rankings are basically punishing us for consistently playing lower tier teams down to the last possession, which is a fair knock. Is it too harsh? Probably, but we are overrated in the human polls. The truth, as usual, is somewhere in the middle.

The poster upthread laughed at the idea of UNT competing with us  - what on Earth have you seen that indicates they wouldn't? They are 6-1 and smashed Arkansas. We have gone down to the wire with worse teams. They aren't appreciably worse than Baylor and certainly not Tulsa.

[Vic Mackey 46919]

USC is 5-2. With one of their losses to us. They are not a bad team.

 

 

[taybo20 2815]

Just now, Vic Mackey said:

USC is 5-2. With one of their losses to us. They are not a bad team.

 

 

They will most likely win the pac12 south and play in the conference title game 

[Huckleberry 32685]

Look guys, we have played 3 games where the results make us look like a top 10 team to a computer (USC, TCU, Oklahoma). We have played 4 games where the results make us look like a completely average team (Maryland, Tulsa, Kansas State, Baylor). Sure a ranking in the 40s still seems low, but in more than half our games we have looked mediocre.

Edited October 14, 2018 by Huckleberry

[JBJ 3647]

S&P+ is their Possession Efficiency measure times Points Per Successful Play.

We are like #15 in Possession Efficiency which scores each drive based on: scoring/flipping field position/stopping opponents from scoring (of course adjusted to opponent).

It's their IsoPPP (Points per Successful Play) that kills us.  Largely because we have a large volume of successful plays. 

In their system getting stuffed twice then scoring on a75-yard bomb is 10 times better than a sustained 10 to 15-play drive.

It's supposed to be that way.  IsoPPP is a counter-balance to drive efficiency.  They use S&P+ over S&P because the two contributing factors to S&P+ are highly uncorrelated, but it's also not as good a predictor as the old method.

Edited October 14, 2018 by JBJ

[Magus Ossis 2773]

Having just one more point than the opponent is monumentally different than one fewer. The next 10 or 100 don’t change the outcome. People are not robots, and the importance of that one point and the accompanying victory can be lost on computers which don’t care, only calculate. 

[ApocalypseTexas 297]

Anyone remember the scores for the 2009 season or do you just remember we went undeafted to the National Title game.  Just win nothing else matters. We win out we will be right where we should be

Edited October 14, 2018 by ApocalyseTexas

[Huckleberry 32685]

17 hours ago, JBJ said:

S&P+ is their Possession Efficiency measure times Points Per Successful Play.

We are like #15 in Possession Efficiency which scores each drive based on: scoring/flipping field position/stopping opponents from scoring (of course adjusted to opponent).

It's their IsoPPP (Points per Successful Play) that kills us.  Largely because we have a large volume of successful plays. 

In their system getting stuffed twice then scoring on a75-yard bomb is 10 times better than a sustained 10 to 15-play drive.

It's supposed to be that way.  IsoPPP is an offset to drive efficiency.  They use S&P+ over S&P because the two contributing factors to S&P+ are highly uncorrelated, but it's also not as good a predictor as the old method.

Yes, this is a huge factor. I haven't crunched their numbers enough but it's something I might disagree with if and when I do it. Drive efficiency is the point of football, it's essentially efficiency with each possession. For example if you have an offense that gains 4 yards every play, so a mean of 4 and a standard deviation of 0, you will score a touchdown on every single drive where you don't commit a turnover. That is a perfect offense.

But I have a feeling it would score lower in S&P+ than an offense that averages 8 yards per play with a standard deviation of 16. But that "twice as explosive offense" has a probability of gaining a first down on three plays of about 70% compared to the 100% probability of the less explosive offense.

Edited October 15, 2018 by Huckleberry
possession, not position

[HornOnTheBayou 8877]

10 minutes ago, Magus Ossis said:

Having just one more point than the opponent is monumentally different than one fewer. The next 10 or 100 don’t change the outcome. People are not robots, and the importance of that one point and the accompanying victory can be lost on computers which don’t care, only calculate. 

Not really when you're trying to predict how strong a team is. One point could change our perception of how strong the team is, but not how how strong they actually are. If Dicker misses that kick and we go on to lose to OU, it would definitely change all of our perceptions of how strong UT is. However, UT would still be the same team. One made or missed kick isn't going to change the strength of a team.

Edited October 14, 2018 by HornOnTheBayou

[NeverMarryAStripper 21780]

A reminder: S&P+ is intended to be predictive and forward looking.

Good predictive ratings are not résumé ratings, and they don’t give you bonus points for wins and losses. They simply compare expected output to actual output and adjust accordingly. That’s how a given team can win but plummet or lose and move up.

Through seven weeks, the S&P+ rankings are performing well, hitting 54 percent against the spread and 53 percent on the over/under point totals for the year.

 

A predictive poll that shows that we would lose to 1/3 of the teams we beat seems like it sucks

[Hank Chinaski 6310]

1 hour ago, Hozz said:

This is some baseball level nerd shit.  I don't know how it works but when you beat #6 and they stay put while you drop 8 spots, that is clearly retarded.  538 should hire the S&P+ guy, he'd fit right in.

There are 2 big differences between this and baseball stats that immediately jump to mind:

(1) baseball provides very large sample sizes. This S&P+ is going to lack power because it is based on a bunch of teams that have played 6 games. 6. Things like run differential in baseball are pretty good predictors of how good a team actually is, but not after 6 or 12 or 18 games. 

(2) strategy in baseball doesn't change with game situations as dramatically as it does in football. In general, the hitter is always trying to hit the ball hard and the pitcher is always trying to get him out. Of course there are times when making contact or pitching around a hitter are prioritized, but that, again, is where the sample size comes in. In football, sometimes the goal is to run clock, sometimes the goal on defense is just to prevent quick scoring, etc. 

I'm not saying S&P+ doesn't tell you anything, but it seems more like it is more useful as a betting tool than a "ranking" system in terms of ranking how good the teams actually are. 

Edited October 14, 2018 by Hank Chinaski

[texasstrong12 4668]

Oklahoma State is 25th in S&P. They lost by 24 at home to Tech (4-2), lost by 6 at home to ISU (3-3), lost by 19 to K-State (3-4) on the road. Their best win is Boise State (4-2) at home.

This makes absolutely no sense. Same with Auburn being ranked 23rd. It's tough to take S&P seriously when Auburn and Oklahoma State are ranked where they are. 

 

Edited October 14, 2018 by texasstrong12