Jump to content

S&P+ rankings: Overperformance or overcorrection?

satyanash

By satyanash
in Football

 Share

Recommended Posts

  • Replies 1.1k
  • Created
  • Last Reply

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

Dahobbs
Dahobbs

Fozzz is a fucking idiot. 

Magus Ossis
Magus Ossis

A model that drops the winner and sustains the loser of a game needs adjusting.

NeverMarryAStripper
NeverMarryAStripper

Exactly.  Just because you beat someone doesn't mean than you can beat them.

Posted Images

satyanash

satyanash 14276

Posted
  • Author
Posted

Nov. 26: No. 3 Texas 49, Texas A&M 39

112609_TexasTexasAM.png

Long before they were passive aggressively spreading recruiting conspiracy theories about each other, Texas and Texas A&M were actual, on-the-field football rivals. And this was the first time Texas had had to sweat in quite some time.

Dec. 5: No. 3 Texas 13, No. 21 Nebraska 12

120509_TexasNebraska.png

God, December 5 was amazing. We’d already had Cincy-Pitt and Alabama-Florida, and then this beautiful disaster of a game was still waiting for us in the evening. Yes, there was one second left on the clock when the ball hit out of bounds. No, Nebraska didn’t get screwed. Nebraska’s defense got screwed by not having a better offense, though.

Jan. 1: No. 1 Alabama 37, No. 2 Texas 21

010710_BamaTexas.png

A fun, sloppy, underrated season ends with a massive what-if when Colt McCoy goes down. Texas' defense dominated the offense that had just dominated Florida. I’m always going to assume the Horns win this game if McCoy’s healthy. I assume Mack Brown feels the same.

  • Like 1
Link to comment
Share on other sites
Magus Ossis

Magus Ossis 2773

Posted
Posted

For today's installment of "Texas fans are irrationally exuberant," we will compare and contrast Texas' deficiencies with the strengths of a team with a higher initial S&P+ rating for 2019, the Minnesota Golden Gophers.

Texas: embarrassing early September loss to Maryland, 29-34 Minnesota: embarrassing late September loss to Maryland, 13-42

Texas: racked up 4 losses over the season, including only one over its last 5 games, to #5 OU (71% wins on season) Minnesota: 6 total losses, 2 in last 5 games, both to unranked teams (54% wins on season)

Texas: won Sugar Bowl over #5 UGA, holding them to 72 yards rushing (30% of average) Minnesota: beat unranked GT in Quick Lane Bowl, limited GT to 206 rushing (63% of average)

Texas: 5 wins over currently ranked teams Minnesota: winless against ranked opponents

Texas: trending up, improved 5 wins in 2018 over previous regime in 2016 Minnesota: trending? still 2 wins in 2018 worse than previous regime in 2016

Texas: turns over much of defense, leaving only 7 returning defensive players who were rated over .95 composite Minnesota: loses both DTs but returns one defending player who was at least .89

In short, Texas did not do well enough, is not trending favorably enough, and looks exposed on defense compared to the higher-rated Gophers. Hopefully, this breakdown will help you Longhorn homers stop complaining that S&P+ is flawed or somehow has the 'Horns underrated at #35.

 

  • Like 5
  • Haha 2
  • Fuck You 1
Link to comment
Share on other sites
Valmy77

Valmy77 13092

Posted
Posted
On 2/22/2019 at 2:19 PM, Magus Ossis said:

For today's installment of "Texas fans are irrationally exuberant," we will compare and contrast Texas' deficiencies with the strengths of a team with a higher initial S&P+ rating for 2019, the Minnesota Golden Gophers.

Texas: embarrassing early September loss to Maryland, 29-34 Minnesota: embarrassing late September loss to Maryland, 13-42

Texas: racked up 4 losses over the season, including only one over its last 5 games, to #5 OU (71% wins on season) Minnesota: 6 total losses, 2 in last 5 games, both to unranked teams (54% wins on season)

Texas: won Sugar Bowl over #5 UGA, holding them to 72 yards rushing (30% of average) Minnesota: beat unranked GT in Quick Lane Bowl, limited GT to 206 rushing (63% of average)

Texas: 5 wins over currently ranked teams Minnesota: winless against ranked opponents

Texas: trending up, improved 5 wins in 2018 over previous regime in 2016 Minnesota: trending? still 2 wins in 2018 worse than previous regime in 2016

Texas: turns over much of defense, leaving only 7 returning defensive players who were rated over .95 composite Minnesota: loses both DTs but returns one defending player who was at least .89

In short, Texas did not do well enough, is not trending favorably enough, and looks exposed on defense compared to the higher-rated Gophers. Hopefully, this breakdown will help you Longhorn homers stop complaining that S&P+ is flawed or somehow has the 'Horns underrated at #35.

 

Only seven returning players on defense? I see big steps backwards next year. 

Link to comment
Share on other sites
  • 1 month later...
Valmy77

Valmy77 13092

Posted
Posted
2 minutes ago, jebusshuttlecock said:

Connelly dicked around with his model and now has Georgia as the second best team in 2018.  2018 Georgia's percentile ranking (90.0%) is the 8th best since 2005.  2005 Texas (98.9%) is #9

Well they did beat down Austin Peay.

Link to comment
Share on other sites
Walden Ponderer

Walden Ponderer 12639

Posted
Posted
1 hour ago, jebusshuttlecock said:

Connelly dicked around with his model and now has Georgia as the second best team in 2018.  2018 Georgia's percentile ranking (90.0%) is the 8th best since 2005.  2005 Texas (98.9%) is #9

He's clearly just fucking with Texas fans now. Nobody who saw that 2005 Texas team would rank them lower than 3rd, at worst.

Link to comment
Share on other sites
satyanash

satyanash 14276

Posted
  • Author
Posted

Here are his updated S&P+ rankings: https://www.footballoutsiders.com/stats/ncaa

- The 2017 Florida Gators (4-7 with a cancellation) are the 34th best team O_O

- The 2017 FSU Seminoles (7-6) are in the top 10 O__O

Basically, bad teams in good conferences got big bumps because of the new formula's increased weight on conference strength and pre-season polls. Keep in mind that S&P+'s purpose isn't to rank teams, but to pick games vs. the spread.

Link to comment
Share on other sites
satyanash

satyanash 14276

Posted
  • Author
Posted

Bill's new emphasis on pre-season polls/conference strength basically kills every BCS buster in his aggregate rankings. Given a) how many variables he's stuffed into S&P+ at this point and b) his decision to tweak just one of them like this, for the sole reason that it makes the model more predictive for the years he back-tested it on... this is starting to look like a glorified exercise in p-hacking.

At some point Bill should look into utilizing machine learning to identify the best variables to target. Choosing variables manually to fiddle with like this hurts trust in the rankings because you know he'll just do this again for a different variable next year.

Link to comment
Share on other sites
sushihorn

sushihorn 3233

Posted
Posted (edited)
55 minutes ago, satyanash said:

Bill's new emphasis on pre-season polls/conference strength basically kills every BCS buster in his aggregate rankings. Given a) how many variables he's stuffed into S&P+ at this point and b) his decision to tweak just one of them like this, for the sole reason that it makes the model more predictive for the years he back-tested it on... this is starting to look like a glorified exercise in p-hacking.

At some point Bill should look into utilizing machine learning to identify the best variables to target. Choosing variables manually to fiddle with like this hurts trust in the rankings because you know he'll just do this again for a different variable next year.

Automating stupid doesn't help.  It just produces bad results faster and more efficiently.  The radical alteration to his formula in order to favor pre-season polls and weaker teams in "strong" conferences is a tacit admission that his formula was bad and needed to be altered dramatically.  His emphasis on "strong" conferences of course tends to become a self-fulfilling prophecy, especially in the largely closed system the SEC uses to shield their teams from real competition by playing a bunch of non-conference cupcakes.

edit: the new emphasis on pre-season polls is also a tacit admission that the humans are producing better predictions than whatever computer formula he's using.

Edited by sushihorn
Link to comment
Share on other sites
Huckleberry

Huckleberry 32685

Posted
Posted

All of the last few posts are obviously correct. He's gotten to the point of absurdity with his tinkering. My biggest personal issue with it is that there is no logical philosophy that underpins any of his changes. He has undertaken this huge experiment to try to consistently beat the spread and has abandoned any and all sound reason in the pursuit. There is zero reason to keep preseason polls in your formula until the end of the season. That completely corrupts and invalidates your system as, quite clearly, you are saying that players and coaches who no longer have anything to do with the program affect the current season ratings.

He's correct that a college football season doesn't have enough data points for a full analysis, but that doesn't mean you throw in data points that you know don't belong. And let's discuss this conference input to his formula now. What he should have realized is that conferences are just a proxy for something else his formula must be missing (if he is going to refuse to stop this quest). An arbitrary grouping of teams that all play a more strongly interconnected subset of the schedule graph is already overemphasized in a ratings set because of that interplay. My first inclination is that bumping up that contribution is a proxy for the fact that his system still isn't accurately determining garbage time despite his efforts. Conference mates are typically closer in strength to each other than non-conference games, so those games will indeed be a typically more accurate representation of true team strength than non-conference games.

But instead of various other possible tweaks that would make more sense (weight closer games significantly more heavily, including plays occurring outside of garbage time in the play-by-play analysis, instead of just throwing out garbage time; there are several different methods to do this) he just declares that he's going to throw in conference strength because he back-checked it against a few seasons.

  • Like 6
Link to comment
Share on other sites
satyanash

satyanash 14276

Posted
  • Author
Posted (edited)

I read somewhere that on average, the preseason AP poll is actually a better predictor of bowl results than the pre-bowl AP poll. I think this ties into what @Huckleberry was saying, the CFB season is very short and random and if you're trying to bet on/pick games then in-season rankings aren't based on enough data to be used effectively for that purpose.

Let's say team #1 is playing team #2. You can think team #1 is more likely to beat team #2 head-to-head, yet still rank team #2 ahead of team #1 because of their body of work up to that point in the season.

Edited by satyanash
Link to comment
Share on other sites
HtownHorn

HtownHorn 14446

Posted
Posted (edited)
1 hour ago, Huckleberry said:

All of the last few posts are obviously correct. He's gotten to the point of absurdity with his tinkering. My biggest personal issue with it is that there is no logical philosophy that underpins any of his changes. He has undertaken this huge experiment to try to consistently beat the spread and has abandoned any and all sound reason in the pursuit. There is zero reason to keep preseason polls in your formula until the end of the season. That completely corrupts and invalidates your system as, quite clearly, you are saying that players and coaches who no longer have anything to do with the program affect the current season ratings.

He's correct that a college football season doesn't have enough data points for a full analysis, but that doesn't mean you throw in data points that you know don't belong. And let's discuss this conference input to his formula now. What he should have realized is that conferences are just a proxy for something else his formula must be missing (if he is going to refuse to stop this quest). An arbitrary grouping of teams that all play a more strongly interconnected subset of the schedule graph is already overemphasized in a ratings set because of that interplay. My first inclination is that bumping up that contribution is a proxy for the fact that his system still isn't accurately determining garbage time despite his efforts. Conference mates are typically closer in strength to each other than non-conference games, so those games will indeed be a typically more accurate representation of true team strength than non-conference games.

But instead of various other possible tweaks that would make more sense (weight closer games significantly more heavily, including plays occurring outside of garbage time in the play-by-play analysis, instead of just throwing out garbage time; there are several different methods to do this) he just declares that he's going to throw in conference strength because he back-checked it against a few seasons.

The out of conference games should mean more to his "mathematics" because they are truly the only games that differentiate the schedules between conference mates. Thus a conference like the SEC that routinely schedules poorly would correctly have their teams downgraded.  

25-33% of the schedule seems like a good chunk of data.

Edited by HtownHorn
Link to comment
Share on other sites
Tex Long

Tex Long 4326

Posted
Posted

SECSECSEC most certainly does NOT "schedule poorly". The routine scheduling of four patsies instead of three like almost everyone else is clearly because of their dedication to winning. It is brilliant scheduling technique and is routinely rewarded with higher rankings from coaches and writers. Advantage agi.

Link to comment
Share on other sites
Burt Macklin

Burt Macklin 18988

Posted
Posted
40 minutes ago, Tex Long said:

SECSECSEC most certainly does NOT "schedule poorly". The routine scheduling of four patsies instead of three like almost everyone else is clearly because of their dedication to winning. It is brilliant scheduling technique and is routinely rewarded with higher rankings from coaches and writers. Advantage agi.

That's not really true at all. Most SEC teams schedule at least 1 solid OOC opponent, even A&M. The real difference is they have one less conference game, so they get an extra weak opponent regardless, and now with schools like Arkansas and Ole Miss being dogshit, SEC West teams get 5 almost automatic wins each year. 

 

1 hour ago, HtownHorn said:

The out of conference games should mean more to his "mathematics" because they are truly the only games that differentiate the schedules between conference mates. Thus a conference like the SEC that routinely schedules poorly would correctly have their teams downgraded.  

25-33% of the schedule seems like a good chunk of data.

This is true, but OOC games against weak teams are relatively meaningless, and it's extremely difficult to figure out how to weigh performance over some of these terrible teams.  When most P5 teams, not just SEC, play only one solid OOC opponent, it's hard to give a ton of weight to that single performance, and it creates a ton of small sample size issues. 

He needs to just admit that creating a system like this is really fucking hard and his isn't' perfect. Connelly's reverse engineering is so misguided. I agree with @Huckleberry.  IMO S&P+'s biggest flaw has always been over-weighting garbage time and weak opponents. This new change will only increase that. 

This whole weight pre-season AP polls and strong conferences is only going to exacerbate the issues he already had.  

Link to comment
Share on other sites
  • 1 month later...
texifornia

texifornia 56434

Posted
Posted
16 minutes ago, Sejjr said:

Why the fuck does this dipshit add projected wins with an extra TD and with one less TD per game? 

I'm fine with that. It's basically the "well I think our boys are going to be really coached up and ready" number to expect, and the "our rival's kids are in disarray under that cheating bastard of a coach" number to expect.

He doesn't like Texas as a Mizzou grad from Oklahoma, but I think his system is put together in good faith.

Link to comment
Share on other sites
Valmy77

Valmy77 13092

Posted
Posted (edited)

I kind of feel embarrassed for Texas Tech, Kansas State, and Kansas for losing to us.

Charlie Strong will be proud of the 5-7 season we are about to have.

Quote

He doesn't like Texas as a Mizzou grad from Oklahoma, but I think his system is put together in good faith.

It is not his bias I question but rather his competence.

Edited by Valmy77
Link to comment
Share on other sites
PencilPusher

PencilPusher 454

Posted
Posted

This is probably the time to remind folks that Connelly's system, at least at the outset of a season, factors in past performance significantly Which means we have both Herman's first season plus some Strong suckitude factoring in.  Case in point - West Virginia. Their preseason rank of 38 is just behind us and ahead of Iowa State. Yet, they lost their NFL drafted QB, several receivers, and a ton of production, not to mention their head coach. While it is conceivable they could be favored (this is allegedly a betting system, after all) and win by 2 or more points in Morgantown, the system implies this would be a "pick em" at a neutral site and based on what we know about what both teams return, that seems a  big stretch. 

Other eye-rollers:

LSU -14......in Austin. LSU will be a tough game and yes they can win in Austin because they have a lot of talent. But.......I'll take LSU fans giving me 14 on the road in Austin any day of the week. Only the really drunk Cajuns (which may or may not include their coach) would offer that bet up today. Again, past history is inflating the number.

OU - 16. Don't need to explain this, but the last time OU won by greater than this margin was at the tail end of FUPM Mack Brown era in 2012. Since 2014, the regular season OU wins have been by 5, 5, and 5 and in the Big 12 Championship, the margin was only 12. Any OU fans want to give 16 points on this game right now? 

Even the Baylor number looks inflated....we have less of a chance to win in Waco than in Ames? Iowa State will be a better team than Baylor this year and Ames is a tougher environment than Waco, where 40%+ of the fans will be clad in Burnt Orange at the Toilet Seat. 

Plus you have a coach that historically has outperformed the model. Even Connelly acknowledges that with his joke that we will beat OU and LSU, but lose to LaTech and KU. 

In short, if anyone gives you the O/U on TX wins at 6.7, bet the over. You actually might find some Ags that will do that......to them I say

Related image

 

 

  • Like 1
Link to comment
Share on other sites
Tex Long

Tex Long 4326

Posted
Posted

FBC.

He's selling a complicated pile of shit, and his marketing strategy is based on generating buzz via controversy. Hence, he says stupid shit about his stupid shit.

If his "method" really worked, he wouldn't tell a fucking soul about it, he'd just hang out by the pool with his mePad, running his numbers and cashing in his betting slips.

  • Like 1
Link to comment
Share on other sites
  • 2 weeks later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
 Share
Go to topic listing


Football ... Basketball ... Baseball ... Other Sports ... Futbol ... 🤫995🤫 ... Gambling ... Movies & TV ... Music ... Hobbies ... Lulz ... Food & Travel ... Daily Texan ... Business and Markets ... Cloak Room ... Help ... For Sale ... Board Discussion ... Subscribe!... Donate!... Advertise... COOKIE MONSTER!

×
×
  • Create New...