S&P+ rankings: Overperformance or overcorrection?

[rickyspub 801]

I didn't realize just how simplistic his recruiting model was. Some teams are great at mental evals and some are great at development and others suck. Sometimes you bring in 30 players and others years you bring 20. Then you have the myriad of new hires that change those dynamics and generally cause roster churn. Not to mention coaches coming up from G5 who may be in over their head. A lot of moving pieces to just throw in some percentages and stale recruiting rankings.

Still feel his system overreacts to blow out wins over weak opponents. Woodshedding 3+ sub-P5 teams gives you a big S&P boost against a team that only plays 1 sub-P5 team.

[Burt Macklin 18,988]

3 hours ago, Valmy77 said:

Yeah I just do not understand his model, I think it highly unlikely he set out to make one that made Texas look bad. We do that on our own in the 4th quarter.

I don't get how returning Cosmi, CJ, Duvernay, Keontay Ingram, Sam Ehlinger, Caden Sterns, BJ Foster, Brandon Jones, Malcolm Roach, Jeffrey McCulloch and so forth is the 8th ranked returning group in the Big 12 and the 51st ranked returning group in the nation. If your statistics are producing that result then I question your model. 

 

1 hour ago, Valmy77 said:

It seems impossible to get a straight answer to my question.

Yes I get that we are having lots of guys leave for the NFL and graduate. I presume most teams are not starting an entire roster of Freshman and Sophomores though. Surely this does not make us among the worst in the Big 12 as far as returning players. I mean the entire reason we are #4 in the Big 12 by his numbers seems to be inferior returning players to OU, OSU, Baylor, Tech, TCU, West Virginia, and Iowa State. So the whole conference outside of Texas and the Kansas schools are bringing everybody back and therefore Texas should lose five games? Because I don't see how his other two columns would lead him to this conclusion.

I think you’re a little confused here. The category you’re referencing is returning production. It’s basically the percentage of production you return multiplied (based on a weighted formula) by your teams S&P+ from last year. He’s not ranking based on individual players coming back; he’s using a broad set of data that indicates teams get worse when they lose a lot of production. We lost 8 of 11 starters on defense, so we lose a ton of our production from last year. Similar on O, though not quite as drastic. We lose our top receiver and rusher, plus 3/5 of our OL. In general UT will be misleading in this stat because 1) we were under ranked by last year’s system and 2) no school in the country has a greater talent disparity than what was left of the 2015, 2016, and 2017 classes compared to what came in 2018 and 2019. For example,  we likely won’t be hurt by losing guys like wheeler and Locke, even though they made up a large portion of our production (I.e. tackles) from last year. 

One of the biggest the biggest issues with his system is that it weighs the last 2-5 years of the program as one of its 3 metrics. Well, no team in the country will be more misleading based on that metric than Herman with his guys in there compared to the dumpster fire that was Charlie strong for 3 years plus Herman going 7-6 with the mess Charlie left. 

This would also indicate that last year, UT entered the year ranked too low because the Strong years were weighing it down. 

Overall, UT Is an extreme outlier in 2/3 of these categories and there’s good reason to believe we’re underrated based on this metric. It would be nice if Connelly admitted these factors as a reason why UT could be an outlier instead of doubling down on his system being perfect, but I’m sure he gets a lot of flack and it gets old for him.

 

Edited February 11, 2019 by Burt Macklin

[GabrielsHorn 2,142]

6 hours ago, satyanash said:

 

He's such a cunt about this system and not just recognizing that Texas is an outlier. Also, Tennessee being ranked above us is a fucking joke.

[MrPhlegm 3,413]

I think you’re a little confused here. The category you’re referencing is returning production. It’s basically the percentage of production you return multiplied (based on a weighted formula) by your teams S&P+ from last year. He’s not ranking based on individual players coming back; he’s using a broad set of data that indicates teams get worse when they lose a lot of production. We lost 8 of 11 starters on defense, so we lose a ton of our production from last year. Similar on O, though not quite as drastic. We lose our top receiver and rusher, plus 3/5 of our OL. In general UT will be misleading in this stat because 1) we were under ranked by last year’s system and 2) no school in the country has a greater talent disparity than what was left of the 2015, 2016, and 2017 classes compared to what came in 2018 and 2019. For example,  we likely won’t be hurt by losing guys like wheeler and Locke, even though they made up a large portion of our production (I.e. tackles) from last year. 
One of the biggest the biggest issues with his system is that it weighs the last 2-5 years of the program as one of its 3 metrics. Well, no team in the country will be more misleading based on that metric than Herman with his guys in there compared to the dumpster fire that was Charlie strong for 3 years plus Herman going 7-6 with the mess Charlie left. 
This would also indicate that last year, UT entered the year ranked too low because the Strong years were weighing it down. 
Overall, UT Is an extreme outlier in 2/3 of these categories and there’s good reason to believe we’re underrated based on this metric. It would be nice if Connelly admitted these factors as a reason why UT could be an outlier instead of doubling down on his system being perfect, but I’m sure he gets a lot of flack and it gets old for him.
 

Well when you design a shitty allegoryithim and feed in garbage data like this clown did you show that you suck rancid green donkey ass all day every day and night and are addicted to taking it in the ass from multiple syphilitic baboons every Tuesday and Thursday.

[Ricky Butler 2,657]

4 hours ago, Huckleberry said:

So it's consistently coaches whose offenses rank better at drive efficiency (yards per possession) and turnovers than they do at explosiveness (yards per play). That makes perfect sense.

Can you elaborate on this? 

[texifornia 56,434]

It's pretty funny that, as currently set up, his system overrates Mizzou and underrates Texas.

Edited February 12, 2019 by texifornia

[texasstrong12 4,668]

5 minutes ago, texifornia said:

 

Herman #1 in diff wins/year. 

[satyanash 14,276]

23 minutes ago, texifornia said:

It's pretty funny that, as currently set up, his system overrates Mizzou and underrates Texas.

Poor Bill, suffering  persecution from a few Texas fans on Twitter. What a martyr.

[Huckleberry 32,685]

Seems like he's identified one of the common threads but hasn't figured out how to account for it, that being possession-based offenses like the triple option. There are of course other versions of possession-based offenses like the one Texas rolled out in 2018. Sometimes they're possession-based out of necessity rather than desire or design.

Something about his system is still underrating those teams, enough that he felt obliged to publish this article.

[Fozzz 152]

55 minutes ago, texasstrong12 said:

Herman #1 in diff wins/year. 

Some elite company on that listlike Rich Brooks, Brian Polian, Korl Dorrell...

I think the emphasis on YPP is misplaced - S&P+ is based on success rate, not YPP, and thus is not influenced by explosiveness like YPP.  It seems like the outliers are often a product of maxing out their 3rd/4th down conversion rates.  Maybe some styles of offense are more reliable at converting 3rd/4th down.  For instance, Army did both an excellent job of producing a lot of manageable 3rd/4th conversion opportunities (#1 in average 3rd down distance) and were practically impossible to stuff (#1 in stuff rate), so they converted those opportunities like no other team.  However, while Texas' offense was good at producing manageable conversion opportunities (28th in average third down distance), it had a mediocre stuff rate, so I don't know if this applies to them.

[Huckleberry 32,685]

Without looking at the specific numbers in detail, being average at stuff rate but having a lot shorter than average conversion distance should lead to an above average conversion rate.

[satyanash 14,276]

9 minutes ago, Fozzz said:

Some elite company on that listlike Rich Brooks, Brian Polian, Korl Dorrell...

I think the emphasis on YPP is misplaced - S&P+ is based on success rate, not YPP, and thus is not influenced by explosiveness like YPP.  It seems like the outliers are often a product of maxing out their 3rd/4th down conversion rates.  Maybe some styles of offense are more reliable at converting 3rd/4th down.  For instance, Army did both an excellent job of producing a lot of manageable 3rd/4th conversion opportunities (#1 in average 3rd down distance) and were practically impossible to stuff (#1 in stuff rate), so they converted those opportunities like no other team.  However, while Texas' offense was good at producing manageable conversion opportunities (28th in average third down distance), it had a mediocre stuff rate, so I don't know if this applies to them.

We were also fantastic on 4th down. 11/14, 2nd-best in the FBS.

[WBT 20,397]

Quote

With Bill Snyder retired, which active coaches now do the best job of defying math?

If he didn't say stupid shit like that, I'd be more inclined to cut him some slack.

[Huckleberry 32,685]

Ha, yeah. He has decided to take on a persona in his writing where he assumes his system is perfect and so any team whose results run counter to his system's expectations must be using some sort of black magic. In reality he understands that his model is evolving and that such situations are actually evidence he needs to continue tinkering, but in his writing he comes across as though he doesn't realize that.

It's understandable from the context of his content creation expectations and driving readership and engagement, but it certainly isn't a plus for engendering faith in his work.

[Fozzz 152]

Has he ever said that his system is perfect?  The fact that he updates it yearly seems to belie that assertion.  The butthurt itt over every little thing he says is pretty funny.

[Huckleberry 32,685]

You should try reading better. I specifically said he understands his system isn't perfect and needs to evolve. What's even funnier than people's butthurt over his system is your butthurt whenever anyone criticizes him in any way. He's married with kids, man, he's not interested.

[texasstrong12 4,668]

12 minutes ago, Huckleberry said:

What's even funnier than people's butthurt over his system is your butthurt whenever anyone criticizes him in any way. 

Funny enough that's the biggest issue with Connelly.

Most people realize there's still value in his system. People give him shit because when someone questions or criticizes his system he makes it personal and gets defensive. 

Edited February 12, 2019 by texasstrong12

[Fozzz 152]

6 minutes ago, Huckleberry said:

You should try reading better. I specifically said he understands his system isn't perfect and needs to evolve. What's even funnier than people's butthurt over his system is your butthurt whenever anyone criticizes him in any way. He's married with kids, man, he's not interested.

Please point out where he has "on a persona in his writing where he assumes his system is perfect."  The fact that he's open about the changes he makes indeed does contradict that assertion.  If he did take on such a persona then he either would not change it or at the least he wouldn't broadcast the fact that he does make changes (including dramatic changes like the reduction in importance of explosiveness) with his system.  Moreover, he doesn't mechanically ascribe all variation between actual and 2nd order wins as being entirely due to luck.  Indeed, he even states in the very article we have been discussing (you don't appear to have actually read it) that Army's overperformance has led him to experiment with 3rd/4th down conversion rates to see if he could make S&P+ more predictive. 

[rickyspub 801]

10 minutes ago, texasstrong12 said:

Funny enough that's the biggest issue with Connelly.

Most people realize there's still value in his system. People give him shit because when someone questions or criticizes his system he makes it personal and gets defensive. 

Of course there is value in his system, but let's not overvalue it either. There is value in Sagarin's system as well, but Jeff doesn't go on Twitter and act like a little bitch or write articles throwing shade at certain teams when his system shows a shortcoming. 

[texasstrong12 4,668]

Just now, rickyspub said:

There is value in Sagarin's system as well, but Jeff doesn't go on Twitter and act like a little bitch or write articles throwing shade at certain teams when his system shows a shortcoming. 

Yeah, that's my point. It's why he gets shit. He acts like a baby on twitter when people question his system. 

[Pato del Muerto 52,322]

21 minutes ago, Fozzz said:

Please point out where he has "on a persona in his writing where he assumes his system is perfect."  The fact that he's open about the changes he makes indeed does contradict that assertion.  If he did take on such a persona then he either would not change it or at the least he wouldn't broadcast the fact that he does make changes (including dramatic changes like the reduction in importance of explosiveness) with his system.  Moreover, he doesn't mechanically ascribe all variation between actual and 2nd order wins as being entirely due to luck.  Indeed, he even states in the very article we have been discussing (you don't appear to have actually read it) that Army's overperformance has led him to experiment with 3rd/4th down conversion rates to see if he could make S&P+ more predictive. 

 

1 hour ago, WBT said:

If he didn't say stupid shit like that, I'd be more inclined to cut him some slack.

Already done. 

[sushihorn 3,233]

23 minutes ago, Fozzz said:

Please point out where he has "on a persona in his writing where he assumes his system is perfect."

It's right there in the second headline if you bother to look.

Quote

With Bill Snyder retired, which active coaches now do the best job of defying math?

Connelly conflates "math" with his model.  That is empirically wrong, incredibly arrogant and terribly grating.  The outliers show weaknesses in Connelly's model, not problems with mathematics or some arcane powers on the part of certain coaches.  That is exactly what Huck is saying about the way the guy writes.  Even if he later admits the need for adjustments, leading with a statement like this just leaves a bad taste.  And that attitude is a perfect match for his defensive behavior whenever someone questions his model.

[Machinator 32,097]

 

[Burt Macklin 18,988]

Other thing I’ll say about why Texas is underrated by S&P+ is that this coaching staff on offense and defense both seem to just try to run out the clock when we get a lead. Herman quits running Sam as much and Orlando quits bringing pressure. I think it’s a flaw in the coaching, but there wasn’t a single game we lost last year because of that flaw. This flaw is easily correctable and isn’t representative of our team’s ability if the game were close. 

S&P+ assumes each team is trying to maximie it’s quality of play and doesn’t account for a team choosing to lower its quality of play in order to run out the clock. Like I said, I’d prefer Herman and Orlando quit doing this, but It’s pretty easily changeable and S&P+ downgrades Texas a ton for this. We had a lot of close wins that S&P+ downgraded us for.  

Herman’s teams definitely play up and down to their competition, but I also think that by 2020, our roster will be so talented that we’ll be a lot less likely to lose any of those games, even if the wins are closer than they should be.

[Fozzz 152]

lol the posters itt are so not butthurt that they are negging posts and claiming that S&P+ is rigged to make look Texas look bad.  

Edited February 12, 2019 by Fozzz

[Burt Macklin 18,988]

Just now, Fozzz said:

lol the posters itt are so not butthurt that they are negging posts and claiming that S&P+ is rigged to look Texas look bad.  

You get negged because you’re an obnoxious asshole. Has nothing to do with S&P+. 

[Fozzz 152]

Shut up, Burt.

[rickyspub 801]

39 minutes ago, Fozzz said:

Shut up, Burt.

I love the smell of fresh butthurt in the afternoon...smells like victory.

[Magus Ossis 2,773]

We criticize Connelly and his S&P+ because he makes statements implying it is more than it is. He acts defensive when it is publicly noted to have an area of weakness rather than acknowledge failings. Huck doesn’t spout off making excuses for his adjusted stats when they fail to model everything about future contests. It isn’t complicated— the stronger claims one makes, the more strongly he should expect them reviewed.

[Fozzz 152]

43 minutes ago, Magus Ossis said:

We criticize Connelly and his S&P+ because he makes statements implying it is more than it is. 

In his article outlining S&P+ this is what he actually claims:

Quote

In terms of performance, S&P+ tends to hit between 51 and 54 percent against the Vegas spread, a solid range.
It thrives in the realm of win probability — teams given, say, 66 percent chances of winning will win 66 percent of those games over time.

The S&P+ ratings are an overall prediction tool, but honestly, the predictions are just intended to provide affirmation. If you know that the ratings can be used to at least slightly beat Vegas from year to year, then you can trust them in your analysis as well.

I'm sure I'll get negged for this and/or homophobic insults will be hurled at me, but the above seems fairly defensible.

[Burt Macklin 18,988]

8 minutes ago, Fozzz said:

In his article outlining S&P+ this is what he actually claims:

I'm sure I'll get negged for this and/or homophobic insults will be hurled at me, but the above seems fairly defensible.

Lulz. Don’t start playing the victim. What he said above but is fine. His UT-specific comments/defensiveness are in his tweets, which you keep conveniently ignoring. 

Edited February 12, 2019 by Burt Macklin

[Fozzz 152]

I didn't think he was dismissive at all at the possibility of something real and not imaginary potentially explaining the difference between Herman's actual and 2nd order wins, he just wants a larger sample before coming to any conclusions.  The statement below is hard to square with the caricature presented in this thread that ostensibly claims his system is perfect and anything it can't explain is the product of chance alone:

Quote

Texas’ fans collective response to being projected in the mid-30s in S&P+ was, shall we say, high in volume. And Herman’s presence atop the overachievers list here seems to verify that S&P+ is not well-equipped to handle a Herman team.

Maybe. But I need to see more because really, this four-year overachievement is a two-year overachievement.

Rather it appears to be Texas fans who for some mysterious and wholly inexplicable reason are ready to come to the conclusion that you can bank on Herman markedly outperforming his 2nd order wins on a yearly basis.

[Valmy77 13,092]

7 hours ago, texifornia said:

It's pretty funny that, as currently set up, his system overrates Mizzou and underrates Texas.

I suspect that is because Missouri has a kind of offense that is more suited to running up big scores than ours. 

[Magus Ossis 2,773]

8 minutes ago, Fozzz said:

I didn't think he was dismissive at all at the possibility of something real and not imaginary potentially explaining the difference between Herman's actual and 2nd order wins, he just wants a larger sample before coming to any conclusions.  The statement below is hard to square with the caricature presented in this thread that ostensibly claims his system is perfect and anything it can't explain is the product of chance alone:

Rather it appears to be Texas fans who for some mysterious and wholly inexplicable reason are ready to come to the conclusion that you can bank on Herman markedly outperforming his 2nd order wins on a yearly basis.

How much would you bet on the under for Texas wins next year if we set the O/U at 7.5? Because S&P+ says we are most likely to win only 7. Or do you think we “defy math” when we win? Or that we objectively look like we shouldn’t even be in the 6 or 7 teams outside the polls “also receiving votes” to start next year? Or that our wins, as opposed to the ones he expected, were more characterized by luck last year? Connelly, as your second quotation from him in this post of yours shows, still doesn’t truly acknowledge that his model is missing something. With one breath he will say one modest thing about his model and with the next he will ascribe all its failings to “luck.”

[Fozzz 152]

I don't think it's inconceivable for them to win 7 games but I would be a little disappointed if that were the case.  I think one thing that has really hurt Texas over the past five years has been a lack of continuity on the coaching staff, the offense was totally ruderless under Strong and the defense clearly had difficulty in adapting to the changes required for executing Gilbert's offense.  Next season Texas' players will be playing in both the same offensive and defensive systems for the third season in a row.  That is something that is not easy for a system like S&P+ to capture but will likely pay dividends.  

[texifornia 56,434]

Just now, Fozzz said:

I don't think it's inconceivable for them to win 7 games but I would be a little disappointed if that were the case.  I think one thing that has really hurt Texas over the past five years has been a lack of continuity on the coaching staff, the offense was totally ruderless under Strong and the defense clearly had difficulty in adapting to the changes required for executing Gilbert's offense.  Next season Texas' players will be playing in both the same offensive and defensive systems for the third season in a row.  That is something that is not easy for a system like S&P+ to capture but will likely pay dividends.  

If you would be only "a little disappointed" with 7 wins, you'd be the only Texas fan who felt that way.

[Fozzz 152]

Just now, texifornia said:

If you would be only "a little disappointed" with 7 wins, you'd be the only Texas fan who felt that way.

Texas fans are revered for their rationality.

[Valmy77 13,092]

Just now, Fozzz said:

Texas fans are revered for their rationality.

So you think 7-6 is a rational expectation for this team next year? And everybody expecting anything more is irrational? Just trying to figure out your dodge and weave routine here.

[MrPhlegm 3,413]

Lulz at all this arguing over a twit with a kludge of a "system" based on bad assumptions using inferior/incorrect data and faulty metrics.

Didn't this twit work on the Mars Climate Orbiter programming a few years back?

[MrPhlegm 3,413]

That is something that is not easy for a system like S&P+ to capture but will likely pay dividends.  

That is because the S&P+ "system" is ass my dude.

 

Looch's top 44 horseshit is more legit and valid than s&p+.

 

[Magus Ossis 2,773]

20 minutes ago, Magus Ossis said:

How much would you bet on the under for Texas wins next year if we set the O/U at 7.5? Because S&P+ says we are most likely to win only 7. Or do you think we “defy math” when we win? Or that we objectively look like we shouldn’t even be in the 6 or 7 teams outside the polls “also receiving votes” to start next year? Or that our wins, as opposed to the ones he expected, were more characterized by luck last year? Connelly, as your second quotation from him in this post of yours shows, still doesn’t truly acknowledge that his model is missing something. With one breath he will say one modest thing about his model and with the next he will ascribe all its failings to “luck.”

 

3 minutes ago, Fozzz said:

I don't think it's inconceivable for them to win 7 games but I would be a little disappointed if that were the case.  I think one thing that has really hurt Texas over the past five years has been a lack of continuity on the coaching staff, the offense was totally ruderless under Strong and the defense clearly had difficulty in adapting to the changes required for executing Gilbert's offense.  Next season Texas' players will be playing in both the same offensive and defensive systems for the third season in a row.  That is something that is not easy for a system like S&P+ to capture but will likely pay dividends.  

 

1 minute ago, Fozzz said:

Texas fans are revered for their rationality.

It sounds to me, and correct me please where I have missed or misinterpreted your reply, as though you grudgingly admit you think S&P+ is too low in its 2019 win projection and offer no rebuttal to anything else I argue in the other rhetorical questions. And yet you sarcastically put down Texas fans’ rationality. I suppose we are irrational for agreeing with Vegas and virtually every major college football expert that Texas is likely to be at or near the top of the Big12 next year.

[Fozzz 152]

2 minutes ago, Magus Ossis said:

 

 

It sounds to me, and correct me please where I have missed or misinterpreted your reply, as though you grudgingly admit you think S&P+ is too low in its 2019 win projection and offer no rebuttal to anything else I argue in the other rhetorical questions. And yet you sarcastically put down Texas fans’ rationality. I suppose we are irrational for agreeing with Vegas and virtually every major college football expert that Texas is likely to be at or near the top of the Big12 next year.

So there is not any confusion, here is what I said:

Quote

Sushi was referring to a comment that Connelly had previously made about Herman having overpeformed his S&P "2nd order wins" record (not the pre-season prediction) in 2015, but Connelly never said his model (S&P) was terrible at assessing Herman's teams.  In 2016 Herman's 2nd order wins was 8.1 and in 2017 it was 7.5, indicating that Herman does not have some hidden ability to consistently outperform his S&P 2nd order wins.  

His pre-season projections have tended to overrate Texas, but given where we finished in S&P last year and what we lost from last year's team, I don't see any problem with his projection for next season.  

Thus, considering the inputs that go into pre-season S&P+ projections, I don't have any problem with his S&P+ projection for 2019.  That is not saying that pre-season S&P+ projections capture all available information that impinges upon Texas' likely win total for 2019.  No projection system could ever do such a thing.

[sushihorn 3,233]

33 minutes ago, Valmy77 said:

So you think 7-6 is a rational expectation for this team next year? And everybody expecting anything more is irrational? Just trying to figure out your dodge and weave routine here.

Why not?  It looks like the same passive-aggressive routine we get from Connally.  I mean in one breath he talks about the limitations of his model.  In the next he whines and plays the victim card because other people question it. 

[Katfid54 8,591]

All of you assholes quoting Foz all the time sure are making it difficult for those of us who blocked him long ago.

[Magus Ossis 2,773]

17 minutes ago, Katfid54 said:

All of you assholes quoting Foz all the time sure are making it difficult for those of us who blocked him long ago.

Sorry. I’ll just block him.

[Poolflood 845]

@Huckleberry could you or @satyanashgive me an explanation on how SOS is calculated?  

I'm fascinated on this as it's mentioned in rankings but not fully explained.  

SEC teams have high SOS but play about 20-25% of their games against garbage.  

[Huckleberry 32,685]

There's no one way to do it so that question can't really be answered. It depends on the set you're looking at. 

[Valmy77 13,092]

4 minutes ago, Poolflood said:

@Huckleberry could you or @satyanashgive me an explanation on how SOS is calculated?  

I'm fascinated on this as it's mentioned in rankings but not fully explained.  

SEC teams have high SOS but play about 20-25% of their games against garbage.  

My theory is that there is a bit of a feedback loop there. Their non-garbage opponents have higher rankings because they have good records and they have good records because they play garbage opponents. So the garbage opponents benefit everybody.

[Poolflood 845]

1 hour ago, Huckleberry said:

There's no one way to do it so that question can't really be answered. It depends on the set you're looking at. 

So if I'm looking at a website like adjustedstats.com. How do you think this jabroni calculates SOS?

[Huckleberry 32,685]

Different ways in different sets. The schedule rating on the regular ratings page (not linked in the menu right now) is just the average power rating of their opponents. The schedule rating on the EWP ratings page is the PWP rating the team would need to have in order to expect a .500 record against the schedule they faced.