Jump to content

S&P+ rankings: Overperformance or overcorrection?


satyanash

Recommended Posts

23 hours ago, Huckleberry said:

SoYes, this is a huge factor. I haven't crunched their numbers enough but it's something I might disagree with if and when I do it. Drive efficiency is the point of football, it's essentially efficiency with each possession. For example if you have an offense that gains 4 yards every play, so a mean of 4 and a standard deviation of 0, you will score a touchdown on every single drive where you don't commit a turnover. That is a perfect offense.

But I have a feeling it would score lower in S&P+ than an offense that averages 8 yards per play with a standard deviation of 16. But that "twice as explosive offense" has a probability of gaining a first down on three plays of about 70% compared to the 100% probability of the less explosive offense.

So it's one of those systems that rewards the big play spread offenses that Oregon or WSU runs at the expense of a power running team?

Link to comment
Share on other sites

47 minutes ago, bejezuz said:

You would be correct. 

https://www.footballstudyhall.com/2017/6/1/15723948/college-football-coaching-underachievers-overachievers

Check out who landed number three on the overachiever list.  

Shocking. Also of note, Urban and Mack, the two coaches that molded Herman the most are also in the Top 15.

Link to comment
Share on other sites

7 minutes ago, notre dame joe said:

So it's one of those systems that rewards the big play spread offenses that Oregon or WSU runs at the expense of a power running team?

Not exactly.  It rewards terrible offenses that break a long one every now and again - over grinding offenses. 

That description matches Texas Tech and Georgia Tech equally.

And it works against Texas or Army equally.

  • Like 1
Link to comment
Share on other sites

2 minutes ago, Huckleberry said:

JBJ - Earlier you posted the scores for various outputs, do you have a link for the formula they use for those possession versus play efficiency calculations?

No, that was an example based on yards, which should be close to what they do between the 30s.  Their calculations are based on their Expected Points values for a given down, distance, and yardline; and they guard that closely.  A successful play increases the EP value, and an unsuccessful one decreases the EP value. As far as I know, actual yards or scoring don't factor in except through that EP value system.

However, whenever they change something, Bill Connely or Aaron Schatz will post something on their blogs, SBN, or the website itself giving enough to reproduce it in nature even if not exactly.

I also just don't like the S&P+ in general, it goes against what most modellers do, and I feel that it is designed as controversial and clickbait.  The original S&P (discontinued) or the FEI give results that more closely match actual expections and what every other model looks like.

Link to comment
Share on other sites

OSU's continued relatively high rankings in S&P+ is confounding to me.  Although I do know that we are SHITTY in terms of penalties. We're averaging damn near 100 yards in penalties per game, and we tend to specialize in penalties of the back-breaking variety.   We convert a 3rd and 15?  HOLDING. CHOP BLOCK. OFFENSIVE PASS INTERFERENCE.

We get the opponent in 3rd and long and hold? DEFENSIVE PASS INTERFERENCE. OFF SIDES. PERSONAL FOUL. And on and on and on.

So maybe our shitty knack for killing drives with penalties is overriding our generally otherwise efficient play?  No idea.  But I do know that we suck and we shouldn't be favored to beat you guys here in a few weeks.

Edited by oSuJeff97
Link to comment
Share on other sites

Thankfully sports are played by humans on the fields with real life emotions and variables. And not in a fucking nerdy Excel spreadsheet. Any model that says Appalachian St is 9pt favorites and we would only win 36% of the time is so fucking stupid. I can't even that that serious.

I get advanced metrics but some people take the shit too far and you get junk like those models. This S&P rankings are idiotic.

  • Like 2
Link to comment
Share on other sites

1 hour ago, oSuJeff97 said:

OSU's continued relatively high rankings in S&P+ is confounding to me.  Although I do know that we are SHITTY in terms of penalties. We're averaging damn near 100 yards in penalties per game, and we tend to specialize in penalties of the back-breaking variety.   We convert a 3rd and 15?  HOLDING. CHOP BLOCK. OFFENSIVE PASS INTERFERENCE.

We get the opponent in 3rd and long and hold? DEFENSIVE PASS INTERFERENCE. OFF SIDES. PERSONAL FOUL. And on and on and on.

So maybe our shitty knack for killing drives with penalties is overriding our generally otherwise efficient play?  No idea.  But I do know that we suck and we shouldn't be favored to beat you guys here in a few weeks.

Well we’re in for a fun game in two weeks. Texas also specializes in dick tripping penalties. 

Edited by Dark Horse
  • Like 1
Link to comment
Share on other sites

3 hours ago, Huckleberry said:

Cool. I'm considering a lot of different things with AdjustedStats right now and expected points values would possibly be part of that at some point. Getting the play-by-play data appears like it might be costly, but it shouldn't be hard to build from there.

Talk nerdy to me

Link to comment
Share on other sites

Any model that says Appalachian St is 9pt favorites and we would only win 36% of the time is so fucking stupid. I can't even that that serious.


Obviously we can’t test that prediction as Texas and App State aren’t even going to play once, let alone play the numerous times necessary to determine the correctness of that singular prediction, but S&P+ generally performs fairly well. Collectively over time, teams that are projected to win 36% of the time do win roughly 36% of the time. Of course, when examining a single result, that means it would appear the prediction was “wrong” roughly 36% of the time.
Link to comment
Share on other sites

17 hours ago, JBJ said:

I also just don't like the S&P+ in general, it goes against what most modellers do, and I feel that it is designed as controversial and clickbait.

Given that I’m just now learning about this metric, you seem dead on accurate. Couple that with those tweets pointing out stats to rile fan bases, it seems like a desperate attempt to generate attention and dollars 

Link to comment
Share on other sites

2 minutes ago, Valmy77 said:

Is the scoring margin +? Then it works for me.

Yeah, that really is the point. Consistency gets slammed in those models, but the truth is, a team that wins by 3 all 14 or 15 times in a year is better than one that averages winning by 25+, but loses two or three games that year. Just keep winning.

Link to comment
Share on other sites

Well, we are still expected to lose the next 3 under the S&P model though the projected margin of victory has dropped in all of our games and the win probabilities have improved. Guess all of the haters will get their day in the sun if this plays out. While I think the S&P model has its flaws, I am curious as to why we are about to embark on a 3 game losing bender. So lets look at performance through the lens of efficiency and explosiveness vs our remaining opponents. 

Our offensive efficiency rates (ability to stay on schedule) rank close to the top quartile. (Success rate-43; Marginal Efficiency-34). Explosiveness however is close to the bottom quartile (Iso Points per Play-91, Marginal Explosiveness -97). No surprise to anyone watching our games. We can consistently move the sticks, but do not generate that many "explosives" in the game. Wonder why Herman is so concerned with offensive speed?

However, to gauge success against the remaining spate of games, we need to evaluate what other defenses prevent. Our achilles heel would be a team that defensively looks like Texas in that it is hard to sustain drives against, but gives up the big play. Is there a team like that remaining on our schedule? Lets see

Ok State - Defense success rate - 46; Marginal Efficiency -57; ISO PPP-83; Marginal Explosiveness-80. Possible problem spot. They are about middle of the road on efficiency and below average on explosives given up. Ok State's big problem is that they are in the bottom quartile of turnover margin, which doesn't help their defense's cause (not to mention penalties-118 in the country on penalty yards ). We will have to play better than against Baylor for sure.

West Virginia -  Defense success rate - 68; Marginal efficiency - 77; ISO PPP- 52; Marginal Explosiveness - 32. We are not a good matchup for WVa because their defense has shown the ability to allow teams to sustain long drives which suits our gameplan. Couple that with lots of penalty yards (104th) and turnovers, we should be set up for success in this game

Tech - Defense success rate - 71; Marginal efficiency - 100; ISO PPP - 114; ISO Explosiveness - 102. A defense that is exactly what we would want to see to have success against. These numbers are actually worse than Baylor's were coming into the game last week. They are also bad at penalties (110th), though favorable at turnovers so far this year. 

Iowa State - Defense success rate - 35; Marginal efficiency - 65; ISO PPP - 55; Marginal Explosivieness - 36. By metrics, this would be the toughest remaining defense on the schedule which shouldn't surprise anyone who has been paying attention this year. They also rank close to the top quartile in fewest penalties. Best bet for a 21-17 type game down the stretch of the season. 

I won't bother with Kansas at this point since we are still expected to win that one by double digits. 

So, there isn't a defense that should completely scare us down the stretch, but it looks like OSU and Iowa State may be the defenses we have the hardest time with. My $ is on Iowa State defending us the best down the stretch. 

Will look at the inverse (our defense against remaining offenses) later. 

  • Like 2
Link to comment
Share on other sites

Quote

Texas

  • AP rank: seventh
  • S&P+ rank: 43rd

Only Tom Herman teams seem capable of pulling off this balancing act. The only way to possibly understand how this team can be 6-1 and unbeaten in its (power) conference, with a win over an S&P+ top-five team, is to pretend that Texas has been two different teams this year.

One Texas is 3-0 against the three top-50 S&P+ teams it has faced (No. 3 Oklahoma, No. 30 USC, No. 46 TCU), winning by an average of 38-25. The Horns pulled away against USC and rode turnovers and late-game defense to a comfortable win against TCU. Then they surged to a commanding lead against Oklahoma (with help from the turnovers fairy), gave it all away, then calmly drove for the game-winning field goal.

The other Texas is a top-60 team at best. The Horns lost to No. 55 Maryland, beat No. 59 Baylor by six at home, beat No. 88 Kansas State by five, and beat No. 93 Tulsa by seven.

That has created one of the strangest dichotomies you’ll see:

  • Average scoring margin vs. the top 50: plus-13.7
  • Average scoring margin vs. the bottom 80: plus-3.5

That is not how that’s supposed to work. This only makes sense when you realize it is a Herman team. In Herman’s first year at Houston, his Cougars beat three 10-win teams (Navy, Temple, and FSU in the Peach Bowl), survived Louisville and Lamar Jackson, and destroyed the SEC’s Vanderbilt on their way to a 13-1 finish. They also needed quite a few bounces to get past Cincinnati and Memphis and lost to UConn. Final S&P+ ranking: 46th.

The next year, they beat Oklahoma and Louisville by a combined 36 and lost to SMU.

This is the Herman m.o. at this point. You will always get the best version of his team in big games, and that team will turn around and show you as little as possible against everyone else. It makes them almost impossible to project or to trust.

S&P+ is basically throwing up its hands at this point. It says the Horns have a 20 percent chance of going 10-2 or better and a 15 percent chance of going 7-5 or worse. But let’s not pretend the 11-1 scenario — win at Oklahoma State and Texas Tech, dominate WVU at home, and then just barely survive with the bare minimum against Iowa State and Kansas — isn’t on the table. Herman knows how to navigate this, as frustrating as it might be to watch.

From the link above.

So even the S&P+ guy admits his model doesn't work for Tom Herman teams. 

Link to comment
Share on other sites

On 10/14/2018 at 1:06 PM, satyanash said:

Wanted to bring this up as a curious footnote to our 6-1 start to the season so far.

We've been plummeting like a rock in the S&P+ rankings week by week. For whatever reason (possibly because they take into account MOV), the S&P+ ratings has been hating us more and more with each successive game. Using S&P+ ratings to calculate win probabilities for individual games paints an even more dire picture.

The current S&P+ ratings have us #43 in the country. Not only are we behind Oklahoma, A&M, and USC, we're also behind teams like North Texas, Purdue, and Missouri. S&P+ ratings. This week, we dropped 5 spots after beating Baylor (#59). Last week, we dropped 8 spots after beating Oklahoma 48-45, while Oklahoma somehow stayed put at #6. That was the same week Bill Connolly wrote a piece claiming we weren't back, and that we wouldn't be until we started beating lower-ranked teams by bigger margins.

So here's the thing. S&P+ ratings are meant to be predictive of future success. In other words, they're not bashing our resume so much as they're saying our W-L record is not at all indicative of how good we are as a team, and that a course correction would involve us losing several games in the back half of our schedule. So is this a fault of the model itself? Or are we due to come back to Earth against Oklahoma State and the rest of our schedule?

Image result for revenge of the nerds

Who cares with the nerd stats. Just keep winning and all will work itself out.

 

Image result for revenge of the nerds

Link to comment
Share on other sites

1 hour ago, Walden Ponderer said:

Yeah, that really is the point. Consistency gets slammed in those models, but the truth is, a team that wins by 3 all 14 or 15 times in a year is better than one that averages winning by 25+, but loses two or three games that year. Just keep winning.

The model is not intended to measure which team has had the better season. It's intended to measure which team will be better next week.

Link to comment
Share on other sites

3 minutes ago, DanRydell said:

The model is not intended to measure which team has had the better season. It's intended to measure which team will be better next week.

The problem, which is one that all models face, is that it assumes that all coaching staffs prepare equally for all opponents.

  • Like 2
Link to comment
Share on other sites

Memphis is 27th in S&P. 4-3 record. 

52 point win against Mercer (3-3), 17 point win against South Alabama (2-5), 41 point win against UCONN (1-5), 37 point win against Georgia State (2-4). 

Loss by 1 to Navy (2-4), loss by 16 to Tulane (2-4), and loss by 1 to UCF (6-0). 

Memphis being 27th is laughably bad. They haven't beaten a team that will make a bowl game and they have 2 losses to teams that are 2-4. It's fairly obvious this "model" puts entirely too much value in big wins against bad opponents. 

 

Link to comment
Share on other sites

6 minutes ago, texasstrong12 said:

Memphis is 27th in S&P. 4-3 record. 

52 point win against Mercer (3-3), 17 point win against South Alabama (2-5), 41 point win against UCONN (1-5), 37 point win against Georgia State (2-4). 

Loss by 1 to Navy (2-4), loss by 16 to Tulane (2-4), and loss by 1 to UCF (6-0). 

Memphis being 27th is laughably bad. They haven't beaten a team that will make a bowl game and they have 2 losses to teams that are 2-4. It's fairly obvious this "model" puts entirely too much value in big wins against bad opponents. 

 

Aggy is -6 against the Top 50 and +16 against Bottom 80. But take out their 38 pt win over #117 and they are only +5 against non-Top 50 teams. But they are ranked #18. 

  • Like 2
Link to comment
Share on other sites

7 hours ago, satyanash said:

 

The other Texas is a top-60 team at best. The Horns lost to No. 55 Maryland, beat No. 59 Baylor by six at home , beat No. 88 Kansas State by five, and beat No. 93 Tulsa by seven.

 

hey Bill....our starting QB and the epicenter of our offensive resurgence left the game during the first series of the game.

where is that in you "algorithm?"  

Link to comment
Share on other sites

43 minutes ago, texasstrong12 said:

Memphis is 27th in S&P. 4-3 record. 

52 point win against Mercer (3-3), 17 point win against South Alabama (2-5), 41 point win against UCONN (1-5), 37 point win against Georgia State (2-4). 

Loss by 1 to Navy (2-4), loss by 16 to Tulane (2-4), and loss by 1 to UCF (6-0). 

Memphis being 27th is laughably bad. They haven't beaten a team that will make a bowl game and they have 2 losses to teams that are 2-4. It's fairly obvious this "model" puts entirely too much value in big wins against bad opponents. 

 

Bob Stoops says "fuck you, S&P+ is the shit!'

  • Like 1
Link to comment
Share on other sites

games after Maryland:

Tulsa - 21-0 lead at halftime

USC - 37-13 after 3 quarters

TCU - 24-16 after 3 quarters

KSU - 19 -0 lead at halftime

OU - 45-24 after 3 quarters

Baylor - 23-10 lead at halftime

I think this shows that we are a much better team than the final scores have been indicating if the coaches can change how they are closing out games. Even with what they are doing we are winning, but the last 3 have been too close for comfort and it will bite us in the ass at some point.

Link to comment
Share on other sites

8 minutes ago, po elvis said:

games after Maryland:

Tulsa - 21-0 lead at halftime

USC - 37-13 after 3 quarters

TCU - 24-16 after 3 quarters

KSU - 19 -0 lead at halftime

OU - 45-24 after 3 quarters

Baylor - 23-10 lead at halftime

 I think this shows that we are a much better team than the final scores have been indicating if the coaches can change how they are closing out games. Even with what they are doing we are winning, but the last 3 have been too close for comfort and it will bite us in the ass at some point.

ESPN showed a stat during the Baylor game. Apparently we're being outscored 52-17 in the 4th quarter of our games so far this year. That's shockingly bad, and you'd never guess we're 6-1 by looking at that stat.

  • Like 1
Link to comment
Share on other sites

Continuation of my post above about S&P with a look at defense. Here again, Texas’ profile is similar to offense. On efficiency (Success Rate-40, Marginal Efficiency-21), we are close to or clearly in the top quartile. Explosiveness however (IsoPPP – 91; Marginal Explosivness-97), we are near the bottom quartile. Basically, we are a bitch to sustain drives against consistently but are susceptible to the big play. Again, anyone who has watched Texas this year would agree with that. So, explosive offenses would be our achilles heel. Texas has only given up more than 21 points twice this year. Oklahoma (who is ranked 2nd in both efficiency and explosiveness) and Maryland (who is ranked 5th in explosiveness). Pretty simple to figure out which metric matters against the Texas defense. So lets look at the statistical profile of the next 4 opponents (again, excluding KU at this point

OSU – Efficiency (Success Rate – 34, Marginal Efficiency – 28); Explosiveness (IsoPPP – 12; Marginal Explosiveness -11)

WVa - Efficiency (Success Rate – 7, Marginal Efficiency – 3); Explosiveness (IsoPPP – 31; Marginal Explosiveness -38)

Tech - Efficiency (Success Rate – 23, Marginal Efficiency – 18); Explosiveness (IsoPPP – 81; Marginal Explosiveness -71)

ISU - Efficiency (Success Rate – 109, Marginal Efficiency – 106); Explosiveness (IsoPPP – 33; Marginal Explosiveness -74)

Ruh-roh…..the next 2 opponents are not good matchups for Texas' defense from the explosives side since they are top quartile. That means expect both to hit the low 30’s which means Texas' offense has to play closer to how they played against OU to win. Now both OSU and WVa struggle with penalties and turnovers which can equalize.  The metrics would indicate that Tech may be the easiest to defend as their strength plays to Texas' strength and they are not particularly explosive. ISU’s offense is a mixed bag, but limit their big plays and they can be held in check. So, as has been posited above, explosiveness is really skewing the evaluation when it comes to the Longhorns.

So is Texas doomed to lose 3 in a row? Highly doubtful. S&P shows a loss to Tech, but I think that may be the “easiest” win from a statistical profile perspective since Texas' strengths match up well vs their weaknesses. etc. Simply put, the next 2 games will be the toughest to win and you may need to pull out the Greg Davis “explosives” counter because either Texas needs to limit big plays or keep up offensively. If OSU and WVa hit significantly more big plays, Texas is 0-2 in that stretch. 3-2 should be the floor for the remainder of the season. Of course, Tom Herman confounds S&P more than most coaches so perhaps the real floor is 5-0. Lets see how it plays out.

  • Like 3
Link to comment
Share on other sites

2 hours ago, PencilPusher said:

So is Texas doomed to lose 3 in a row? Highly doubtful. 

To be clear, while S&P+ has Texas as underdogs in the next 3 games (42.5% vs. OSU, 41.8% vs. WVU, 48.6% vs. Tech), it is not predicting Texas losing 3 in a row as the most likely result. In fact, it has Texas going 2-1 or 3-0 as being more than twice as likely as Texas going 0-3. For the next 3 games, it gives the following probabilities for Texas:

0-3 - 17.2%

1-2 - 41.3%

2-1 - 32.8%

3-0 - 8.6%

  • Like 2
Link to comment
Share on other sites

That is an excellent point. I was trying to get a handle on why the model showed us as underdogs for the next 3 games even though we are probably better than 2 of those teams and even with the third. 3 game losing streak was a bit of hyperbole on my part. I'd probably more strongly correlate us to the 2-1 record, but I don't think it matters much and we will have another week of data on Tech and WVa to look at next week. 

Link to comment
Share on other sites

20 minutes ago, PencilPusher said:

That is an excellent point. I was trying to get a handle on why the model showed us as underdogs for the next 3 games even though we are probably better than 2 of those teams and even with the third. 3 game losing streak was a bit of hyperbole on my part. I'd probably more strongly correlate us to the 2-1 record, but I don't think it matters much and we will have another week of data on Tech and WVa to look at next week. 

You aren't wrong that S&P+ is very pessimistic about us the next 3 games. While 0-3 might not be where their data settles as the most likely result, it does have us with a 60% of going 0-3 or 1-2. In fact they have the likelihood that we go 1-2 over that stretch as more likely than we go either 2-1 or 3-0. 

Link to comment
Share on other sites

On 10/16/2018 at 9:02 AM, formermav43 said:

I’m ambivalent on S&P+ in general, but nothing he said there about Texas seemed unreasonable.

Truth. Texas has the record to be #7 in the country. But god damn have they been a weird team to follow with how they've really struggled with some shitty ass teams at times.

Link to comment
Share on other sites

We should definitely avoid indices and rankings based on moral victories. We are not as reliably morally victorious as, for example, aggy. We will have to settle for W/L, conference standings, and major national polls. For now, at least.

Seriously, though. The S&P+ does expose evidence of a real trend: we play to the competition and have yet to put together a complete game. Its problem is that it doesn’t realize that is what we are doing. Its virtue is pointing out the statistical risk of living on the edge.

Edited by Magus Ossis
Link to comment
Share on other sites

21 hours ago, DanRydell said:

To be clear, while S&P+ has Texas as underdogs in the next 3 games (42.5% vs. OSU, 41.8% vs. WVU, 48.6% vs. Tech), it is not predicting Texas losing 3 in a row as the most likely result. In fact, it has Texas going 2-1 or 3-0 as being more than twice as likely as Texas going 0-3. For the next 3 games, it gives the following probabilities for Texas:

0-3 - 17.2%

1-2 - 41.3%

2-1 - 32.8%

3-0 - 8.6%

I don't think that these probabilities are unreasonable or that the "rankings" themselves are useless...I would just take the precision of these numbers with a grain of salt. Yardage and point differential are blunt instruments over 6 games. 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.



×
×
  • Create New...