S&P+ rankings: Overperformance or overcorrection?

[satyanash 14276]

October 8: No. 2 Texas 45, Oklahoma 12

OU was re-tooling in 2005. Texas was not. Zero. Percent. Success rate. On passing downs.

October 29: No. 1 Texas 47, Oklahoma State 28

For the second straight year, OSU bolts out to a huge early lead (35-7 in 2004, 28-9 in 2005) and then watches Vince Young erase every bit of it.

December 3: No. 2 Texas 70, Colorado 3

lol this was a conference title game. Honestly, I was surprised to see that the yardage margin was this close. I’d have guessed something more like 600-50. (This was the utter low point for the Big 12 South. Colorado stunk, Nebraska, Missouri, and Kansas weren’t ready yet, Bill Snyder was about to retire at K-State, and a mediocre Iowa State damn near won the division.)

 

Edited January 22, 2019 by satyanash

[satyanash 14276]

 

[Girdwood 118]

On 1/22/2019 at 5:45 AM, satyanash said:

October 8: No. 2 Texas 45, Oklahoma 12

OU was re-tooling in 2005. Texas was not. Zero. Percent. Success rate. On passing downs.

October 29: No. 1 Texas 47, Oklahoma State 28

For the second straight year, OSU bolts out to a huge early lead (35-7 in 2004, 28-9 in 2005) and then watches Vince Young erase every bit of it.

December 3: No. 2 Texas 70, Colorado 3

lol this was a conference title game. Honestly, I was surprised to see that the yardage margin was this close. I’d have guessed something more like 600-50. (This was the utter low point for the Big 12 South. Colorado stunk, Nebraska, Missouri, and Kansas weren’t ready yet, Bill Snyder was about to retire at K-State, and a mediocre Iowa State damn near won the division.)

 

Colorado and Nebraska were in the Big 12 South? Maybe sit a few plays out, Bill.

[OrangEngr 1034]

12 hours ago, Girdwood said:

Colorado and Nebraska were in the Big 12 South? Maybe sit a few plays out, Bill.

He referred to all 6 Big XII north teams, he clearly just typed south by accident. Please criticize him as needed, just don’t be pedantic. It’s unbecoming.

[Huckleberry 32685]

Does he not count penalties in success rate stats? Oklahoma converted a first down on a pass interference penalty on a 2nd and 8 in that game.

[Girdwood 118]

2 hours ago, OrangEngr said:

He referred to all 6 Big XII north teams, he clearly just typed south by accident. Please criticize him as needed, just don’t be pedantic. It’s unbecoming.

[TXSooner518 2432]

I mean he's right, unless you think he actually thought all the teams in the north of the Big 12 geographically were in the Big 12 South division for some reason. 

[EastTexan 662]

After reading this thread, I totally understand s&p+. 

[satyanash 14276]

Updated 2006 S&P+ rankings. Texas finished at #9 in S&P+ this year (8th in offense, 24th in defense).

 

[satyanash 14276]

Oh, and here are his 2005 rankings. The 2005 Texas squad comes in at #1 in S&P+ (#2 offense, #10 defense).

 

[Pig Bellmont 8679]

45 minutes ago, satyanash said:

Oh, and here are his 2005 rankings. The 2005 Texas squad comes in at #1 in S&P+ (#2 offense, #10 defense).

 

What a relief. I’ve been on pins and needles 

[Steel Shank 9405]

I'd like to see S&P run the numbers on my laying pipe in '05.

[Girdwood 118]

On 1/25/2019 at 1:16 PM, TXSooner518 said:

I mean he's right, unless you think he actually thought all the teams in the north of the Big 12 geographically were in the Big 12 South division for some reason. 

[NeverMarryAStripper 21780]

Based on the current climate in this country it seems like that sign would have generated some kind of protest

[Walden Ponderer 12639]

58 minutes ago, NeverMarryAStripper said:

Based on the current climate in this country it seems like that sign would have generated some kind of protest

Why, do colorblind people really need to know what the year was?

[Magus Ossis 2773]

7 hours ago, NeverMarryAStripper said:

Based on the current climate in this country it seems like that sign would have generated some kind of protest

I’m going to go out on a limb here and guess that the ditch sign offends people so often that it wouldn’t be terribly newsworthy were it to slight minority dwarf transgender deaf special olympians with some colossally crude pun.

[satyanash 14276]

September 9: No. 1 Ohio State 24, No. 2 Texas 7

This year’s Game of the Century was a rematch of Texas’ 2005 win in Columbus. This time around, the road team once again took both the victory and a spot in the national title game. Ohio State won by being Ohio State — solid and balanced offensively (as with most games, Troy Smith’s numbers were good but not eye-popping), mistake-free from a turnovers perspective, dominant in field position, and unforgiving in big-play defense. Colt McCoy was not yet ready for Ohio State.

October 21: No. 5 Texas 22, No. 17 Nebraska 20

Nebraska fans will go to their graves talking about how they were hosed in the 2009 Big 12 title game against Texas. But this is the one the Huskers absolutely, positively should have won. TWENTY POINT SEVEN POINTS’ WORTH OF TURNOVERS LUCK.

November 11: Kansas State 45, No. 4 Texas 42

Ron Prince: 2-0 against Texas.

[satyanash 14276]

[VIDEO] The play that saved the game against Nebraska in 2006

[Girdwood 118]

This guy really hates Texas. 

[Machinator 32097]

https://www.sbnation.com/college-football/2019/1/31/18204093/2019-ncaa-football-returning-starters-experience

Per his metrics, we have the lowest returning production in the P5.

[satyanash 14276]

19 minutes ago, Machinator said:

https://www.sbnation.com/college-football/2019/1/31/18204093/2019-ncaa-football-returning-starters-experience

Per his metrics, we have the lowest returning production in the P5.

Returning just 40% of our defensive production is pretty brutal, especially for a team that relies on exotic disguised coverages that take a while to master. Probably going to be a rough start to the season as the underclassmen struggle to catch up.

Quote

Fresh off of a 10-win campaign — the school’s first since 2009 — and a Sugar Bowl win, Texas is all but guaranteed to begin 2019 in the preseason top 10. The Horns bring back quarterback Sam Ehlinger and receiver Collin Johnson as headliners, plus the fruits of successful recruiting.

They do not, however, return their leading rusher (Tre Watson), leading receiver (Lil’Jordan Humphrey), three honorable mention all-conference offensive linemen, their top three tacklers on the defensive line, their top two linebackers, and three of their top five defensive backs, including corner Kris Boyd, who led the team in havoc plays (tackles for loss, forced fumbles, and passes defensed).

At just 48 percent returning production, the Horns aren’t in the “guaranteed regression” range like, say, UAB and Fresno State. But Tom Herman’s recruiting classes are going to have to break through quickly if Texas is to live up to expectations.

 

[satyanash 14276]

 

[ClubWhatever 12085]

Tweet these metrics at CDC and see if he will cancel the season

[sushihorn 3233]

5 minutes ago, satyanash said:

 

The problem is this system measures returning production with no attempt to estimate future production.  I see a TON of DB tackles, PBUs and TFLs with Sterns, Foster, Overshown, Cook and Green who will all have a year in system under their belts.  Watson and Owens from the 2019 class could contribute next season but there's no reason to rush them into major roles unless they dominate in practice.

[satyanash 14276]

2 minutes ago, sushihorn said:

The problem is this system measures returning production with no attempt to estimate future production.  I see a TON of DB tackles, PBUs and TFLs with Sterns, Foster, Overshown, Cook and Green who will all have a year in system under their belts.  Watson and Owens from the 2019 class could contribute next season but there's no reason to rush them into major roles unless they dominate in practice.

Generating an estimate of future production that correlates as strongly as returning production is probably exceedingly difficult, if not impossible. In an offensively-minded conference like the Big 12 there's no substitute for on-field experience.

[Hank Chinaski 6310]

42 minutes ago, Machinator said:

https://www.sbnation.com/college-football/2019/1/31/18204093/2019-ncaa-football-returning-starters-experience

Per his metrics, we have the lowest returning production in the P5.

To what exactly is he correlating these "returning production" figures - is it to year-to-year change in S&P+ ranking? 

For example...these are the offensive correlations:

• Receiving yards correlation: 0.324
• Passing yards correlation: 0.234
• Rushing yards correlation: 0.168
• Offensive line starts correlation: 0.153

So this is illustrating that the % of offensive line starts that are returning has a correlation coefficient of .15 with relation to the team change in S&P+ rating from 2018 to 2019...did I get that correct?  

[satyanash 14276]

 

[sushihorn 3233]

13 minutes ago, satyanash said:

Generating an estimate of future production that correlates as strongly as returning production is probably exceedingly difficult, if not impossible. In an offensively-minded conference like the Big 12 there's no substitute for on-field experience.

You are absolutely correct from the perspective of S&P+ with their universe being every program in FBS.  Serious fans of the individual team in question can make a better estimate with more detailed information as long as they can control for homer bias.  According to this formula, PJ Locke was the most important defender on the field (most tackles of any DB) and his loss should be crippling.  Does any Texas fan actually believe that?

To me it illustrates the weakness of quantitative analysis.  They have to use concepts that represent median outcomes and apply them in every case - whether that is appropriate or not.

[satyanash 14276]

Lol. Bill's getting irritated

 

[Hank Chinaski 6310]

1 hour ago, sushihorn said:

You are absolutely correct from the perspective of S&P+ with their universe being every program in FBS.  Serious fans of the individual team in question can make a better estimate with more detailed information as long as they can control for homer bias.  According to this formula, PJ Locke was the most important defender on the field (most tackles of any DB) and his loss should be crippling.  Does any Texas fan actually believe that?

To me it illustrates the weakness of quantitative analysis.  They have to use concepts that represent median outcomes and apply them in every case - whether that is appropriate or not.

And, as far as I can tell (don't have time/inclination to go back and read all the methodology behind this), we are now talking about (essentially) correlations to correlations.. That means we are getting further away from the actual data. (Unless I am misunderstanding this.)

Edited January 31, 2019 by Hank Chinaski

[rickyspub 801]

1 hour ago, sushihorn said:

You are absolutely correct from the perspective of S&P+ with their universe being every program in FBS.  Serious fans of the individual team in question can make a better estimate with more detailed information as long as they can control for homer bias.  According to this formula, PJ Locke was the most important defender on the field (most tackles of any DB) and his loss should be crippling.  Does any Texas fan actually believe that?

To me it illustrates the weakness of quantitative analysis.  They have to use concepts that represent median outcomes and apply them in every case - whether that is appropriate or not.

Yeah, the fact that the correlations are keyed to all teams rather than a correlation based on how those positions performed at each school means there is a potential for wide variation. Connelly claims his number account for things like returning DB tackles, #2 and #3 WR, and QB, but how could it do that effectively for any given team when all teams share the same position correlations? If Team 1 throws the ball around evenly to six guys and loses their #1, how is that comparable to a team that threw to their departing #1 60% of the time? How does his system account for a team like Georgia Tech? They will get dinged more if they change all their WRs rather than RBs. I bet their performance is affected less by changing WRs, which Connelly says is more important overall.

You note Locke, but Hager is another guy whose output could actually be worse than the guys behind him. Losing Wheeler is probably more harmful because we also lose Johnson, but in a vacuum just losing Wheeler might not be a bad thing. On offense, losing Watson could hurt, but it ignores the fact that Watson was a grad transfer (as was Anderson), so what if we get another one this summer? It also can't full account for the fact that Ingram would have likely been our leading rusher if he weren't hurt.

 

[El Hornarino 367]

1 hour ago, satyanash said:

Lol. Bill's getting irritated

 

Fifty percent of the time, it works every time.

[sushihorn 3233]

One fairly common situation illustrates the potential absudity of Connelly's formula.  Let's say an outstanding underclassman is injured early and the senior backup plays in his place.  That team will "lose" 100% of the production from that position the following year according to this algorithm and their predicted performance will be downgraded accordingly when the real starter returns to the line up.  You can extend the same principle any underperforming senior who is replaced by a more talented player the following year.

Edited January 31, 2019 by sushihorn

[Magus Ossis 2773]

He is like the “scientist” concluding that pulling the legs off a grasshopper impairs its senses because it no longer jumps when approached. Correlation tells you a summary of what you see. It doesn’t tell you how or why. He is looking at aggregate correlation data from varied circumstances and using it not to make aggregate predictions but specific ones. That’s like worrying about breast cancer in the child of someone who died of breast cancer— before confirming whether the person is man or woman.

Edited January 31, 2019 by Magus Ossis

[DanRydell 799]

The problem is this system measures returning production with no attempt to estimate future production.  I see a TON of DB tackles, PBUs and TFLs with Sterns, Foster, Overshown, Cook and Green who will all have a year in system under their belts.  Watson and Owens from the 2019 class could contribute next season but there's no reason to rush them into major roles unless they dominate in practice.

 

That’s becaude returning production is only one component of S&P+ projections. Recruiting rankings and the prior year’s S&P+ rating are also factored in.

[satyanash 14276]

 

[Magus Ossis 2773]

If Army can survive next year's slate with anything like the success they had this year, maybe they deserve playoff consideration. They will run the gauntlet during their regular season, facing production return rank* #5 Rice, #9 Hawaii, and #14 Western Kentucky. That's three top 15 teams. Not everyone fills the schedule with easy outs such as #121 Texas.

Speaking of #121 Texas, they will have their hands full early against not only #15 LSU, but even more serious, a week 3 tilt against the #5 Rice Owls. They complete the tough OOC trifecta with a La Tech squad that fell just outside the top 25 at #30.

*Connelly SB returning production rank. AP, coaches, and CFP returning production ranks still pending

If someone asked you, ceteris paribus, whether 300 pound guys make better OL or whether 250 pound guys do, we know the answer. But what kind of dipshit would become indignant at the suggestion that his predictions for future football game outcomes, modeled based in part on mean OL weight, may be flawed in the case of certain teams or players? Older college football players are often stronger and more mature in a number of helpful ways than are 17-year-olds. Correlating average age of OL would likely suggest some benefit to older players. But that doesn't mean every set of 5 60-year-olds is far superior to every line of 5 20-year-olds.

[texasstrong12 4668]

Bill Connelly wouldn't get so much shit from Texas fans if he would just admit that S&P+ isn't that great at projecting Herman coached teams. Instead he digs in. 

[Braff Zacklin 13399]

 

[satyanash 14276]

 

[satyanash 14276]

Bill has Texas as the #2 team in the country this year, behind #1 USC and ahead of Florida at #3.

The conference-level adjustments I added to S&P+ did very, very happy things to the Big 12’s 2008 S&P+ ratings. Oklahoma State, Texas Tech, Kansas State, Colorado, and Iowa State all saw their rankings rise by at least 10 spots, and not only were there seven league teams in the Off. S&P+ top 10, there were five in the overall top 10. I’ve maintained for a while that Missouri’s 2008 team, which went 10-4, was quite possibly/likely better than the 2007 edition that went 12-2 and finished fourth in the AP poll. The numbers back me up. But while the Tigers improved a little, much of the rest of the conference improved a lot. OSU was suddenly awesome, and unlike in 2007, Missouri had to play Texas. That made quite a difference. By the next year, Mizzou had lost Daniel and Jeremy Maclin, Tech had lost Harrell and Michael Crabtree, OU had lost Bradford to injury, etc. But 2008 was indeed a perfect convergence of innovation and experience, and if a 4-team CFP had been in place in this season, the conference almost certainly would have had two teams in it. (It’s possible we’d have had an OU-Texas rematch in the semifinals, too. That wouldn’t have sucked.)

[gmr548 16463]

I don't think it's heresy, or even a stretch to expect the defense to have issues early. Major pieces are being replaced at every level. That always takes time and even veteran groups struggle in this conference. The key will be rounding into form by conference. Fortunately, LSU doesn't scare you offensively, and there are a couple of cupcakes, including one away from DKR (won't call that any actual road game). The schedule sets up well for the defense to progress as the year goes on.

Sent from my SM-G920V using Tapatalk

[Magus Ossis 2773]

48 minutes ago, gmr548 said:

I don't think it's heresy, or even a stretch to expect the defense to have issues early. Major pieces are being replaced at every level. That always takes time and even veteran groups struggle in this conference. The key will be rounding into form by conference. Fortunately, LSU doesn't scare you offensively, and there are a couple of cupcakes, including one away from DKR (won't call that any actual road game). The schedule sets up well for the defense to progress as the year goes on.

Sent from my SM-G920V using Tapatalk
 

Yeah, saying that replacing key players causes problems early in the season isn't evidence that you have hit upon a great computer model. A great model would at least account for the factors that a casually analytic fan would assess: whether the departing player performed better than his own predecessor, the circumstances under which he came to be a starter, whether he was playing out of position, whether he was playing hurt, a variety of measureables on his replacement, whether his replacement is an experienced backup, etc.

[Huckleberry 32685]

Boy I just can't figure out why he seems so happy about the Big 12 doing well in his system in 2008 but fluffs the SEC every chance he gets in the more recent past.

Truly a mystery.

[Ricky Butler 2657]

On 1/31/2019 at 3:54 PM, texasstrong12 said:

Bill Connelly wouldn't get so much shit from Texas fans if he would just admit that S&P+ isn't that great at projecting Herman coached teams. Instead he digs in. 

I think Herman's first team started in Bill's pre-season top 20 and ended the season 49th. So he's been wrong both Herman years.

I respect what he's trying to do, but it's not my fault his magnum opus will take 20 years to fine tune. 

I'm a Texas fan. Each season takes a year of my life, so if you're wrong even 30% of the time, then your model is just for fun. And please don't make me defend our victories during the season because your model doesn't like us. That's just annoying.

[satyanash 14276]

2019 S&P+ projections are in. These combine returning production with an estimate of recruiting impact to produce the preliminary projected S&P+ rankings for 2019.

Bill has Texas at #35 with a +8.9 S&P+ rating. That puts us at fourth in the Big 12 conference (behind blOU at #4, Oklahoma State at #22, and TCU at #34) and third in the state of Texas (behind Texas A&M at #13 and TCU at #34).

 

[satyanash 14276]

Further down, Bill explains why Texas is ranked so low at #35 (we're projected to go 7-5 in 2019).

Quote

Texas is 35th???

In 2015, Tom Herman’s Houston Cougars enjoyed a magical run. They went 13-1, rolled to the AAC title, and beat Florida State in the Peach Bowl. The numbers were unimpressed. UH ranked just 53rd in (the updated version of) S&P+, looking more like a 10-4 team on paper and propped up by five points per game of good turnovers luck. In Houston’s 2016 preview, I wrote this:

    Houston is going to be good. In 2016, something like a 9-3 record would be considered disappointing. This scenario plays out a lot in this sport, and it shouldn’t a surprise that it’s what the skeptical S&P+ ratings are projecting.

Houston improved to 39th in S&P+ ... and went 9-3. You can defy the numbers once, but it’s really hard to do it twice in a row.

Herman’s team might have something familiar going on heading into 2019.

In 2018, another Herman team defied both expectation and statistics. In his second year at Texas, his Longhorns ranked 32nd in S&P+ and, per second-order wins, had the look of an eight-win team. For every strong performance (namely, wins over Oklahoma and Georgia), there was a dud or near-disaster — a loss to Maryland, near-losses to Tulsa, Baylor, Texas Tech, Kansas, etc. Against anyone but the top teams, they did the bare minimum; it bit them once and nearly did so many other times. Still, they won 10 games, finishing with a win over a depleted but talented UGA in the Sugar Bowl. From that point forward, they were all but guaranteed to find a spot in everyone’s preseason top 10.

    The final piece of the puzzle for Herman in Austin might not be filling in holes on the two-deep. It will be figuring out how to field a team that plays every game like the Sugar Bowl.

S&P+ doesn’t tend to trust teams that perform so inconsistently. Plus, Texas must replace a higher percentage of last year’s production than any other power conference team.

    The Horns bring back quarterback Sam Ehlinger and receiver Collin Johnson as headliners, plus the fruits of successful recruiting. They do not, however, return their leading rusher (Tre Watson), leading receiver (Lil’Jordan Humphrey), three honorable mention all-conference offensive linemen, their top three tacklers on the defensive line, their top two linebackers, and three of their top five defensive backs, including corner Kris Boyd, who led the team in havoc plays (tackles for loss, forced fumbles, and passes defensed).

Herman has signed two straight dynamite classes, and his Horns have what appears to be a manageable schedule for a top-10 team, if they can get by LSU at home. But they’ve got a lot of churn to overcome, and they benefited from a lot of good fortune last year. S&P+ is going to project them to win about seven games. A Herman team has defied stats a couple of times now, but they haven’t yet done it back-to-back.

 

[Valmy77 13092]

What "good fortune" did we benefit from last year? 

Quote

The final piece of the puzzle for Herman in Austin might not be filling in holes on the two-deep. It will be figuring out how to field a team that plays every game like the Sugar Bowl.

We did play every game like the Sugar Bowl. We had a big lead and then let Georgia come back and make it a one score game.

I would rather we stop playing games like that.

[Pato del Muerto 52322]

If you give him LSU and ou as losses, I wonder which teams he picks for the other 3 losses. 

It’s weird that he only talks about final scores, presumably because that’s what his model looks at. But when his model doesn’t perform, why not take a deeper dive to explain why?  Close win vs Tulsa...but one time 28-0 lead before limping home. Close win vs Baylor...while playing our backup qb all game long. Close win vs tech...while playing all backups in the secondary much of the second half against an air raid team due to multiple injuries.

are injuries not bad luck?  Just because you can’t account for them doesn’t mean they don’t exist and affect performance. 

[sushihorn 3233]

19 minutes ago, Valmy77 said:

What "good fortune" did we benefit from last year?

For fumbles he uses a projected recovery rate rather than actual recoveries and attributes the difference to luck.  It ignores context but if all you have is aggregate data it makes sense within those limitations.  Of course,= in reality there is a very big difference between a fumbled snap and a ball ripped out with a crowd of defenders waiting to pounce.