It helps to be good nationally on defense when your conference plays god awful offense. Antiquated schemes and mediocre, underdeveloped talent galore up and down the PAC. So I take any claims of Cal even being all that good defensively in the first place with an absolutely massive grain of salt. Then looking deeper Sirmon only recently was made Co-DC and he isn't even the primary guy. Tim Deruyter is their primary DC who calls plays, as I understand it, and above them both is their HC Wilcox who himself has a background in being a DC and is probably far more responsible for whatever success they've had then the guys under him. Nor have Sirmon nor Dereuyter shown themselves to be weapons on the recruiting trail though maybe someone can correct me on that if I'm mistaken? I know its hard to land bigger fish at Cal but the quality of talent they would now be expected to bring in, and the level of competition they will have trying to recruit, would be a sizable step up.
I'm not saying he is a shit tier coach. To me A-tier = Whales like Muschamp, B tier = really good guys with some questions or not as long a track record like Freeman, Arnett, etc. and then C-tier are people that have been... fine, but hardly mind blowingly impressive, and are only getting hired because they are friends of friends.