Tell that to the 2017 Auburn team (who beat two of the playoff teams that season) who had its hands full with UCF or 2015 Florida State team who had their fill of Houston.
Teams can only play whoever is on their schedule and you can't fault many of those programs if many of the P5s avoid scheduling them, which many do.
Also, keep in mind that the P5/G5 divide is artificial (man made) in that it was a direct result of CFB being hijacked rigged and now controlled by the P5 conferences so they wouldn't have to share the $ with the other conferences, not as a result of merit. Had that not taken place and had we had a true playoff in place, then there would be alot more parity among programs and conferences and you'd see a much bigger field of contenders each season and less disparity in talent and coaching among programs, making CFB much more interesting then the 10-15 team beauty contest we have now.