His problem is that he is a Republican senator in, as you say, a BLUE state. His most important bills would likely get support from most Democrats, meaning they are non-starters for Republican senate leadership. He has tried to shift left on some issues but has little chance to influence the GOP agenda, and GOP leadership has hung him out to dry in the center.
The fact that voting Gardner out is pretty much a prerequisite for a change in control of the chamber is not lost on Colorado voters. As the most vulnerable Republican senator, I'd expect the people to know that if he wins, the Senate stays in Republican hands and all he ends up being is a rubber stamp for Trump and McConnell's agenda, no matter how well-meaning he may be.
Basically, I agree with you that this is a referendum on Trump and the GOP more so than on any sitting senator.
Here's a good article on his marijuana legislation that McConnel probably lost somewhere underneath the pile of passed House bills he's refusing to take up and how it adversely affects his potential support from the pot industry.
https://www.politico.com/news/2020/05/23/cory-gardners-marijuana-problem-275029