Agreed. But as a thought experiment, if there is a predominant religion that says "all shirts must be blue", then there is a chance that laws / regulations will be passed mandating blue shirts. As someone who likes to wear green shirts I have skin in the game. So it may be accurate to say that atheists / agnostics don't have skin in the game about whether scripture is true or not, but they certainly have skin in the game regarding whether scripture is perceived as true when it is a major cultural force. And given that skin the game, many atheists / agnostics have incentive to understand the scripture, if nothing else to try to influence how it is perceived.
Bring this specifically to homosexuality, I believe your view that it is biblically wrong but you don't want to pass laws discriminating against homosexual behavior. I don't share your view, but I believe it. At the same time, while an agnostic, I do have skin in the game about scriptural beliefs / understandings because many Christians who have similar views on Christianity as you do want to discriminate against homosexual or other non-Christian behavior.