That is a legitimate sticking point. I want to make sure my exit point isn't too easily hit on normal market swings but I don't want to to lose 20+%. As a result, it's hard to pull the trigger.
A quick look at VOO from 2010 to today, there are only a few sustained runs where the closing price is lower than 10% off of the all-time high closing price. For all of them, if you re-entered the ETF after 3 days of the closing price being within 10% of the all-time high, the price set a new record within a few weeks. I don't know if that's an actual indicator of when to get back in or if it's just a coincidence since there's so few extended downturns. Most of the downturns are less than a week except for MAR2020-JUN2020 and APR2022-JUN2023.
Since I do have enough to retire in that account already, I think erring on the side of wealth protection is probably an okay decision. There are worse things in the world than sitting in a Fidelity cash account earning a bit of interest.