As for silver, my conclusion is to keep any pieces that I want to hold long-term, because I just like their shiny looks or whatever, and sell the rest as convenient, but I have given up on investing in it. I did take some profits on a 100 oz bar that I bought just over 20 years ago, at about $5.30/oz, which seems like a nice profit, but it probably came up short of an S&P 500 fund, and was a lot more complicated to sell, so I will not try that anymore.
Gold is for keeping indefinitely, unless there comes some emergency situation, like when I need to flee the country with only the clothes on my back (and the gold in my pockets), or having to barter with dragons. But I have not actually had much success accumulating gold, I always think it is too expensive. About 2 years ago, I was excited because the price of a US Saint Gaudens $20 in nice condition was little more than metal price, and I was patiently waiting for a drop in the spot price so I could snag a nice buy of a cool coin that I have always wanted one of. But then a bunch of strange stuff happened, and I still have not had my chance...