Just last week we did 4 days there with our 5 and 7 year olds. We stayed at the Hotel Vitale on the corner of Mission and the Embarcadero nearly directly across the street from the Ferry Building. The hotel was fine with kids and the location was great. We walked across the street and ate breakfast in the Ferry Building every morning (a bunch of little bakeries and they have a killer farmers' market on Saturday) and then headed down the water from there (trolley a couple of times, pedicab a couple of times, walked a couple of times) to see the wharf, exploratorium, aquarium, sea lions, etc. One day we ubered to golden gate park, explored that area, and then ubered to North Beach to eat at a little bitty Italian joint that's been there since the 1930's (trying to remember the name will ask my wife and report back). It was great but my favorite part was the Saturday night post-supper walk through North Beach back to the hotel. That area is super dense and has a true urban feel. It also still has a little bit of an edge to it and it was packed Saturday night. Seeing all of those people (and all kinds of people -- homeless, tourists, long time locals, partying young professionals, etc.) packed into such an urban setting really blew my kids' minds. They loved it and I loved seeing them react to it. My 7 year old daughter told me the next morning that she wanted to go to college in SF and live in North Beach. We also stopped at City Lights book store and told them about the beat stuff and a great little Italian bakery.
We also had a great teaching moment that night as a homeless guy went apeshit pretty close to us when we were walking back to the hotel. I told the kids to just ignore him keep looking straight ahead and keep walking and when we got back to the hotel we had a good talk about how that kind of shit happens sometimes and that it's ok and this is how you handle yourself, etc.
The last day we rented a car through the hotel (it was very easy (they bring it to hotel and you leave it at valet that night when you get back) and we got a decent rate) and did Muir Woods and then saw some family in Sonoma. Muir Woods was great and the kids really enjoyed it. Sonoma was fine and we had to go to see family but I probably wouldn't recommend going all the way out there when there's so much other cool kid stuff to do.
Also had really good dim sum at Yank Sing (I think they have 2 locations and both are walking distance from Vitale).