[context: I am literally a certified AI/ML specialist and builder and have taken enough high-order math at UT to at least accurately conceptualize what's happening in the black box]
The end state of "unable to walk and falling down" is effectively the end-state of a "Hapsburg" model. When you train a model on synthetic data produced by a similar model, you get the same concentration of error effect that is observed in genetics whenever inbreeding happens in a population.
In the current and active development state of AI, these LLM companies like Anthropic and Gemini and OpenAI are scraping the internet for ANY data they can, and they don't really care about the provenence of its origin. As a result, the "for-profit" LLM's are getting trained on a heavily poisoned and synthetic dataset that contains a bunch of data that will embed defects into the model and train it to produce worse errors. Not to mention the general problem with model poisoning that Anthropic published a paper on a month or two ago