| JEMC said: We're running instead of walking, and that always ends in disaster. The thing, tho, is what date will they use to train those high.level models? Because I've seen online how current models get dumber because most of the new data you can find online is made by AI bots, not humans, and that kind of data has no value and is full of errors. |
IIRC, there's a few hundred trillion tokens worth of human-created data available in the internet, so in theory that should be enough until 2030 - 2032.
If transformers can't do a relatively general purpose tool after this timeframe, I'm not sure what these companies would do. Pivot to a new paradigm like neuromorphic computing seems more promissing than pay people to create more content or feed syntethic content back into these models.
China, by the way, is years ahead of everyone else in that field.
