cross-posted from: https://kbin.social/m/ArtificialIntelligence/t/483290
Training AI models like GPT-3 on “A is B” statements fails to let them deduce “B is A” without further training, exhibiting a flaw in generalization. (https://arxiv.org/pdf/2309.12288v1.pdf)
Ongoing Scaling Trends
10 years of remarkable increases in model scale and performance.
Expects next few years will make today’s AI “pale in comparison.”
Follows known patterns, not theoretical limits.
No Foreseeable Limits
Skeptical of claims certain tasks are beyond large language models.
Fine-tuning and training adjustments can unlock new capabilities.
At least 3-4 more years of exponential growth expected.
Long-Term Uncertainty
Can’t precisely predict post-4-year trajectory.
But no evidence yet of diminishing returns limiting progress.
Rapid innovation makes it hard to forecast.
TL;DR: Anthropic’s CEO sees no impediments to AI systems continuing to rapidly scale up for at least the next several years, predicting ongoing exponential advances.