Nice article. As you suggest, I’m sure it’s wrong, but that’s what makes it so good. You framed the problem really well and that makes finding solutions easier.
One that I can imagine is having societies of AI. We can have carefully curated mini LLMs that are better at logic (for instance) or good at finding dark patterns. Those can operate in parallel or provide data for foundational or big mama models.
Another approach might be developing robust systems that get stronger the more they reflect on themselves. Rather than degrading in quality due to lack of human input, they actually get better. That’s a hypothetical, but I think it is plausible that AI generated logic can help train more robust logical understanding. And, closed gardens of AI generated spam might help better identify fluff from substance.
In any case, I think it will quickly go from “AGI as a monolith” into a bizarre Darwinian ecology of competing and cooperating intelligences. I do think there is something happening now that we will all feel nostalgic for soon.
Thought provoking piece. Sure, a lots of assumptions, but not farfetched and all within the realms of possibility. I'm definitely stealing the analogy of peak oil. It matches with some of the comments of Sam Altman about the fact that improving future models will not be about growth in model size. It looks like OpenAI acknowledges that they have already reached somewhat of a glass ceiling there.
It's good to see the world is opening their eyes, slowly, to the limitations and adversarial vulnerabilities that come with deploying LLMs. But at the same time, thousands of applications are being build on top of these models everyday and rolled out to millions of users...
It doesn't take a fortuneteller to know it'll only be a matter of time before we see the downstream effects of that.
I'm thinking training only on Wikipedia post 2023 is probably a good option. Wikipedia isn't without biases, but it is aggressively curated (some would say maybe too aggressively, but that's probably better than the alternative for this usage)
Nice article. As you suggest, I’m sure it’s wrong, but that’s what makes it so good. You framed the problem really well and that makes finding solutions easier.
One that I can imagine is having societies of AI. We can have carefully curated mini LLMs that are better at logic (for instance) or good at finding dark patterns. Those can operate in parallel or provide data for foundational or big mama models.
Another approach might be developing robust systems that get stronger the more they reflect on themselves. Rather than degrading in quality due to lack of human input, they actually get better. That’s a hypothetical, but I think it is plausible that AI generated logic can help train more robust logical understanding. And, closed gardens of AI generated spam might help better identify fluff from substance.
In any case, I think it will quickly go from “AGI as a monolith” into a bizarre Darwinian ecology of competing and cooperating intelligences. I do think there is something happening now that we will all feel nostalgic for soon.
Thought provoking piece. Sure, a lots of assumptions, but not farfetched and all within the realms of possibility. I'm definitely stealing the analogy of peak oil. It matches with some of the comments of Sam Altman about the fact that improving future models will not be about growth in model size. It looks like OpenAI acknowledges that they have already reached somewhat of a glass ceiling there.
It's good to see the world is opening their eyes, slowly, to the limitations and adversarial vulnerabilities that come with deploying LLMs. But at the same time, thousands of applications are being build on top of these models everyday and rolled out to millions of users...
It doesn't take a fortuneteller to know it'll only be a matter of time before we see the downstream effects of that.
I'm thinking training only on Wikipedia post 2023 is probably a good option. Wikipedia isn't without biases, but it is aggressively curated (some would say maybe too aggressively, but that's probably better than the alternative for this usage)
What is SSO-optimization? Do you mean SEO? Or SSO for Single Sign On?