Scaling the 2019-2023 recipe — same architecture, bigger model, more data — is not enough; further progress depends on new architectural ideas.
0 members hold this · 1 public figure on record
Sign in to hold thisPublic figures on record
These are quotes korents collected from things people published elsewhere. They did not post here, and have not endorsed this site.
-
20 Dec 2024
François Chollet quoted
o3's improvement over the GPT series proves that architecture is everything. You couldn't throw more compute at GPT-4 and get these results. Simply scaling up the things we were doing from 2019 to 2023 -- take the same architecture, train a bigger version on more data -- is not enough.
↗OpenAI o3 Breakthrough High Score on ARC-AGI-Pubarcprize.org