In the longterm? Sure, definitely full automation. I am a bit skeptical about the bullish timelines of Dario Amodei et al, but at this point, it seems clear to me that it is just a matter of time, before full AGI is here.
Biggest limitation today? Other than token costs?
Well, for me, the inability to learn outside of a context window, i.e. to learn new stuff or acquire new abilities after training. It is not so much learning about a code base, as parsing through it every time (although i am sure that things like codex jot things down in a compressed fashion, so that the model doesn't have to parse it _every_ time _completely_). In any event, the context windows are getting so large, though, that this problem is less and less obvious.
Second issue for me: hallucinations, they are unfortunately still a thing, but also here you can see a lot of progress. And it usually self-corrects at some point anyway...
And, thirdly, those silly reasoning errors they sometimes still make. For me, that underlines the fact that we are not dealing with human minds here, but something else entirely.
Having said all that, i'm still blown away by Claude and even Gemini everyday. And i'm getting pretty enthusiastic about models you can start to run on consumer hardware, like Qwen 3.8 27B. Of course, nowhere near the larger models, but they are getting useful. And that is really significant, if you are working with code which you'd rather not send to an external company.
