So, yeah, my mother tongue is German, so the English texts may be a little bumpy here or there. I feel confident enough with my English but that doesn’t change the fact that it’s a secondary language for me and this is not really a professional project where I could pay some natively English speaking nerd to fix my mistakes.
That said I’m aware how debuggers attach to the processes they’re analysing, I just wasn’t aware that this would turn the word exclusive if used in this context. Thanks for bringing it up though! Learned something!
Yeah, that’s the technical lingo meaning. Attach a debugger means that you have a running process without a debugger running with it, and then you attach the debugger to it to get debug output from that process.
But I still don’t quite get the intended process.
So you run the code the LLM outputs with a debugger and let the LLM interact with the debugger? Not really sure if that helps, because for the LLM would need to know how to operate the debugger and would need to understand what problems it should be looking for.
Current systems (e.g. Github Copilot) combine the LLM with static code analysis and compiler outputs to find and fix errors. They can also execute the code and run tests and compare outputs with expected outputs.
Current systems (e.g. Github Copilot) combine the LLM with static code analysis and compiler outputs to find and fix errors. They can also execute the code and run tests and compare outputs with expected outputs.
Am I misunderstanding something or does this sound like programmers making themselves unnecessary?
It’s not a “themselves”. It’s large corporations trying to make programmers unnecessary.
It’s like blaming Tesla test drivers for making taxi drivers unnecessary.
But it’s very far from the point where programmers would be unnecessary.
LLMs lack any understanding of the bigger picture. They work for tiny throwaway tasks that you won’t use in the future. The stuff they make is hardly runnable, let alone maintainable. It usually needs multiple rounds of asking it to do the right thing before you get anything remotely worth your time.
The feedback loop with static code analysis and so on generally doesn’t work. Once an LLM gets lost enough to produce compiler errors (which happens quite fast) it gets completely lost, since it then only focusses on fixing the compiler errors while totally forgetting what the actual task was. So it will change the program so that it compiles, but then it will not do what it was supposed to do.
It’s a hot mess.
Studies say that using LLM while programming reduces measured performance of the programmer by 20-30% as soon as the task isn’t something entirely trivial.
Edit: When an LLM gets into that compiler error loop, it’s like taking a non-compiling program written by a first-year student and handing it and the compiler output to the rest of the class and everyone sequentially gets a shot at fixing it while completely ignoring anything the others have done befora apart from taking their code as input.
Only the very junior ones who are writing code without understanding why they’ve been asked to write it. Anyone with more than about 18 months experience will be able to start deciding what to actually build, and I haven’t seen LLMs be particularly helpful with that yet.
No worries, and please remain critical.
So, yeah, my mother tongue is German, so the English texts may be a little bumpy here or there. I feel confident enough with my English but that doesn’t change the fact that it’s a secondary language for me and this is not really a professional project where I could pay some natively English speaking nerd to fix my mistakes.
That said I’m aware how debuggers attach to the processes they’re analysing, I just wasn’t aware that this would turn the word exclusive if used in this context. Thanks for bringing it up though! Learned something!
Yeah, that’s the technical lingo meaning. Attach a debugger means that you have a running process without a debugger running with it, and then you attach the debugger to it to get debug output from that process.
But I still don’t quite get the intended process.
So you run the code the LLM outputs with a debugger and let the LLM interact with the debugger? Not really sure if that helps, because for the LLM would need to know how to operate the debugger and would need to understand what problems it should be looking for.
Current systems (e.g. Github Copilot) combine the LLM with static code analysis and compiler outputs to find and fix errors. They can also execute the code and run tests and compare outputs with expected outputs.
Am I misunderstanding something or does this sound like programmers making themselves unnecessary?
It’s not a “themselves”. It’s large corporations trying to make programmers unnecessary.
It’s like blaming Tesla test drivers for making taxi drivers unnecessary.
But it’s very far from the point where programmers would be unnecessary.
LLMs lack any understanding of the bigger picture. They work for tiny throwaway tasks that you won’t use in the future. The stuff they make is hardly runnable, let alone maintainable. It usually needs multiple rounds of asking it to do the right thing before you get anything remotely worth your time.
The feedback loop with static code analysis and so on generally doesn’t work. Once an LLM gets lost enough to produce compiler errors (which happens quite fast) it gets completely lost, since it then only focusses on fixing the compiler errors while totally forgetting what the actual task was. So it will change the program so that it compiles, but then it will not do what it was supposed to do.
It’s a hot mess.
Studies say that using LLM while programming reduces measured performance of the programmer by 20-30% as soon as the task isn’t something entirely trivial.
Edit: When an LLM gets into that compiler error loop, it’s like taking a non-compiling program written by a first-year student and handing it and the compiler output to the rest of the class and everyone sequentially gets a shot at fixing it while completely ignoring anything the others have done befora apart from taking their code as input.
Only the very junior ones who are writing code without understanding why they’ve been asked to write it. Anyone with more than about 18 months experience will be able to start deciding what to actually build, and I haven’t seen LLMs be particularly helpful with that yet.