As a senior developer, I don’t find copilot particularly useful. Maybe it would have been more useful earlier in my career, but at this point writing a prompt to get copilot to regurgitate useful code and massaging the resulting output almost always takes as much or more time as it would for me just to write whatever it is I need to write. If I am able to give copilot a sufficiently specific prompt that it can ‘solve’ my problem for me, I already know how to solve the problem and how to write the code. So all I’m doing is using copilot as a ghost writer instead of writing it myself. And it doesn’t seem to be any faster. The autocomplete features are net helpful because they’re actually what I want often enough to offset the cost of reading the suggestion and deciding if it’s useful. But it’s not a huge difference (vs writing it myself) so that by itself is not sufficiently useful to justify paying the cost myself nor sufficient motivation to go to the effort of convincing my employer to pay for it.
You must log in or register to comment.
The autocomplete is fucking fantastic for writing unit tests, especially when there’s a bunch of tedious boilerplate that you frequently need SOME OF. I’m also really impressed by its ability to generate real code from comments or pseudocode.
Generally, though, I find it pretty awful for writing non-test code. It too often hallucinates an amazing API and I kick myself for not knowing it existed. Then I realize it’s because the API doesn’t actually exist, and the dumb fucker is clearly borrowing from a library from a completely different stack.
The autocomplete is amazing
It just takes the mental strain out of syntax and I can focus more on the logic and structure of the code
Like you said much better for writing unit tests, but also for log messages and comments
Even the chat is sometimes useful but not as much. I treat it as much junior engineer assistant/rubber duck. I.e. I never completely accept what it outputs and usually review everything but quite often it gives me a different idea/approach even if it doesn’t write it 100% correct the first time. Because I always review it, having it autocomplete the next line is my preferred approach so I verify each line as they come.
Once you start to past 10/20+ lines in a row you will have a bad time.
Not just the mental strain of syntax, but also variable naming, which I find to be a much bigger hurdle. Copilot will nearly always give you a “good enough” name so you can just move on to solve the actual problem. You can always come back to rename it if necessary.
Interesting. The way I work, variable naming is one of the key areas that I would never want to outsource to an AI – careful choice of variable names is a key part of code quality for me: unimportant things should have neutral, non-distracting names while mportant things often cause me to break out a thesaurus for just the right word.
It’s important for sure. It just so happens to also be one of those things that are very easy to verify but hard to do, which is what makes it perfect for automation.
The other nice thing about letting AI do naming is that these are names that are very statistically likely given the context. That means it’s more likely to be understood by others. If I come up with something myself, it might make sense to me, but it might not to someone else reading the code. I think this is especially important when you’re working in your own little bubble and don’t get many eyes on your code.
I often felt that current ML speeds up newbie devs by effectively teaching them the language and libraries — but slows down experts that already know the stack well from memory. I started coding in a new language and system, and ML can be a bit faster to teach me things and provide simple snippets than stack overflow
But over time I’ve learned that there are very specific things that ML can do really well, and I can save time when I apply those techniques. For example, it’s excellent at converting from one language or style to another, ex migrating configs from json to yaml. It’s also pretty good at writing configs or generating template code based on them. It’s good at picking an emoji from a list. It can write small functions or provide a template html layout. So I humbled myself and started integrating it into my workflow where it actually works
I often felt that current ML speeds up newbie devs by effectively teaching them the language and libraries — but slows down experts that already know the stack well from memory.
It depends. You don’t need LLMs to write stuff for you that you already know. You use them to take.care of the drudge work or explore things you are not familiar with. Things like Copilot’s /explain can speed up onboarding even for seasoned developers, and Copilot can also help you speed up iterations on proofs of concept. For example, I’ve been using Copilot to experiment with some changes to the software architecture of some projects I own, and it’s fantastic at generating code following specific design patterns. It’s also fantastic to get it to iterate designs in near-real.time by prompting it with directives such as “repeat the last example but implementing X with design pattern Y and moving the implementation to Z”. You are presented with examples that you can browse through and get a taste of what you’d get, but with a fraction of the time. To top things off, you can prompt Copilot to present pros and cons and even propose optimizations.
Like any tool, it has its purposes. You just need to learn how to use it.
i find it useful for things outside my areas of expertise. been doing a lot of devops lately and even though it usually fails in the specifics it can generate the broad structure of a yaml pipeline and enough pointers that i can find the right thing in the docs.
I’m actively campaigning at work to have github be declared no longer safe for data we need to keep sovereign. This includes all our ansible junk. As the corporation is based in America and is comically ignorant of the privacy rules this particular rest-of-world sub-organization needs to work under to keep data from getting to America - arm’s length no-shared-data kind of op and everything - it’s continually baffling we used github.com before. With this it seems a sad joke for security, privacy, and compliance.
Yep. Github Copilot is good enough for me to move all my stuff off of Gitgub.
Whats the best alternative that you know?
Nah, it’s all hyped up bullshit that has to be babysat and manipulated to a degree that you may as well just write your damn code.
But beyond that, I’d argue that it’s actually damaging for engineering organizations, because it means the org is incurring the maintenance cost of code not written by its engineers and that has no real thought put behind it. Maybe you can eventually coax it to produce code that’s not completely broken shit, but it’s code that your org doesn’t actually “own” from a maintenance and knowledge-base perspective. The social aspect of code maintenance with this shit is always massively overlooked.
I have not and will not ever use AI generated code that I don’t thoroughly understand. If you properly understand the code you’re committing there shouldn’t be any damage. And beyond AI you should never commit code that you don’t properly understand unless it’s a throw away project.
Like most have already said, the auto complete is top tier while the chat is hallucination-riddled and not always useful. I find that if I’m asking Chat a question, my problem is already so complex that the AI struggles to answer it without the entire context of the application. It will give me unrelated answers, fake answers, or extremely basic ones that miss the broader context. It’s really a coin flip on whether it will help.
I have also had the autocorrect make a mistake once and that was extremely annoying. It was the type of mistake I would have made but took way longer to figure out because I trusted it too much
Like most have already said, the auto complete is top tier (…)
My experience is the exact opposite. Even though it has its moments, more often than not it just hallucinates and proposes a lot of stuff that neither matches definitions nor could possibly compile. I guess that this might reflect the impact of having classes with similar names in multiple namespaces but it’s bad to the point I prefer to rely exclusively on plain old autocorrelation.
It could also be the language choice, which one are you utilizing? I could see some languages having a worse experience than others.
I’ve found it is exceptionally smart with bash. It often knows what to do better than I can, because I’m no master at bash. I’m proficient enough to know when it’s right though, and it’s usually pretty on point.
The autocomplete is nice but I don’t find it a game-changer. The comment about writing tests is on point though, but that’s the only place I found out useful.
I rarely use the prompt stuff, but it’s one of the best autocompletes I’ve ever used though since it usually finishes a line or two when I have part of it typed. Works great for things like conditionals and loops, not as good at more complicated code so I usually ignore it.
Copilot and that whole model is a bunch of shit
Get Claude.ai, do the thing of pasting the code you want it to modify into the chat window, and it’ll spit back out some modified sections of code you can put back into the file. It is more time consuming, but it actually works.
For intense operations you can make a project, upload the main headers and documentation, and it’ll absorb them into its sphere of awareness and start being able to take account of them when it does the above in chats within that project.
It is not perfect (as no LLM assist is) but it saves a lot of time and is not beset with the growing incompetence and failure with which OpenAI seems to be afflicted more and more with every passing month
Somewhat useful with (neo)vim, where it’s (however tiny af) context window spans every open buffer
I don’t use chat, it’s usually useless. Autocomplete is good enough that I can worry about concepts and Copilot will tab me the SQL blocks, loops and functions; I feel like it’s a better flow and I’m faster over all.
For stuff like Angular it knows 95% of what you’re trying to do since the possibilities are limited.
I don’t use chat, it’s usually useless.
I think Chat is the most useful feature of Copilot. Prompts like /docs work impeccably, but /explain and /optimize is also pretty good. /tests is hit-and-miss if you have zero tests and require too much context if you already have them. More often than not /fix is a waste of time.
Where I found Copilot to be quite useful is something unexpected: naming things. You can prompt it to give suggestions, you can ask it to refactor things for you. Quite nice.
I think that Claude is far better at generating code, and explore new stuff, but Claude is also down and broken extremely often,not to mention it’s annoying limit of 10 questions per half a day.
Nop, it’s slow and dumb imo. It has only ever been able to write stuff I could type quicker and with less bugs anyways.
chatgpt has been helpful two or three times. And it makes a decent ish code monkey when you give it a template. But in any editor with multiline editing it’s redundant
I use chat the most. It’s pretty good once you understand the importance of building context, set up personas, and feed it workable prompts. The biggest mistake I see people do is presume that you can expect it to output gold when inputting garbage.
Once you build up an understanding of what personas work for your personal tastes and what context you need to have, it can output impressive results. The most success I’ve been having is with somewhat complex refactorizations. Stuff like “refactor X so that Y and Y” can save you a lot of time.
The most disappointing experience has been with writing unit tests. copilot has this infuriating tendency to remove old tests when you’re prompting it to add new ones. You need to explicitly request it to append tests to file X without overwriting existing tests for it not to mess up, and even then results are sketchy. For unit tests it’s also important to setup good contexts otherwise whatever time you save by prompting copilot to write them will be wasted refactoring code to use specific frameworks and follow specific styles.
I’ve run into that exact issue with copilot (deleting my tests). It is infuriating.
I don’t think I’d trust it to refactor code for me, not for anything important. I’d need to completely understand both the initial state and the result on a statement-by-statement level to be confident the result wasn’t secretly garbage and at that point I might as well write everything myself.
I primarily use it for C++ in Unreal Engine and use it almost exclusively to write log statements. The way to log something is done via a macro like so:
UE_LOG(LogCategory, Warning, TEXT(“My variable: %s”), *SomeStringVar)
Writing that boilerplate soup gets tiresome after a while, so having Copilot autocomplete the log statement for me based on other statements in the same file and the context of the function is godsend.
It does of course happen that the text contents are wrong, but then I have that skeleton to work with. Just erase the text and type the correct contents I want. Saves so much time.
I’d create my own macro or function for that. I have enough ADD that I cannot stand boring shit like that and I will almost immediately write a pile of code to avoid having to do boring crap like that, even with copilot.
My company pays it for me. My use cases are split between new development, refactoring and debugging.
For new code, given that our code base is proprietary but very extensive, it provides nice code snippets that would be a pain to write by hand (it’s mostly C code) such as test code
I can focus on the concepts and have the autocomplete do the rest for me. I swap keypresses for reviewing code, which is not bad.
For refactoring, I seldom use it because I haven’t found an use case for it. Most of the changes involve moving code around, adding glue or deleting dead code.
For debugging, I sometimes use the chat to get documentation on public APIs from Microsoft or other places. I use this documentation to check for invariants and to reduce the scope of what I’m trying to find out.
