GitHub Copilot vs JetBrains AI Assistant vs Duet AI
This article complements “Beware of AI Coding Assistants” by giving more insight on the comparison I did between GitHub Copilot, JetBrains AI Assistant and Duet AI by Google. As I already mentioned in the other article and related video, I decided to create a subscription with GitHub Copilot because it’s far superior when it comes to code prediction and auto-completion. However, there is more information I have to share. I also want to highlight that these tools are rapidly evolving so I will make sure to retest them in 6 months to check if anything changed.
JetBrains AI Assistant
On paper, JetBrains AI Assistant stands out as the top choice for existing users of JetBrains solutions. It promises a deep integration with their IDEs, and I wouldn’t bet on anyone else to do it better than them. It also leverages different large language models under the hood, selecting the best one to solve the current problem.
As expected, there are a lot of goodies like the ability to autogenerate a commit message or analyze a stack trace with just a click which I really loved. You can select any portion of code and ask the assistant to explain it to you, fix an issue, or also improve it.
Another useful feature is the ability to autogenerate tests. I know everyone loves writing tests, but now you can do it with just a few clicks!
That’s wonderful!
On the other hand, while the inline code completion and chat interactions show promise, I noticed some issues.
The inline suggestions tend to activate only at the beginning of a line, becoming stagnant once typing starts. I tried to figure out if there was some setting to fine tune this behavior, but I couldn’t find anything. There’s the option to prompt the AI within the code editor, but I find it clunky and distracting. If you had a different experience, let me know in the comments!
Another item that did not convince me is that the code suggested by a chat prompt is often logically correct, but it does not align perfectly with project content, requiring manual refactoring once copied in a target class.
Let’s move onto the next one.
Duet AI
Next up, we have Duet AI, which has the smallest feature set among the three contenders, but it does leverage the power of Google’s Codey, which is a family of data models built on PaLM2. While its feature set may be less extensive, Duet AI does shine in certain areas.
Its inline code completion feature was more responsive than the JetBrains AI, providing multiple suggestions as I typed, saving me from the hassle of writing repetitive boilerplate code. Test generation was quite good as well.
The chat interface was responsive. However, its understanding of project context seemed limited. For instance, when I asked all three solutions to suggest a method for persisting an entity to a database while having the spring mongodb dependency already included in my project, Duet AI suggested using the Mongo Driver instead of the more appropriate Spring Repository interface. Duet AI Chat also lacked quick action buttons to copy paste the suggested code snippets.
Finally, I’d like to mention one more issue that I found quite frustrating: the citation warnings. Whenever I copied a snippet of code from the chat, my editor would flag it as potentially unlicensed code and prompt me to manually dismiss the warning. This became so tiring that I assumed there must be a setting to disable this behavior, but I couldn’t find one.
Anyway, now time for the last contender.
GitHub Copilot
GitHub Copilot is powered by a generative AI model developed by GitHub, OpenAI, and Microsoft. It’s best integrated in Visual Studio Code, but I’ll showcase you its integration with a JetBrains IDE to make a fair comparison with the JetBrains AI.
Something I want to highlight is that Copilot Chat is not currently publicly available for JetBrains IDEs. You need to join a waitlist and there’s no guarantee that you’ll be accepted. Fortunately, I was. Anyway…
GitHub Copilot’s inline code completion is very responsive. Not only that, but you can ask for a different suggestion with a shortcut while in the code editor. Alternatively, you can browse multiple suggestions in a dedicated tab and accept the one you prefer.
Their chat, despite being in beta, is already in very good shape. The responses are accurate and take into account the context of your project. It was also able to understand my packaging structure and suggest classes that are consistent with my conventions. You also get two quick action buttons that you can use to insert the snippets into the code editor.
Like JetBrains AI Assistant, I was also impressed by its ability to analyze code selections. You can explain them, fix issues, or even attempt to simplify them. You can also generate tests.
Conclusion
AI Coding Assistant Comparison Matrix
In the end, I decided to stick with GitHub Copilot because of its responsiveness and accuracy. JetBrains AI has the potential to be the best choice for JetBrains IDE users if they improve their implementation of code completion. Duet AI offers less features and it is far from the competition. In addition, it is the most expensive and forces you to one year commitment.
If you are wondering whether you should have a subscription with an AI Coding Assistant, my answer is a definite yes. The AI Coding Assistant will make you a better developer overall; not only by improving your efficiency, but also by freshening up the way you write code. There are obviously some risks and pitfalls you need to be aware of. If you didn’t read my previous article on the topic, I suggest you to continue reading: “Beware of AI Coding Assistants“.