Beware of AI Coding Assistants | GitHub Copilot vs JetBrains AI Assistant

This article is the transcription of my video on AI Coding Assistants. As promised, I compared the most popular AI-powered coding assistants powered. The plan was to create a test and show you the results at the end of the article. However, I noticed immediately that I had more important lessons to share with you other than the fact I prefer GitHub Copilot over the competition. There are features you should use and features you should avoid. But let me start by explaining very briefly why I chose GitHub Copilot.

What’s an AI Coding Assistant?

A Coding Assistant is a tool capable of helping software developers in multiple ways.

Its primary function is to perform autocompletion and code prediction. You start by typing a few letters and it will suggest lines or even entire functions. It’s really fascinating, especially when you use them for the first time.

But how do they do it?

They are built on top of large language models that have been trained on basically all open-source code and some private codebases. A human would never be able to process the same amount of data… not even in multiple lifetimes. Because of that, these models have learnt all the patterns we use while coding. So… when they analyze the context of our project, they can correlate it to patterns they’ve already seen and suggest accurate predictions.

Why did I choose GitHub Copilot over the JetBrains AI Assistant?

The reason I chose GitHub Copilot is that it’s really good at autocompletion. It’s very responsive and accurate and it offers multiple predictions at one time. You can browse them in a dedicated tab or cycle through them by using shortcut keys.

In contrast, the JetBrains AI Assistant was less responsive. It would predict code only on empty lines and that forced me to use explicit prompts. I had to tell it exactly what I wanted, and I found that disruptive. I want the AI assistant to read my mind… to predict based on what I’m typing… I don’t want to use direct prompts.

And this is exactly the first important lesson I want to share with you. Don’t over rely on prompts to code your application, especially if you are a junior. There’s much more I need to tell you… but let me explain why prompts are bad.

Why prompts are bad?

First, the AI generated code is full of imperfections. The model can predict what you’re looking for, but it cannot guarantee that the suggested code will compile. If you overuse this approach, you’ll end up spending a lot of time refactoring code to fix syntactical errors. I personally felt I was demoted to a code monkey.

The imperfections are not only syntactical but can also be logical. There’s no guarantee that the AI will generate bug-free code. Think about it, do you want to spend most of your time writing code or reviewing code? I prefer writing code. I prefer using my problem-solving skills.

There’s also another issue… especially if you are a junior. Are you sure that you can spot bugs or logical errors? I know the answer. It’s no. I overlooked many bugs while using these tools. And it’s completely normal. I know from over two decades of coding that it’s hard to spot a bug in code that I’ve written myself, let alone finding on a piece of code I’m copying pasting without even understanding well. That’s a recipe for disaster.

Test auto-generation

Another feature I strongly advise you to avoid is the test auto generation. First of all, it’s incompatible with test driven development or behavior driven development where the tests are written before the implementation. It also has the same issue of prompting, which is that will spend most time refactoring and reviewing the auto-generated code. But the most important concern I have is that it can give you a false sense of security.

The AI will most probably generate tests that are aligned with the logic of your implementation. If your code is wrong to start with, the generated tests will reinforce your errors and make them more difficult to spot and solve.

Explain and Simplify Code

That said.. it’s not all bad. I recommend that you start using an AI coding assistant and I’m strongly convinced they will make us not only more efficient and productive, but better software engineers.

There are two features I loved, and I really recommend that you use them extensively: explain code; and simplify code.

I was really surprised by how good these tools are at explaining code. I’m convinced this feature can flatten the learning curve for junior and even senior developers that are trying to learn a new programming language, a framework, or maybe when they need to get accustomed with a new codebase.

But my most favorite feature is the simplify code or refactor code. As you know, a problem can be solved in multiple ways. However, over time we get comfortable with a few patterns and that makes writing code very repetitive. With this feature, the AI will analyze our code and offer alternatives, and there’s nothing more exciting than discovering a new approach which is more elegant and efficient than the one we’re used to. I really really love this feature and every time I see a convoluted function, I make sure to check what alternatives the AI has to offer.

The future of Software Development with AI

In conclusion, all I wanted to say is that despite I think we all need to get accustomed to a new way of writing code with AI assistance, I want you to stay in control of what you’re writing. You are the pilot. The AI is your copilot, your pair programmer. Keep enjoying the thrill of writing code, of using your creativity and problem-solving skills. I cannot predict if and when the AI will replace us completely as software developers, but I’m sure it’s not now.

Let’s keep in control of what we produce.

Bonus: Decision Matrix