GitHub Copilot: Software delivery teams’ best friend or worst enemy?

85% of developers felt more confident with their ability to deliver quality code output.

88%

88% of developers find the tool helps their focus and lowers frustration.

But just because individual developers are leaning into the latest AI trend doesn’t mean companies should rely on the tool in every scenario.

The first thing Kforce Engineering Strategist John Wright thought of when he read about GitHub Copilot was Abraham Maslov’s famous quote: “If all you have is a hammer, everything looks like a nail.”

Wright equated it to driving a car. Modern models come with all sorts of tools to improve environmental awareness, such as stopping in an emergency or notifying us if someone is in our blind spot. But ultimately, it is up to the driver to ensure they are driving as safely as possible. The tools do not eliminate the need for us to pay attention—so it is with Copilot.

Companies and professionals should carefully consider when and why they’re using Copilot. Doing so effectively can increase adoption, productivity and team velocity. But in the wrong use case, it can provide limited-to-no value and even set a team back.

So, what exactly is GitHub Copilot? Just like an aviation copilot assists the pilot with technical steps throughout a flight, GitHub and OpenAI developed a tool that helps developers write code. Both in aviation and coding, the copilot is trained for the task, but is only used when needed. There is still an expert at the helm guiding the result.

Copilot is not a replacement for a well-educated development staff, and it is not a replacement for developers themselves. A company’s development staff needs to be aware of the tool’s limitations and take responsibility for ensuring it is securely implemented.

“Ultimately, it’s up to the company to ensure that the code Copilot produces is safe and secure, protecting any critical data from breaches,” Kforce Engineering Practice Leader Bobby Vitrano said. “Copilot was trained on petabytes of source code from dozens of programming languages, not all of which were vetted for best coding practices. It is not a replacement for anything but more of an assistant to speed up cycles for educated users. It is only as good as its input, instruction and oversight.

This article will explore different scenarios in which a developer could use GitHub’s Copilot. Some are ideal for the tool, and others should be reconsidered. By understanding the strengths and limitations of the trending technology, leaders can feel confident in deciding when and how to implement the powerful asset.

GitHub Copilot’s sweet spot is in streamlining tedious tasks and giving coders a foundation to work from.

Take web APIs as an example. Copilot is great at reading any backing interfaces/procedures that a developer wants to expose via API and creating most API CRUD operations that are needed.

The tool is well trained on this type of work. The developer simply points Copilot to a series of interfaces/procedures and asks it to create the corresponding API interfaces. Copilot is then able to create the corresponding Web API endpoints for clients. This scaffolding can save developers a lot of time and repetitious coding, allowing them to focus on specific rules, processes and security.

However, that is not a replacement for validating and managing the output. The developer should still examine the code to ensure it is secure and performant. Copilot replicates baseline code in a sufficient way, but ultimately, it is not responsible for the work. Developers are the owners of the code that is produced, and as such, should ensure the result is quality codes as defined by the company.

GitHub Copilot is particularly skilled at generating unit tests based on existing code. As the developer writes test cases, they can ask the tool to examine the material and create appropriate unit test output. The developer can evaluate and accept the code, often reducing the amount of typing and time needed while creating efficiency around a tedious and simple but critical task.

Creating quality unit tests has long been a pain point – or simple annoyance – for developers. As a result, they often struggle to create quality unit tests. Common excuses for not writing test cases include:

a. I don’t have the time to do unit testing

b. I am paid to develop not write tests

c. I know how to break my own code

d. That is QA’s job

While excuses will always exist, Copilot can help eliminate some barriers that developers run into. The technology is skilled at generating many basic tests, freeing them up to instead focus on more complex tests.

But caution must be taken while using Copilot in this scenario. Copilot is good at generating simple tests based on the code it can examine, but it has no way of creating tests that are more custom or may be required to meet governing oversite. Developers still need to examine the output, write specific tests or write more complex tests that may require more oversight, interaction or coverage.

Also, Copilot is not a substitute for a good QA staff, robust CI/CD pipeline, proper stage gates or quality oversight. Relying on Copilot to replace QA can leave a company open to untested, non-performant and unsecure code being released into production.

We’ve covered two scenarios where GitHub Copilot is helpful. Just as important is understanding where the tool may fall short.

In some instances, a junior developer may be assigned a task that they don’t have the expertise to complete. So, they turn to Copilot as a “mentor” to guide them. As Copilot churns out code based on the developer’s limited experience, the junior team member accepts the code at face value and implements it. If your company does not have good policies in place for code reviews or a mature CICD pipeline, this junior developer just risked putting code into production that may introduce errors and instability. Worse, this dependence risks potentially exposing sensitive data to outside entities.

Companies need to be cautious about using Copilot with new developers or those who have bad habits. Copilot is not a replacement for knowledge and experience. It is easy to assume that Copilot will act as a “mentor” or senior level developer and provide insights and experience. This is not the case. New developers should have a good foundation of OOP development skills and solid design skills, as well as basic understanding of programming fundamentals. A developer must be able to understand and evaluate the code Copilot generates. It is their responsibility to know if the code follows company standards or introduces potential security problems.

A junior developer should be able to describe the code functionality and how it incorporates into the program to a senior team member. The senior developer needs to perform this code review until they are comfortable with the junior developer’s knowledge and understanding.

Make sure your company has policies and procedures in place for developers to follow to keep them on the right track.