Taking My New GitHub Copilot ‘AI Pair Programmer’ for a Spin — Visual Studio Magazine


Taking My New GitHub Copilot ‘AI Pair Programmer’ for a Spin

I tried to get GitHub Copilot to write this article for me, but it couldn’t do it. It can do lots of other things. Some are pretty cool.

If you haven’t heard of GitHub Copilot, it’s a cutting-edge coding assistant tool similar to IntelliSense/IntelliCode that can do a lot more than suggest how to complete your lines of code. It’s based on advanced OpenAI Codex technology that allows natural language interaction. It just recently became generally available for $10 per month (or $100 per year). It’s also available in a 60-day trial, which I used.

I’m no coder, but I figured I could install it and try a few things. Here’s how that went.

Setup, Authentication, Pairing … and Mistakes
One reason I’m not a coder is that I’m impatient and impulsive and don’t follow directions. I just dive in and try things until something works. (If you’re like that — probably not a coder, then — this article can serve as a lesson in what not to do in order to get started.)

So I didn’t follow the instructions outlined in “Getting started with GitHub Copilot in Visual Studio Code” documentation that says to go to the VS Code Marketplace and search for the extension and install it. I didn’t read that documentation. I just installed the extension from within the editor.

Upon trying to use it, I got a notification to sign in to access GitHub Copilot.

Sign In Notification
[Click on image for larger view.] Sign In Notification

I didn’t really expect that to work, but it appeared to.

Signing In
[Click on image for larger view.] Signing In

I was notified “The extension ‘GitHub Copilot’ wants to sign in using GitHub,” but the process timed out, generating this message: “You have not yet finished authorizing this extension to use GitHub. Would you like to keep trying?”

I managed to work my way to an authorization screen.

Authorize GitHub for VS Code
[Click on image for larger view.] Authorize GitHub for VS Code

To finish authenticating, I had to navigate to GitHub and paste in the one-time code generated from the authorization page.

Finishing Authentication
[Click on image for larger view.] Finishing Authentication

That took be to GitHub again where I entered the code.

Device Activation
[Click on image for larger view.] Device Activation

I was finally able to authorize GitHub for VS Code (with GitHub reminding me that I live in the Montana town of Libby).

[Click on image for larger view.] Authorize

And I was notified that I was all set.

All Set
[Click on image for larger view.] All Set

Except I wasn’t. Upon trying to use the tool, I had to allow the extension to open up the authentication URI.

Another Step
[Click on image for larger view.] Another Step

Then I was notified my experience wasn’t fully configured and I needed to complete my setup.

Complete Setup
[Click on image for larger view.]Complete Setup

I went to Settings and reached an “Enable GitHub Copilot” screen.

[Click on image for larger view.]Enablement

The 60-Day Trial
After which I reached the 60-day trial page.

Someone Remind Me to Cancel this in 59 Days
[Click on image for larger view.]Someone Remind Me to Cancel this in 59 Days

And confirmed my payment details.

Payment Details
[Click on image for larger view.]Payment Details

Some preferences were next, where I could indicate if I wanted GitHub to allow or block suggestions matching public code, and whether I agreed to allow GitHub to use my code snippets for product improvements. I obligingly agreed to the latter, though I can absolutely guarantee no one wants to use my code snippets to improve their product.

Select Preferences
[Click on image for larger view.]Select Preferences

And was then presented with yet another step: pairing with my pair programmer.

Paring Up
[Click on image for larger view.]Paring Up

After being notified (with falling, colored confetti, no less) that I had successfully paired, I got a message to install the extension. I had done that first thing, remember, so the documentation informed me in that case I just had to restart the editor.

Finally Up and Running
I did that, and my pair programmer and I were finally up and running! I had to jump through some extra hoops on account of not following setup directions, but it still felt like that even the official process took a lot of steps.

Anyway, I created a JavaScript file and asked my pair programmer to generate a function to add two numbers together. You do that by typing in a request in a commented-out line (preceded with //) and clicking Enter. So the JavaScript file was empty except for this line:

//Generate a function to add two numbers together

Clicking Enter caused the tool to suggest some grayed-out code that could be accepted by clicking Tab. That made Copilot churn for a couple seconds (the little GitHub Copilot icon in the bottom right spins).

Copilot activated the line suggestion — turning the grayed-out text into properly formatted and colored code — and then stood by awaiting more instructions, as indicated by a red tilde.

The Beginning ...
[Click on image for larger view.]The Beginning …

Repeatedly clicking Enter brought in the whole function, complete with a helpful comment that the end of the function had been reached.

The End
[Click on image for larger view.]The End

Upon that initial success, I got frisky and decided to ask Copilot to compute the value of pi (π). As I started typing “Compute pi …” I got a suggestion to use the Leibniz formula for that pi computation. When I started typing “Compute the value of pi …” Copilot suggested “… to the Nth digit.”

You can supposedly cycle through the suggestions with the Windows key combinations (with Mac equivalents) of Alt + ] (forward) or Alt + ] (backward). I couldn’t get the suggestions to cycle through various options, nor could I get Copilot to compute the value of pi at all, in any shape or form, even when accepting suggestions. Update: The day after this article was published I tried the pi calculation again and it worked perfectly. Go figure. I used the same exact command but this time that resulted in this:

//Compute the value of pi
//The value of pi is 3.141592653589793

I tried to generate a function to compute the area of a circle and discovered that if I just kept clicking Tab, Copilot just kept generating comments explaining aspects of the equation, eventually resorting to repeating itself until I clicked Enter and completed the function code.

Repetitive Comments
[Click on image for larger view.]Repetitive Comments

I asked Copilot to spin up a new HTTP server and it did this:

var http = require('http'); //import http module from node js

I tried a series of “Create a …” commands to see what Copilot offered up as suggestions. The suggestions for different comments are shown below, with the suggestions enclosed in parentheses.

Create a new (copilot object)
Create a function (that takes a string and returns the string in reversed order)
Create a table (of contents for the copilot test)

Regarding the latter, all that happened after hitting Tab to accept the suggestion is that Copilot generated this comment:

//Create a table of contents for the copilot test. This will be used to generate the copilot test. The table of contents will be generated from the copilot test.

When I kept pressing Tab it just started to repeat comments.

I tried the // Express server on port 3000 command from official documentation and found out that Copilot also started to repeat itself when I just kept clicking Enter/Tab combinations. It first generated more than 75 lines of code that looked like it may be a complete and correct program. But when I kept clicking Enter/Tab combinations, it eventually started to repeat itself in the Routes section of the code.

Repeating Routes
[Click on image for larger view.]Repeating Routes

Explain this Code
I also tried the “explain this code” functionality provided in the companion GitHub Copilot Labs (preview) extension. As its name indicates, this experimental tool can take a chunk of code and explain in natural language what it does.

I didn’t have to jump through any hoops to install that tool; I did it quickly and easily from within the editor — no context switching or restarts required.

In the sidebar, it provides two options, Explain and Language Translation.

My first attempt at Explain (highlighting some code and clicking the Ask Copilot button) produced an error message: “Error generating Copilot response, please try again. Changes in the prompt or stop sequence might fix this.”

I don’t know what “changes in the prompt” or the “stop sequence” even mean, but reloading VS Code and signing back in to use Copilot worked.

I grabbed a chunk of code off the Mozilla site and asked Copilot what it meant.