First look - GitHub Copilot code reviews

On February 26th, GitHub announced some exciting news - Copilot Code Review has officially entered its public preview phase! In this post, I’ll walk you through my first impressions of the feature and show you step-by-step how to enable it and even manually trigger it. Let’s dive in and explore this new addition!

What is GitHub Copilot code review?

Having been around in our IDEs for nearly two years (or closer to three if you include the technical preview), GitHub Copilot has rolled out its latest feature: GitHub Copilot code review, our new friendly neighborhood code reviewer. This feature provides rapid, AI-generated feedback on code changes. It assists us in refining our updates by spotting issues and correcting typos, all with the aim of preparing the pull request for merging before a human reviewer gives the final approval.

Please note that GitHub Copilot code review does not guarantee the detection of all code issues. This caution is clearly stated in the GitHub.com documentation:

Copilot isn't guaranteed to spot all problems or issues in a pull request, and sometimes it will make mistakes. Always validate Copilot's feedback carefully, and supplement Copilot's feedback with a human review.

Currently, the new feature supports the following languages:

• C#

• Go

• Java

• JavaScript

• Markdown

• Python

• Ruby

• TypeScript

You can use GitHub Copilot code review in Visual Studio Code for an initial assessment of a highlighted section and to review your changes, or on GitHub.com for a more in-depth review of the code changes. In this discussion, I will focus exclusively on the review process available on the GitHub website.

How to enable GitHub Copilot code review

There are multiple ways to enable a code review from Copilot. The simplest method is to manually assign Copilot to the pull request, which initiates the code review process:

Manually assigning Copilot to every pull request can become tedious. Fortunately, there's a more efficient solution: enabling automatic code reviews by Copilot. This can be set up through branch rulesets on either a single GitHub repository or specific repositories on organization level. You'll find the "Request pull request review from Copilot" option within the "Require a pull request before merging" checkbox. The workflow is also documented here for your convenience.

Another approach that I find quite effective is managing the ruleset through code and then importing it into the repository or organization. For a single repository, a ruleset that mandates a Copilot code review for every pull request on the default branch would be structured as follows:

{
  "id": 4111833,
  "name": "enable-copilot-code-review",
  "target": "branch",
  "source_type": "Repository",
  "source": "philwelz/github-rulesets",
  "enforcement": "active",
  "conditions": {
    "ref_name": {
      "exclude": [],
      "include": [
        "~DEFAULT_BRANCH"
      ]
    }
  },
  "rules": [
    {
      "type": "pull_request",
      "parameters": {
        "required_approving_review_count": 0,
        "dismiss_stale_reviews_on_push": false,
        "required_reviewers": [],
        "require_code_owner_review": false,
        "require_last_push_approval": false,
        "required_review_thread_resolution": true,
        "automatic_copilot_code_review_enabled": true,
        "allowed_merge_methods": [
          "squash"
        ]
      }
    }
  ],
  "bypass_actors": []
}

Here is an example of a ruleset at the organization level that ensures every pull request on the default branch of all public repos undergoes a Copilot code review.

I hope that in the future, there will be an option to import rulesets using the CLI or GitHub API, allowing for a policy-as-code-like workflow, rather than having to manually import the rulesets through the UI.

Code review in action

Once we've enabled Copilot code review or manually assigned Copilot to a pull request, we're ready to begin.

If the files changed in the pull request are in a supported language, Copilot will review them and check for potential code issues or typos. In my initial example, it identified a typo in the README.md file:

I can now either commit the suggested changes directly to my branch or validate them by opening Copilot in my workspace for further testing. I chose to commit the changes, and once the conversation was resolved, I received this neat PR overview:

If your code changes are in languages that aren't supported, such as a Terraform and Terraform provider updates, you can expect a really simplified output from Copilot:

Bonus: Although GitHub Actions aren't listed in the supported languages section, it appears that Copilot has a better understanding of changes made to them compared to something like Terraform:

Conclusion

GitHub Copilot code review is an exciting enhancement to the developer's toolkit, offering AI-generated insights to improve code quality and streamline the review process. It provides valuable feedback on supported languages, helping to identify potential issues early and enhance the overall efficiency of code reviews. By integrating Copilot code reviews into the standard CI/CD workflow, I anticipate that the traditional "four-eyes" principle will evolve into a "four-eyes-and-AI" principle. However, it's essential to remember that it doesn't replace human judgment. I'm eager to conduct more in-depth tests with a supported language and will be sharing my findings here.