Over the last few weeks, I’ve built tooling to integrate LLM-based coding agents—Claude Code and Sourcegraph AMP—into my GitHub pull request (PR) workflow. The goal? Use agents as true collaborators by routing review comments directly to them and posting their replies back to GitHub.

This post shows how it works and includes everything you need to build something similar yourself.

🧩 The Components

1. PR Comment Extraction

I start by running a shell script that fetches all open comments from a GitHub PR and reformats them as agent prompts:

python3 ../pr-review-agent/pull_pr_comments.py

🖼️ Terminal Output – Preparing Prompt for the Agent

The script generates a structured prompt like this:

📝 Please address the following PR review comments:

If a comment is a question, don't make code changes right away...

[comment_id: 2134824204]
- swift/.../CronjobConfigurationTests.swift:73 → I don't think we need this test anymore. It tests the factory method of LLMServiceProviderFactory.

2. Feed Prompt to AMP or Claude

Once the comment list is ready, I paste it into my AMP or Claude Code chat. The agent follows the instructions to:

Address each comment
Avoid unrelated refactors
Use the proper comment_id: reply format

🖼️ Agent Response in VSCode / AMP Chat

🖼️ AMP’s Completed Replies

3. Posting Replies to GitHub

Once I get the response from the agent, I paste it into another script:

python3 ../pr-review-agent/reply_to_github_comments.py

The script attempts to:

Reply to the original comment directly
If blocked (e.g., due to a formal review thread), fall back to the same line
If that fails too (e.g., line deleted), fall back to a top-level PR comment

🖼️ Reply Script with Fallback Logic

4. Final Result in GitHub UI

The agent's replies show up in the PR as follow-ups to each review comment—either inline or at the PR level, depending on what GitHub allows.

🖼️ GitHub UI: Agent Replies Posted

💡 Agent-Friendly GitHub Workflow

I’ve documented detailed conventions in CLAUDE.md and AGENT.md, including how agents should:

Wait for approval before committing
Format commits using feat:, fix:, refactor: etc.
Push changes only after confirmation
Ask for a new branch if the current one is main

Example instruction block:

- Only address the above PR comments (do not refactor unrelated code)
- After making changes, create a new commit summarizing what was done
- Then respond to each comment using the format:
  [comment_id: <id> - <path>:<line> - original_comment: "<original_comment_text>"]
  Reply: <1–3 sentence reply>

🛠️ Supporting Aliases

To help agents check build/test results, I’ve also created CLI aliases:

alias gh-latest-status='gh run list --limit 1'

gh-latest-test-logs() {
  gh run view --log --job=$(...)
}

gh-latest-swiftlint-logs() {
  gh run view --log --job=$(...)
}

This gives agents access to build state, lint issues, and test logs—all without needing a custom API.

🚀 What’s Next?

I'm planning to wrap all this into a fully autonomous GitHub agent that:

Detects new PR comments and prompts Claude/AMP automatically
Posts agent replies without copy/paste
Sends notifications when builds or checks complete

If you’re interested in this, the code lives here:
📂 Repo: github.com/mstriebeck/github-agent

📣 Final Thoughts

This system has made my PR process far smoother. Agents respond to review comments with consistent tone, context awareness, and code-level precision—something hard to achieve manually across many files.

If you're exploring LLM workflows or agent-assisted development, I’d love to hear what you're building. Ping me on GitHub or reply below.

Integrating Claude & Sourcegraph AMP with GitHub PRs