Building Your Own Code Review Agent with Claude and GitHub Actions

What This Actually Automates

A code review agent doesn’t replace human review — it catches the mechanical stuff (missing tests, obvious bugs, style inconsistencies, security red flags) before a human reviewer spends time on it, so people can focus on architecture and judgment calls.

The GitHub Actions Workflow

name: AI Code Review

on:
  pull_request:
    types: [opened, synchronize]

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - name: Get PR diff
        run: git diff origin/${{ github.base_ref }}...HEAD > diff.txt
      - name: Run review
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
        run: node scripts/review.js
      - name: Post comment
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const review = fs.readFileSync('review.md', 'utf8');
            github.rest.issues.createComment({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: context.issue.number,
              body: review,
            });

The Review Script

const fs = require('fs');

async function reviewDiff() {
  const diff = fs.readFileSync('diff.txt', 'utf8');

  const response = await fetch('https://api.anthropic.com/v1/messages', {
    method: 'POST',
    headers: {
      'x-api-key': process.env.ANTHROPIC_API_KEY,
      'anthropic-version': '2023-06-01',
      'content-type': 'application/json',
    },
    body: JSON.stringify({
      model: 'claude-sonnet-4-6',
      max_tokens: 2000,
      messages: [{
        role: 'user',
        content: `Review this diff for bugs, missing test coverage, and security issues.
Only flag genuine concerns, not style preferences. Be concise.

${diff}`,
      }],
    }),
  });

  const data = await response.json();
  const review = data.content.map(block => block.text || '').join('\n');
  fs.writeFileSync('review.md', review);
}

reviewDiff();

Scoping the Review to Avoid Noise

An agent that comments on every PR with generic feedback quickly gets ignored. Constrain it to specific, high-signal categories:

const REVIEW_FOCUS = `
Only comment on:
1. Logic bugs that would cause incorrect behavior
2. Missing error handling on operations that can fail
3. Security issues (injection, auth bypass, exposed secrets)
4. Missing test coverage for new business logic

Do NOT comment on: formatting, naming preferences, or anything
a linter would already catch.
`;

Giving the Agent Repo Context

For meaningfully better reviews, feed in relevant existing code alongside the diff — the agent can then flag inconsistencies with established patterns, not just isolated diff-level issues:

const relatedFiles = getChangedFileNeighbors(diff); // related files in same module
const context = relatedFiles.map(f => `// ${f.path}\n${f.content}`).join('\n\n');

Measuring Whether It’s Actually Helping

• Track what percentage of agent comments get acted on vs dismissed
• Compare review turnaround time before and after adoption
• Periodically sample agent comments for false positives and tighten the prompt accordingly

Conclusion

A well-scoped review agent is genuinely useful as a first pass, catching mechanical issues before a human reviewer looks at the PR. Keep its focus narrow and evidence-based — a noisy agent that flags style preferences trains reviewers to ignore it entirely, defeating the purpose.