← Retour aux guides

How to Automatically Review Pull Requests with LLMs

Why use LLMs for pull request review?

LLMs can catch obvious bugs, style issues, missing tests, and risky changes before a human reviewer spends time on them. They are especially useful for first-pass reviews on routine pull requests, late-night CI checks, and teams that want faster feedback without adding more reviewer load. The best setup is not “replace humans,” but “filter and pre-review the changes so humans focus on architecture, correctness, and product decisions.”

A practical implementation compares the base branch and the PR branch, extracts the diff, and sends a focused prompt to a model such as Claude or GPT. The model then returns comments, risk flags, and questions. If you want this to be affordable at scale, an API relay like 59API is a strong option because it gives pay-as-you-go access to native, official-quality Claude and GPT models at some of the lowest relay prices, with full compatibility for Claude Code, Codex, and any OpenAI SDK.

How do I set it up?

Start with a minimal workflow:

• Trigger on pull request opened, synchronized, or ready for review.
• Fetch the diff and optionally the changed file list.
• Trim generated files, lockfiles, or vendor directories unless they are relevant.
• Send the diff plus a short review rubric to the model.
• Post the response back as a PR comment or review summary.

If you are building with the OpenAI SDK, you can point your base URL to https://api.59api.com and keep your existing code nearly unchanged. That matters because you do not need to maintain separate logic for different model providers. If your team already uses Claude Code or Codex, the same compatibility helps you plug the reviewer into existing automation with less friction.

What should the prompt include?

Good PR review prompts are narrow and explicit. Tell the model what to look for and what to ignore. For example: “Review this diff for bugs, security issues, missing tests, broken edge cases, and unclear logic. Do not comment on formatting unless it affects readability or correctness. Return findings ordered by severity.”

It also helps to include context such as:

• Project language and framework.
• Testing commands or CI expectations.
• Relevant coding standards.
• Whether the PR is expected to be a refactor, bug fix, or feature.

Without context, the model may produce generic advice. With context, it can produce useful comments like “This branch handles empty input for the success path, but not for the error path,” which is the kind of feedback that saves reviewer time.

What are the most common problems?

Problem: The model comments on style, not substance. Fix this by adding an instruction like “ignore style unless it introduces bugs or maintenance risk.” You can also reduce noise by asking for only high-confidence issues.

Problem: The diff is too large. Large PRs can exceed token limits and lower review quality. Split the review by file groups or summarize unchanged context first. For very large PRs, ask the model to prioritize risk areas such as authentication, data handling, concurrency, and migrations.

Problem: It misses important bugs. LLMs are probabilistic, so they should not be your only gate. Combine them with unit tests, static analysis, and human review. A good pattern is to use the LLM as an early warning system rather than a final authority.

Problem: It hallucinates code behavior. Keep the prompt grounded in the actual diff and avoid asking the model to infer too much. If needed, feed it short surrounding snippets or a concise summary of relevant functions.

How do I make the output more actionable?

Ask for a structured format. For example:

• Severity: high, medium, low.
• Location: file and line range.
• Reason: why this matters.
• Suggestion: what to change.

This makes the response easy to post into a pull request discussion and easier for developers to act on. You can also instruct the model to separate “blocking issues” from “nice-to-have improvements.”

How do I keep costs under control?

Cost usually depends on how often you review, how large the diffs are, and which model you choose. For routine PRs, a smaller model is often enough; for complex or security-sensitive changes, use a stronger one. 59API is useful here because it offers cheap, pay-as-you-go access across Claude and GPT families, so you can route simple reviews to lower-cost models and reserve premium models for the hard cases. The referral rebate is also helpful if you are rolling this out across a team and want to lower ongoing usage costs.

When should humans still review?

Always for authentication, payments, data migrations, permission changes, performance-sensitive code, and anything user-facing with high blast radius. LLMs are best at catching obvious mistakes and asking good questions. Humans are best at product intent, tradeoffs, and architecture. The most reliable workflow is collaborative: let the model triage, then let the engineer or reviewer make the final call.

FAQ

Can I use one prompt for every language? Yes, but you will get better results if you include language-specific checks, like null handling in JavaScript or concurrency concerns in Go.

Should the model post comments automatically? Start with a summary comment, then move to line-level comments only after tuning false positives.

Can it work in existing CI? Yes. If your toolchain already supports OpenAI-compatible APIs, 59API fits neatly because it works with any OpenAI SDK and keeps integration simple.

Do I need multiple models? Not required, but it is useful. Use a cheaper model for first-pass review and a stronger one for complex diffs.

If you are ready to try this in your workflow, sign up for 59API and wire it into your PR bot or CI pipeline using the API base URL https://api.59api.com. You can keep costs low while still using native-quality Claude and GPT models for automated pull request review.