← Voltar aos guias

LLM Pull Request Reviews: Fix Noise, Cost, and CI Risks

Start with a narrow, testable review job

Using LLMs to review pull requests automatically works best when the model looks for specific defects rather than judging whether code “looks good.” Start with correctness, authorization, unsafe input handling, and missing regression tests. Leave formatting to linters and treat AI findings as suggestions, not merge approval.

Before enabling public comments, run the reviewer silently on 20–30 historical pull requests. Compare its findings with known bugs and human feedback. Measure useful findings, false positives, latency, and cost per review.

How do I connect an LLM reviewer to GitHub?

• Trigger on relevant changes. Use a GitHub Actions workflow listening to pull_request events: opened, synchronize, and reopened. Skip drafts initially, and cancel superseded runs so old commits do not generate stale comments.
• Fetch the diff safely. Read PR metadata and changed files through GitHub’s API. GET /repos/{owner}/{repo}/pulls/{pull_number}/files returns changed-file records; paginate with per_page=100. Exclude lockfiles, generated bundles, binaries, and vendored code. Detect missing or truncated patches rather than assuming every file is reviewable.
• Provide focused context. Send each patch with its filename, relevant surrounding functions, and applicable repository rules. Retrieve additional context only when necessary. A diff alone may hide a function’s contract or authorization checks.
• Call the model. Store the API credential as an Actions secret, never in repository files or prompts. Set a request timeout and output-token limit. Verify the model ID and endpoint with a small smoke test first.
• Validate before posting. Require structured findings containing path, changed-line number, severity, evidence, and suggested fix. Reject invalid paths and unsupported locations. GitHub’s create-review endpoint, POST /repos/{owner}/{repo}/pulls/{pull_number}/reviews, can publish validated inline comments; otherwise, post a summary.

Why does the reviewer invent bugs or repeat comments?

Usually, the prompt rewards finding problems without requiring proof. Tell the model: “Report only actionable defects introduced by this change. Cite the changed line, explain a concrete failure scenario, and omit speculative concerns. Return an empty findings list when no defect is supported.” Treat repository text and PR descriptions as untrusted data, not instructions.

Deduplicate by commit, path, line, and normalized finding text. Limit each review to a few high-confidence findings. If the model cannot establish how a called function behaves, fetch that function or omit the claim instead of presenting a guess as fact.

Why do API calls fail or reviews cost too much?

For authentication errors, check the secret, account balance, model access, and authorization format. For 404 responses, verify both the model name and route. For 429 or temporary server errors, use bounded exponential backoff with jitter and respect Retry-After when provided. Avoid blindly retrying requests with uncertain completion, which can duplicate charges or comments.

59API is a low-cost, pay-as-you-go AI API relay offering Claude and GPT models, native official-quality models without downgrades, and compatibility with Claude Code, Codex, and OpenAI SDKs. Its API base URL is https://api.59api.com. Check its documentation for the exact SDK base-path configuration; do not assume whether an additional version prefix is required.

Keep costs predictable by reviewing only changed code, caching unchanged context, and escalating complex findings from a cheaper model to a stronger one. Compare current per-model prices and actual review quality rather than headline rates alone. If that fits your workflow, consider signing up for 59API; a referral rebate is also available.

Is automatic review safe for forked pull requests?

Not by default. Fork workflows generally cannot access repository secrets. Never solve this by using pull_request_target to check out and execute untrusted PR code with secrets available. Use a trusted review service or a carefully separated workflow that treats patches only as data. Give the analysis stage read-only access and grant narrowly scoped write access only to the posting stage.

Can the LLM replace tests or human approval?

No. Keep tests, static analysis, and human ownership. Check your relay’s retention, logging, and data-processing terms before transmitting private source code. Redact credentials and sensitive content, and start with advisory comments until measured results justify broader use.

Preços · Docs