Skip to content

Codex PR-review bot

A small polling bot that gives the team OpenAI Codex code reviews on GitHub pull requests — on demand (comment /codex review) and, optionally, automatically the moment a PR becomes ready for review. It uses a locally-authenticated codex CLI as the worker and GitHub comments as the transport.

It exists because Codex Cloud's GitHub code-review integration requires a paid plan to review other people's PRs; running the CLI on an always-on machine sidesteps that. Reviews are read-only — Codex never builds, runs, or pushes PR code — and only post as comments.

What runs

Piece File Role
Scheduled Task CodexReviewBot scripts/ci/codex-review-bot/Register-CodexReviewBotTask.ps1 Fires every 5 min on the dev VM, runs as the logged-on user (rakletai)
One polling pass scripts/ci/codex-review-bot/Invoke-CodexReviewPass.ps1 Scans PRs, runs Codex, posts the review, then exits
Setup & ops notes scripts/ci/codex-review-bot/README.md Host setup, trust model, operations
Isolated Codex home C:\codex-bot\.codex (on the host) auth.json + minimal config.toml (model=gpt-5.5, model_reasoning_effort=medium) — kept separate from the desktop Codex config
Bot working clone C:\codex-bot\rakletv3 (on the host) Dedicated clone used only to check out PR heads for review

There is no long-lived daemon: each pass does one sweep and exits, so the only state to reason about is what's already on GitHub (a reaction, or the bot's own review comment).

How it works

Each pass does two things.

1. On demand — scans recent comments for the /codex trigger, reacts 👀 to claim it, runs the review, posts it, and marks 🚀 when done. Dedup keys on the 🚀 reaction, so an interrupted run is retried, not lost.

2. On ready (opt-in, -AutoReviewReady) — for each open, non-draft PR whose author is trusted, that became ready at/after the deployment cutoff, and that has no Codex review yet, it runs the same review and posts it. Here the dedup marker is simply the bot's own ### Codex review comment already being on the PR — no reaction needed. A per-pass cap drains a burst of newly-ready PRs gradually.

flowchart TD
    T[Scheduled Task every 5 min] --> P[Invoke-CodexReviewPass.ps1]
    P --> D{/codex comment<br/>from trusted author?}
    D -- yes --> RX[react 👀 → review → post → 🚀]
    P --> A{non-draft, trusted author,<br/>ready ≥ cutoff,<br/>no Codex review yet?}
    A -- yes --> RV[fetch PR head → Codex read-only review → post comment]
    RV --> M[the '### Codex review' comment<br/>is the dedup marker]

Trigger syntax

Comment Behaviour
/codex or /codex review Full multi-lens Codex review of the PR.
/codex <free text> Codex run over the PR diff with your own instructions (e.g. /codex check the webhook idempotency).

The review brief

Both paths run Codex with a senior-leadership, multi-lens brief. It reviews as the:

  • CTO — correctness, edge cases, concurrency/races, missing tests, regressions, API/contract breaks;
  • CISO — authorization & tenant isolation, injection, secrets, PII, the PR-content trust boundary;
  • CPO / CMO — product & UX impact, only when materially relevant;
  • CEO — cost / scale / customer / compliance risk.

The PR branch is checked out, so Codex reads beyond the diff for context. It is told not to invent findings, and anything it cannot confirm without building or running the code is reported under a "Needs hands-on verification" heading rather than asserted. That section is the signal for whether the -ReviewSandbox workspace-write lever (which would let Codex build/run) is ever worth enabling — see Decisions.

Trust & safety model

The bot runs Codex on our machine against PR content, so input trust matters:

  • Author gate. Only a trusted author_association (OWNER/MEMBER/COLLABORATOR) can cause a review — the commenter on the on-demand path, the PR author on the on-ready path. Fork PRs from untrusted authors never trigger it.
  • Read-only. Codex runs -s read-only: it reads the checked-out tree but never executes, builds, or runs PR code.
  • No PR-supplied instructions. Because the PR head is checked out, both paths pass --ignore-rules and -c project_doc_max_bytes=0 so attacker-controlled .rules / AGENTS.md files in the branch can't be loaded as privileged instructions. The bot's own trusted CODEX_HOME config still loads.
  • No write access to PRs. The bot only posts comments and reactions; it never pushes commits.

Deployment

Runs on the dev VM ai, not ci-vm-1, as Scheduled Task CodexReviewBot (interactive logon — runs while rakletai is logged on), every 5 minutes.

  • The dev VM already has a real codex.exe on PATH (so the bot's [Process]::Start("codex") resolves) plus gh authed as rakletai. The npm build on ci-vm-1 only shipped a codex.cmd shim, which the launcher could not resolve — hence the dev VM.
  • The bot uses an isolated CODEX_HOME (C:\codex-bot\.codex) holding only auth.json and a minimal config.toml, so reviews don't drag in the desktop Codex MCP/plugin/computer-use stack.

To (re)deploy after a code change: pull master into C:\codex-bot\rakletv3, then re-register:

cd C:\codex-bot\rakletv3\scripts\ci\codex-review-bot
.\Register-CodexReviewBotTask.ps1 -IntervalMinutes 5 -RepoPath C:\codex-bot\rakletv3 `
    -CodexHome C:\codex-bot\.codex -Interactive `
    -AutoReviewReady -AutoReviewSince "<ISO-8601 UTC, e.g. deploy time>"

-AutoReviewSince defaults to now, so enabling auto-review never retroactively sweeps the existing backlog — only PRs that become ready afterwards are reviewed.

Operating it

  • Logs: %TEMP%\codex-review-bot.log on the host.
  • Run now: Start-ScheduledTask -TaskName CodexReviewBot
  • Pause: Disable-ScheduledTask -TaskName CodexReviewBot
  • Re-review a PR: comment /codex review (works after an auto-review too), or remove the bot's 🚀 reaction from the original /codex comment.
  • Volume knobs: -AutoReviewMaxPerPass (default 3), -AutoReviewAuthors (allowlist; empty = all trusted), -IntervalMinutes, and model_reasoning_effort in the bot's config.toml.

Posting /codex from a terminal

Post the trigger from PowerShell, not Git Bash — MSYS path conversion mangles the /codex argument into a Windows path (C:/Program Files/Git/codex), which the bot correctly ignores.

Decisions and rationale

Decision Why
Host on the dev VM, not ci-vm-1 The dev VM already has a working codex.exe + gh auth; ci-vm-1's npm install only provided a .cmd shim the launcher couldn't resolve. The bot is just a poller — uptime is the only real requirement.
Read-only reviews by default, workspace-write as an off-by-default lever Read-only is safe and fast and covers most value. Building/running is left to CI (which already runs every PR's code); the bot's "Needs hands-on verification" output is the signal for whether to ever turn workspace-write on.
Auto-review scope = all trusted authors, newly-ready only Comprehensive coverage without retroactively reviewing the in-flight backlog.
Shared ChatGPT/Codex quota → "reduce volume" The bot uses the same Codex account as interactive use. Rather than a second account, we cut per-review cost by dropping reasoning effort high → medium (keeps gpt-5.5 + the multi-lens depth) and using a calmer 5-min cadence.

Known issues & learnings

  • Shared usage quota. The bot's auth.json is the same ChatGPT account used interactively, so heavy auto-review volume can exhaust the daily usage limit — Codex then fast-fails (exited 1, "You've hit your usage limit … try again at
  • BOM dedup bug (fixed). Set-Content -Encoding UTF8 emits a UTF-8 BOM on Windows PowerShell 5.1, so early review comments began with an invisible U+FEFF. The on-ready dedup regex didn't treat it as whitespace, so the bot re-reviewed PRs every pass. Fixed by writing comments without a BOM ([IO.File]::WriteAllText + UTF8Encoding($false)) and making the dedup BOM-tolerant. On-demand was never affected (it dedups on the 🚀 reaction).

References

  • Code: scripts/ci/codex-review-bot/ (README.md has host-setup detail).
  • PRs: bot (#13962), auto-review + multi-lens brief (#14308), BOM dedup fix (#14313).