Skip to content

Daily Sentry triage agent

A Windows Scheduled Task on the dev VM that, once a day, turns the top unresolved prod Sentry issues into Linear cards and draft fix PRs — the automated front end of the ENG-165 no-silence burndown.

It does not mute, ignore, or resolve anything, and every PR it opens is a draft for a human (Zafer) to review. It is a triage assistant, not an auto-merger.

What runs

Piece File Role
Scheduled Task RakletSentryDailyTriage scripts/ci/sentry-daily-triage/register-task.ps1 Fires daily at 08:07 local, runs as the logged-on user
Orchestrator scripts/ci/sentry-daily-triage/run.ps1 Loads secrets, selects candidates, invokes claude -p, logs the run
Candidate selection scripts/ci/sentry-daily-triage/select-candidates.ps1 Deterministic: pull → noise-filter → rank by events × users → JSON worklist
Agent runbook scripts/ci/sentry-daily-triage/RUNBOOK.md What the headless agent does: dedup, bucket, file cards, open PRs

Daily flow

flowchart TD
    T[Scheduled Task 08:07] --> R[run.ps1]
    R --> E[load-agent-env.ps1<br/>SENTRY_TOKEN]
    R --> S[select-candidates.ps1]
    S --> W[(worklist JSON<br/>top 25 ranked)]
    R --> C[claude -p + RUNBOOK.md]
    W --> C
    C --> D{dedup vs<br/>sentry-auto-triage label}
    D --> B{bucket per ENG-165}
    B -->|a real bug| P[Linear card + DRAFT PR<br/>review: zafercaliskan]
    B -->|b mis-leveled| L4[Linear card → ENG-164]
    B -->|c external noise| LN[Linear card → source-filter candidate]
    P --> Z[assignee: Zafer]
    L4 --> Z
    LN --> Z

Guardrails (enforced by the runbook)

  • ≤ 10 new Linear cards per day (all buckets combined).
  • Drafts only — never marked ready, merged, or --admin.
  • Never resolves/mutes a Sentry issue — resolution is human, post-deploy verification (ENG-163).
  • Dedup, two layers: (1) Linear — via the sentry-auto-triage label + a hidden <!-- sentry-auto-triage:RAKLETV3-XX --> marker (and existing [Sentry RAKLETV3-XX] cards), so an issue is never re-carded; (2) open PRs — the agent reads a snapshot of all open PRs and skips any issue already covered by an open PR (matched by shortId / Sentry id / permalink in title, branch, or body), so it never duplicates a human's or another bot's PR.
  • No competing bots: Sentry Seer autofix is disabled (autofixAutomationTuning=off) so it no longer opens seer/fix/* PRs; the open-PR dedup above also covers any of its still-open PRs.
  • Code work is worktree-isolated — the agent never checks out in the main c:\repos\rakletv3 tree or touches master.
  • Low-confidence → investigation PR, not invented code. If the agent can't derive a confident fix from the stack trace, the PR is a root-cause writeup, not a guessed code change.
  • Assignee = Zafer on every card.

Bucketing (ENG-165)

Bucket Example Outcome
(a) Real bug unhandled error/fatal hitting real users New Linear card + draft fix/investigation PR (consumes the daily cap)
(b) Mis-leveled / hygiene info/warning, recaptcha-action-mismatch, anti-forgery on bot POSTs Appended to the ENG-164 hygiene issue — no new card, no PR
(c) Provable external noise scanner/probe paths (/actuator, /.ssh/…) Appended to the ENG-174 source-filter issue (human-gated) — no new card, no PR

Cap priority: the ≤10/day cap is spent on bucket-(a) real bugs first, highest impact first. Ranking by events × users alone would let low-value hygiene noise outrank real bugs and starve them — so (b)/(c) never consume the cap, and deferred real bugs are listed in the run summary for manual pull.

Setup / operations

# Register (run from the main checkout after this lands on master):
pwsh -File scripts\ci\sentry-daily-triage\register-task.ps1

# Validate selection only, no agent, no writes:
pwsh -File scripts\ci\sentry-daily-triage\run.ps1 -SkipAgent

# Full dry run (selects + plans, but no Linear/PR writes):
pwsh -File scripts\ci\sentry-daily-triage\run.ps1 -DryRun

# Run the real thing now:
Start-ScheduledTask -TaskName RakletSentryDailyTriage

# Remove the task:
pwsh -File scripts\ci\sentry-daily-triage\register-task.ps1 -Unregister

Logs land in .tmp-ci-logs/sentry-triage/run-<timestamp>.log (and the candidate worklist alongside).

Requirements & assumptions

  • Runs as the logged-on user (Interactive logon, no stored password) because it needs that profile's credentials: Claude config + claude.ai Linear MCP OAuth token, gh CLI auth (keyring), and git. A SYSTEM/session-0 run would not have these. The dev VM is normally logged on; StartWhenAvailable catches up a missed run.
  • Sentry auth comes from SENTRY_AUTH_TOKEN/SENTRY_TOKEN via load-agent-env.ps1.
  • Linear writes go through the claude.ai Linear MCP (verified working under headless claude -p). If that MCP ever stops authenticating headless, add a LINEAR_API_KEY to .claude/settings.local.json env and switch the runbook's Linear steps to the GraphQL API.