Skip to content

Sentry environment tagging (operator guide)

How the environment tag on Sentry events is decided, and the one setting operators may need to touch. Background: ENG-194.

Why this exists

Every Sentry event carries an environment tag (prod, test, local, …). The production error burndown (ENG-165) only works if environment:prod means real customer traffic. Historically any non-production host that carried a leaked or inherited prod config — CI runners, dev laptops, dev-VM branch previews — reported its events under environment:prod and polluted the board (e.g. ~91% of one top "prod" issue was CI browse-test traffic).

How the environment is resolved

The label still comes from the Environment app setting, but a host guard now sits in front of it (Services/ExceptionLogService.cs, ResolveSentryEnvironment / ResolveSentryEnvironmentCore).

The guard is strictly one-way: it only ever downgrades prod → non-prod, never the reverse, so a genuine production error can never be hidden.

A host with Environment=prod keeps reporting under prod only when it is a real Azure host. It is treated as non-production (downgraded to ci or local) when either:

  1. WEBSITE_SITE_NAME is unset. Every Azure App Service / WebJob instance sets this automatically; plain-IIS CI runners, dev laptops, and dev-VM previews never do. This is the primary signal and needs no configuration.
  2. The machine name is on an explicit denylist (below).

test / local / any non-prod label passes through untouched. A canonical prod label is always emitted for kept-prod hosts, so prod or PROD in config cannot create a duplicate environment.

The one operator setting: Sentry:NonProdMachineNames

A comma-separated app setting that adds machine names to the non-production denylist, so a new CI/dev host can be reclassified without a code change. The built-in baseline already contains CI-VM-1.

<add key="Sentry:NonProdMachineNames" value="CI-VM-2, PREVIEW-BOX-A" />

Rules:

  • Full machine names only. Matching is exact and case-insensitive — not substring. This is deliberate: a substring like ci or sdwk would match a real Azure prod worker name such as wn1sdwk00020L and silently downgrade it, hiding real prod errors. List the complete COMPUTERNAME of each host.
  • You usually do not need this. A plain-IIS non-prod host is already downgraded by signal 1 (no WEBSITE_SITE_NAME). Reach for this setting only to downgrade a host that is on Azure (so it has WEBSITE_SITE_NAME) but should not report as production — e.g. an Azure-hosted preview slot.

Verifying after a deploy

The guard assumes every real prod worker exposes WEBSITE_SITE_NAME. To confirm, sample one event per prod app (api / v3 / Backend WebJobs) in Sentry and check the server_name tag resolves to an Azure worker reporting environment:prod. If a prod app's events show up under local/ci, that worker is missing WEBSITE_SITE_NAME — fix the host, do not weaken the guard.