Browse-tests — coverage & environments

Where the Selenium → browse-tests migration stands, what each environment validates, and what's tested today. Last updated 2026-06-04.

At a glance

104
Browse tests on master
15/15
Batches merged (100%)
3
Envs wired (laptop + ci-vm-1 + .raklet.net)
422
Original Selenium rows

Migration progress

15 merged — migration complete
EM-1 retired
Merged (15) Retired (EM-1 Outlook email)

All 15 planned batches landed by 2026-05-22. The work since then is stabilization on .raklet.net: cluster fixes for selector / timing drift, test-org fixture cleanup (RAK-367), and unblocking the last sandbox-credential-gated rows. Live status — including which tests are currently red on the matrix dispatch — lives in browse-test-failures-status.md.

Batches

BatchScopeStatusPR
RO-1ContactTests (read-only)merged#13780
RO-2LoginTests + UrlRedirect + Reports + AppStoremerged#13757
RO-3SettingsTests (read-only)merged#13761
RO-4SocialTests + EventsTests (read-only)merged#13766
AM-1ContactTests (mutation)merged#13803
AM-2ApplicationFormTests (admin mutation)merged#13759
AM-3SettingsTests (mutation)merged#13773
AM-4SocialTests (mutation)merged#13778
AM-5EventsTests (admin mutation)merged#13769
AM-6JobsTests + MessageModule + Directorymerged#13779
PM-1ApplicationFormTests (payment)merged#13782
PM-2BillingTests + ContactTests (payment)merged#13781
PM-3FundraisingTestsmerged#13785
PM-4EventsTests (payment)merged#13784
PM-5MembershipTestsmerged#13804
EM-1OutlookEmailTests (16 rows)retired#13775

Coverage by feature area

What user journeys we actually exercise today, organized by feature rather than batch. Click any area to see the individual test files and what's not yet covered.

Login & Auth — 12 testssmoke + admin + payment regression
Most login-fixture tests target .raklet.net only (their fixture users don't exist on engineer laptops or ci-vm-1). On other envs they SKIP cleanly. Tracked under #13776.
  • smoke/01-login.ps1 — admin login flow + manager shell all envs
  • admin-regression/13-login-forgot-password-invalid-email.ps1 — forgot password with unknown email stays on flow
  • admin-regression/11-login-invalid-password-member.ps1 — invalid password as a member user stays on login .raklet.net only
  • admin-regression/12-login-invalid-password-custom-domain.ps1 — invalid password on a custom-domain org .raklet.net only
  • admin-regression/14-login-custom-domain.ps1 — custom-domain login reaches the social network shell .raklet.net only
  • admin-regression/15-login-public-pages-session.ps1 — Stripe org public events/donations after manager login .raklet.net only
  • admin-regression/16-login-select-organisation.ps1 — multi-org login picks via permalink .raklet.net only
  • admin-regression/17-url-redirect-marketing.ps1 — marketing hosts redirect to www .raklet.net only
  • admin-regression/18-login-custom-domain-permalink.ps1 — custom-domain manager via central permalink .raklet.net only
  • payment-regression/02-login-invalid-password-iyzico.ps1 — invalid password on Iyzico org .raklet.net only
  • payment-regression/03-login-stripe-custom-destination.ps1 — Stripe Custom Destination manager login .raklet.net only
  • payment-regression/05-login-stripe-custom-destination-permalink.ps1 — same via central permalink .raklet.net only
Gaps: No 2FA-flow tests in browse yet (the original 14 Outlook 2FA rows were disabled in #13775 because they only trigger sends, never read mailboxes). 2FA coverage would need a real inbox-polling fixture as a deliberate scope upgrade.
Page load & manager shell — 11 testssmoke suite (gating)
  • smoke/01-login.ps1 — login flow + manager shell visible
  • smoke/02-contacts.ps1 — contacts dashboard loads
  • smoke/03-membership-plans.ps1 — membership plans page loads
  • smoke/04-reports.ps1 — reports overview loads
  • smoke/05-social-network-page-loads.ps1 — social network module pages
  • smoke/06-manager-module-page-loads.ps1 — manager module pages (cities, countries, etc.)
  • smoke/07-portal-account-page-loads.ps1 — portal-side account pages (profile, payments, tickets)
  • smoke/08-settings-page-loads.ps1 — settings module pages
  • smoke/09-module-dashboard-page-loads.ps1 — per-module dashboards
  • smoke/10-help-billing-page-loads.ps1 — help center + billing
  • smoke/11-apply-signup-no-razor-leak.ps1 — application/signup flow doesn't leak Razor markup
This suite is the pr-ci gate. If any of these fails, the PR is red.
Application Forms — 15 testsadmin mutation + payment
  • admin-regression/19-application-form-update-sections.ps1 — update form sections (AM-2)
  • admin-regression/20-application-form-add-remove-fields.ps1 — add/remove form fields (AM-2)
  • admin-regression/21-application-form-fields-requiredness.ps1 — field requiredness toggles (AM-2)
  • admin-regression/22-application-form-create-custom-field.ps1 — create custom field on form (AM-2)
  • payment-regression/06..08-application-form-plan-stripe-*.ps1 — plan payments via Stripe Custom / Standard Destination / Standard Direct (PM-1)
  • payment-regression/09..11-application-form-fee-stripe-*.ps1 — fee payments via three Stripe modes (PM-1)
  • payment-regression/12..14-application-form-fee-and-plan-stripe-*.ps1 — fee + plan combined via three Stripe modes (PM-1)
  • payment-regression/15-application-form-invalid-email-stripe-standard-destination.ps1 — invalid email rejected (PM-1)
  • payment-regression/16-application-form-auto-approve-quick-register.ps1 — auto-approve after QuickRegister (PM-1)
Gaps: Iyzico-connect application-form rows are disabled in Selenium (carried forward as disabled). No application-form admin-rejection-flow yet. No payment-failure recovery test beyond invalid-email.
Settings (Core, Fields, Roles) — 7 testsadmin mutation + read-only
  • admin-regression/06-login-settings-read-only.ps1 — Login settings preview toggles update without saving
  • admin-regression/07-fields-read-only.ps1 — Custom field admins-only visibility
  • admin-regression/09-exports-read-only.ps1 — Exports page shell + dialog
  • admin-regression/11-fields-subscriber-visibility.ps1 — Subscribers-only field visibility (RO-3) .raklet.net role-fixture only
  • admin-regression/23-settings-core-mutation.ps1 — API tokens + Custom CSS + Exports + Navigation menu CRUD (AM-3)
  • admin-regression/24-settings-fields-mutation.ps1 — Custom-field CRUD across 7 types (system + single/multi-line + checkbox + date + dropdown + file + radio) (AM-3)
  • admin-regression/25-settings-roles-tags-mutation.ps1 — Role + Tag CRUD (AM-3)
Gaps: GeneralSettings (org identity/branding) deferred — mutates shared org-level state. Invite manager flow deferred (needs mailbox fixture). Stripe/Iyzico connect setup flow not covered.
Contacts — 10 testsread-only + admin mutation + payment
  • admin-regression/26-contacts-dashboard-count.ps1 — total contact count from dashboard (RO-1)
  • admin-regression/27-contacts-filter-email-contains.ps1 — filter contacts by email substring (RO-1)
  • admin-regression/26-contacts-add-update-mutation.ps1 — add/update contact lifecycle (AM-1)
  • admin-regression/28-contact-details-mutation.ps1 — detail-page mutations: tags, custom fields, settings (AM-1)
  • payment-regression/09-contact-subscriptions-crud.ps1 — subscription CRUD on contact detail (PM-2)
  • payment-regression/10-contact-scheduled-payments-crud.ps1 — scheduled payment CRUD (PM-2)
  • payment-regression/11-contact-payments-crud.ps1 — one-off payment CRUD (PM-2)
  • payment-regression/12-contact-subscription-reminders.ps1 — subscription reminder generation (PM-2)
  • payment-regression/13-contact-scheduled-donations-crud.ps1 — scheduled donation CRUD (PM-2)
  • payment-regression/14-contact-donations-crud.ps1 — one-off donation CRUD (PM-2)
Gaps: Contact merge/split, contact import flow, contact debts not yet covered. AM-1 cleanup-prefix work tracked under RAK-367 (test-org data accumulation on .raklet.net).
Billing — 3 testspayment
  • payment-regression/06-billing-update-invoice-details.ps1 — update billing invoice email + details (PM-2)
  • payment-regression/07-billing-upgrade-plans-monthly-quickregister.ps1 — upgrade from free to monthly plan (PM-2) SKIP pending QuickRegister helper port
  • payment-regression/08-billing-upgrade-plans-annual-quickregister.ps1 — upgrade from free to annual plan (PM-2) SKIP pending QuickRegister helper port
Gaps: PayoutsAndPayoutDetails disabled in Selenium (not migrated). Billing-plan downgrade not covered. QuickRegister onboarding flow needs to be ported (two SKIPs above depend on it).
Fundraising / Donations — 8 testspayment
  • payment-regression/06-fundraising-campaign-crud.ps1 — campaign add/update/delete (PM-3)
  • payment-regression/07-fundraising-admin-donation-crud.ps1 — admin-side donation CRUD (PM-3)
  • payment-regression/08-fundraising-public-donation-stripe.ps1 — public donations via 3 Stripe Connect modes + invalid card (PM-3)
  • payment-regression/09-fundraising-public-donation-iyzico.ps1 — public donations via Iyzico + fallback + invalid card (PM-3)
  • payment-regression/10-fundraising-private-donation-stripe.ps1 — private donations via 3 Stripe Connect modes (PM-3)
  • payment-regression/11-fundraising-private-donation-iyzico.ps1 — private donations via Iyzico + variants (PM-3)
  • payment-regression/12-fundraising-scheduled-donations.ps1 — scheduled donations (one-time / monthly / yearly) (PM-3)
  • payment-regression/13-fundraising-settings-update.ps1 — fundraising-page settings update (PM-3)
Gaps: The migration covers one Connect mode per file against the wired sandbox sub-org; true 3-mode CI parity needs the fixture contract to grow to BROWSE_TEST_STRIPE_CUSTOM_ACCOUNT / STANDARD_DEST_ACCOUNT / STANDARD_DIRECT_ACCOUNT. Refund flows not covered.
Events — 8 testsadmin mutation + payment
  • admin-regression/23-events-add-venue-event.ps1 — create a venue event (AM-5)
  • admin-regression/24-events-add-online-event.ps1 — create an online event (AM-5)
  • admin-regression/25-events-clone-online-event.ps1 — clone an online event (AM-5)
  • admin-regression/26-events-clone-venue-event.ps1 — clone a venue event (AM-5)
  • payment-regression/06-events-stripe-ticket-type-lifecycle.ps1 — ticket type create/update/delete (PM-4)
  • payment-regression/07-events-stripe-coupon-add-delete.ps1 — coupon add + delete (PM-4)
  • payment-regression/08-events-stripe-limited-ticket-purchase.ps1 — purchase a limited-quantity ticket (PM-4)
  • payment-regression/09-events-stripe-limitless-and-at-door.ps1 — limitless ticket + at-the-door purchase (PM-4)
Gaps: Event publish/unpublish lifecycle not in AM-5 (focused on create/clone). Iyzico event payments not covered (disabled in Selenium). Event refunds, event RSVP-without-ticket flows not covered.
Social Network — 6 testsread-only + admin mutation
  • admin-regression/19-social-dashboard-read-only.ps1 — dashboard setup-card + stat cards (RO-4; About Us link removed per #13777, feature retired)
  • admin-regression/20-social-content-read-only.ps1 — boards / pages / posts create-form shells (RO-4)
  • admin-regression/21-social-career-read-only.ps1 — jobs + jobseeker portal pages (skips when feature disabled for org) (RO-4)
  • admin-regression/26-social-career-mutation.ps1 — jobseeker career settings mutation (AM-4)
  • admin-regression/27-social-content-mutation.ps1 — boards / pages / posts CRUD (AM-4)
  • admin-regression/28-social-settings-mutation.ps1 — social-network settings mutation (AM-4)
Gaps: Social-comments admin flow not yet covered. Subscription notification flows not covered. Old About Us feature removed.
Reports — 4 testssmoke + payment
  • smoke/04-reports.ps1 — reports overview page loads
  • payment-regression/01-reports-payments-read-only.ps1 — reports/payments filter + table shell
  • payment-regression/02-reports-payments-export-read-only.ps1 — export-button affordance against Stripe Custom Destination (RO-3) .raklet.net Stripe sandbox only
  • payment-regression/04-reports-payments-export.ps1 — full export flow click-through
Gaps: Reports/refunds, reports/action-logs migrated to read-only smoke but no deep filter combo coverage. Action logs export disabled in Selenium since Release-578.
Jobs / Careers — 5 testsadmin mutation
  • admin-regression/29-jobs-pending-create-update.ps1 — pending job create + update (AM-6)
  • admin-regression/30-jobs-pending-status.ps1 — pending job status transitions (AM-6)
  • admin-regression/31-jobs-published-approve.ps1 — approve a published job (AM-6)
  • admin-regression/32-jobs-form-requirements.ps1 — required-field validation (AM-6)
  • admin-regression/33-jobs-settings.ps1 — jobs module settings (AM-6)
Gaps: Public job-apply flow not covered. Application reject + comment flows not covered.
Messages / Email — 4 testsread-only + admin mutation
  • admin-regression/05-messages-automated-template-read-only.ps1 — automated email template page (RO)
  • admin-regression/34-messages-email-draft-lifecycle.ps1 — draft create/update/send (AM-6)
  • admin-regression/35-messages-template.ps1 — custom template CRUD (AM-6)
  • admin-regression/36-messages-sender-identities.ps1 — sender-identity CRUD (AM-6)
Gaps: Inbox-content verification not covered (no real mailbox-polling fixture — would be a deliberate scope upgrade alongside the retired Outlook 2FA rows). UpdateEmail row inside AM-6 stays red on .raklet.net until OrganisationAccountActivation=Completed is seeded for the test org — tracked under RAK-368.
Directory — 4 testsread-only + admin mutation
  • admin-regression/10-directory-read-only.ps1 — directory dashboard (RO)
  • admin-regression/26-directory-create-update-delete.ps1 — directory CRUD (AM-6)
  • admin-regression/27-directory-search.ps1 — directory search (AM-6)
  • admin-regression/28-directory-form-requirements.ps1 — directory form required-field validation (AM-6)
Gaps: Public-directory portal view not covered. Directory test-org rows still need broader-prefix cleanup helpers (RAK-367).
Membership — 2 testspayment
  • payment-regression/17-membership-plan-crud.ps1 — membership plan CRUD (PM-5)
  • payment-regression/18-membership-coupon-crud.ps1 — membership coupon CRUD (PM-5)
Gaps: Membership purchase / renewal / downgrade end-to-end flows still uncovered — PM-5 landed the CRUD admin surface (#13804) but the public purchase paths against Stripe + Iyzico stayed deferred.
Other modules (App Store, SMS, Action Logs, Help Center) — 5 testsread-only
  • admin-regression/01-action-logs-read-only.ps1 — action-logs module page
  • admin-regression/02-help-center-navigation.ps1 — help-center module navigation
  • admin-regression/03-sms-read-only.ps1 — SMS module overview
  • admin-regression/04-app-store-zapier-read-only.ps1 — Zapier app store integration card visible
  • admin-regression/08-app-store-core-read-only.ps1 — core app store catalog page
Gaps: No SMS send flow, no app-store install/configure flows, no Zapier OAuth handshake. App-store + SMS mutation flows stayed out of scope for the Selenium retirement.
Onboarding — 1 testpayment
  • payment-regression/10-onboarding-activate-stripe-payment-account.ps1 — activate Stripe payment account during new-org onboarding (PM-4)
Gaps: The 4 other OnboardingTests rows (UpdateContactInfo / DataImportContacts / AddContact / VisitTheSocialNetwork) are now covered by AM-1 contact-mutation tests in the Contacts section.
Retired / out of scopenever to migrate
  • Outlook email tests (16 rows) — the Selenium helpers never actually read mailboxes (they just trigger sends and WaitForSeconds(2)). Moved to backlog in #13775. Re-enable as a real batch only if/when we add inbox-content verification as a deliberate scope upgrade.
  • About Us social feature (2 rows) — feature retired from the product (confirmed 2026-05-19). Disabled in checklist via #13777.
  • Marketing host redirect tests (2 rows) — hit production marketing URLs from CI; turned off in #13783.
  • ~100 other rows marked [disabled] in the Selenium source, mostly because they referenced retired features, flaky-environment-only paths, or never worked.

How a change gets validated

Three environments. Each runs a different subset of the suite depending on what fixtures are wired. Tests that need a fixture not present in the environment SKIP cleanly — they never falsely fail.

1. Engineer’s laptop .\scripts\dev\pr-ready-check.ps1 target: gercek.raklet.org (local IIS) creds: browse-tests/local.settings.json manual pre-flight git push → open PR mark ready when confident 2. ci-vm-1 (pr-ci.yml) target: gercek.raklet.org on the VM trigger: every PR (smoke + admin + payment) creds: same local.settings.json (seeded) automatic gate per PR merge to master approving review + green CI deploy master → .raklet.net AzDO release pipeline 31 3. .raklet.net (browse-ui-tests.yml) AzDO pipeline 31 stage 286 dispatches on release target: shared .raklet.net test environment runners: r2/r3/r4 matrix; creds: 13 repo secrets post-deploy validation, auto-dispatched red → iterate Each environment runs whatever the fixtures support; the rest SKIP cleanly. See the matrix below.

Local pre-flight

Your laptop, your branch
.\scripts\dev\pr-ready-check.ps1
  • Hits gercek.raklet.org on local IIS
  • Reads browse-tests/local.settings.json (gitignored)
  • Tests your branch's code on your local org
  • Locks the machine ~20 min
Gate: nothing — just a sanity check. browse-ui-tests/local commit status posted to the PR.

CI VM (ci-vm-1)

Self-hosted, per-PR
.github/workflows/pr-ci.yml
  • Hits gercek.raklet.org on ci-vm-1's IIS
  • Runs smoke + admin-regression + payment-regression
  • smoke is the gate; the other two are non-gating
  • Posts inline screenshots on failure
Gate: smoke must pass. admin+payment surface failures but don't block.

.raklet.net

Self-hosted matrix, AzDO-dispatched
Actions → Browse UI Tests → Run workflow
  • Hits shared .raklet.net test environment
  • Self-hosted runners r2/r3/r4 in a 3-way matrix; ci-vm-1 stays pinned to pr-ci
  • Pulls creds from repo secrets (13 set) + 6+ sub-org sub-profiles in local.settings.json
  • Sets BROWSE_TEST_FIXTURE_PROFILE=raklet-net so role/payment-fixture tests actually exercise their paths
Gate: AzDO pipeline 31 stage 286 inlines dispatch-browse-ui-tests.ps1 on every Test → Prod release. Manual dispatch still available via the Actions tab.

Where does each test type run?

The answer is “wherever its fixture is wired.” All tests follow the same SKIP-on-missing-fixture contract, so they never falsely fail when a fixture isn't present.

Test typeLocal laptopci-vm-1 pr-ci.raklet.net dispatch
Page-load & admin-login (smoke) Yes Yes (gate) Yes
Role-fixture (member / subscriber / admins-only field) Optional
if engineer has fixture		 Optional
if seeded on VM		 Yes
Payment sandbox (Stripe / Iyzico) Optional Optional Yes
.raklet.net-specific data (custom-domain, stripe-custom-destination, etc.) SKIP SKIP Yes
Mutation cleanup-proof (run twice, no collision) Author attests No (single run) No (single run)

Do we expect every test to run in every environment?

No — by design. .raklet.net has the richest fixture data (seeded users, multiple Stripe Connect modes, real campaigns), so it runs the most. Laptop runs are enough for “does my change break the basic shell?”. The SKIP-on-missing-fixture contract keeps the suite honest: a run that skips 10 tests is fine; the log lists which and why.

What happens after deploy to .raklet.net?

Today:

  1. PR merges to master.
  2. AzDO release pipeline 31 deploys master to .raklet.net.
  3. Pipeline 31 stage 286 inlines dispatch-browse-ui-tests.ps1, which fires the browse-ui-tests.yml matrix on the self-hosted runners (r2/r3/r4).
  4. Matrix results land back on the release. Manual Actions → Browse UI Tests → Run workflow is still available for one-off dispatches.

The cost-out: Selenium ran 4–5h per release and always partial-failed. The browse equivalent runs in 30–60 min and is trusted. Once we're confident in three consecutive green release cycles, the AzDO Selenium stages get removed and Raklet.UI.Test.{1..5} deleted (3-week rollout plan tracked in .agents/context/ci-runners.md).

What you do next

Generated 2026-06-04 · rakletadmin/rakletv3 · see also: browse-test-failures-status.md, browse-tests-fixture-cleanup.md, payment-sandbox-fixtures.md, browser-test-migration-prompt.md, browser-test-migration-checklist.md, Linear card RAK-338.