Skip to content

Remaining Selenium Migration Plan

Summary

Start by creating a tracked migration checklist before migrating more tests. The current browse smoke PR should stay focused on smoke coverage, get merged, and then be used as the first post-deploy or manually triggered browser-test gate.

The remaining Selenium tests will be migrated in controlled batches with explicit environment rules. Tests can run locally on .raklet.org or in test on .raklet.net, but mutating tests must declare which environment they are safe for and how their data is isolated and cleaned up.

Key Changes

  • Add docs/testing/browser-test-migration-checklist.md as the source of truth before writing more tests.
  • Track each Selenium test with: project/class/test name, feature area, current status, target suite, allowed environments, mutation level, fixture strategy, cleanup strategy, and migration checkbox.
  • Extend the browse runner with suite selection. This branch now supports smoke, admin-regression, and payment-regression; keep portal-regression, email-regression, and all-safe as future expansion once the fixture and cleanup rules are proven.
  • Keep the existing smoke PR mergeable first: do not pretend that a GitHub PR runner validates branch behavior by opening shared .raklet.net. Keep the real browse smoke workflow for manual or post-deploy execution, and do not add mutating suites to required CI until the checklist/data strategy is complete.

Migration Order

  1. Inventory and checklist
  2. Build the full checklist from Raklet.UI.Test.1 through Raklet.UI.Test.5.
  3. Mark already migrated page-load coverage as checked.
  4. Mark commented Selenium tests separately; do not migrate them automatically.

  5. Group remaining tests by feature and mutation risk.

  6. Read-only regression

  7. Migrate remaining safe checks first: action-log filters, report filters, help-center navigation, app-store read-only pages, SMS/message read-only views, and directory read-only checks.
  8. Allow these to run in local or test.

  9. Put fast, stable checks in smoke; otherwise put them in admin-regression.

  10. Admin/portal mutating tests

  11. Migrate contacts, fields, roles, tags, navigation, directory, social posts/pages/boards/comments, jobs, fundraising settings, and membership settings.
  12. Allow both local and test only when the checklist defines a unique run prefix, created records, cleanup path, and fallback cleanup/manual recovery note.

  13. If cleanup is unreliable, mark the test local-only until fixed.

  14. Payments, donations, tickets, membership billing

  15. Migrate into payment-regression, not default smoke.
  16. Require explicit test payment fixtures and environment variables.
  17. Prefer .raklet.net test environment for payment-provider behavior when sandbox accounts are stable.

  18. Keep local support only for flows that can run against locally configured sandbox services.

  19. Email, Outlook, onboarding

  20. Migrate into email-regression or manual regression.
  21. Require explicit secrets and mailbox ownership.
  22. Do not make these default PR blockers until runtime and flakiness are known.

CI/CD Plan

  • Current smoke PR:
  • merge after the stable PR-safe automated checks stay green;
  • keep GitHub Actions browse-ui-tests.yml honest on pull requests;
  • use workflow_dispatch for real shared-environment browse smoke.
  • Next iteration:
  • use manual workflow_dispatch for admin-regression and payment-regression on .raklet.net;
  • decide which regression suites are stable enough to graduate from manual runs into required post-deploy CI;
  • allow additional suites on .raklet.org or .raklet.net only after their checklist metadata and cleanup rules are proven.
  • Azure/test deployment:
  • add post-deploy smoke execution against .raklet.net;
  • do not run mutating suites automatically in deployment until each test has checklist-approved cleanup.

Test Plan

  • Checklist PR:
  • no behavior changes;
  • verify every Selenium [TestMethod] is represented once.
  • Smoke PR:
  • run browse-tests/run.ps1 -TestDomain ".raklet.org" locally before marking the PR ready;
  • use browse-ui-tests.yml workflow_dispatch for shared .raklet.net runs;
  • keep Tests / js and Tests / dotnet as the real required PR checks.
  • Regression migrations:
  • each batch must pass locally or on test according to its checklist environment;
  • each mutating test must prove cleanup by rerunning the same test twice;
  • each suite must leave the smoke suite green after it runs.

Assumptions

  • The current smoke migration is mergeable as the first manually triggered or post-deploy browser-test layer.
  • Mutating tests are allowed in local and test environments, but only with explicit per-test environment approval in the checklist.
  • .raklet.net is acceptable for controlled test-environment mutation when data ownership and cleanup are defined.
  • No disabled/commented Selenium test is migrated without an explicit keep/retire decision in the checklist.