Implemented (issue #773):

Eliminated the actual repeated-download source instead of centralizing the build (2026-07-28):

Implemented (issue #2145, 2026-08-20). The "not warranted yet" note this replaces was written when visual-tests ran ~6 min. It became warranted.

What the measurements actually said (56 dotnet.yml runs over 15.5 h, 2026-08-20):

This was never a "CI is slow" problem - 471 integration tests in 4 min 19 s is a good number. It was one job. The 20 cancelled runs are part of the finding rather than noise: run_attempt > 1 was 0 across 210 runs despite 15 failures, so contributors were not re-running a red job, they were pushing over it and letting cancel-in-progress clean up. That is what a gate nobody trusts looks like.

Root cause. 542 tests on a 4-vCPU ubuntu-latest runner, with AssemblyParallelLimit.cs capping concurrency at ProcessorCount - 2 (= 2), is ~271 sequential slots. 1165 s / 271 ~= 4.3 s per slot, which matches the measured median. The cap was not a mistake - it was tightened twice (ProcessorCount → -1 in the 2026-07-28 note above, then -2) precisely because CPU contention between Playwright/axe-core sessions and the Aspire stack they all call into was causing timeouts. But reducing parallelism buys reliability with runtime, and by 542 tests that trade had gone net-negative: it bought 19 minutes and roughly the same failure rate.

Why B (sharding) and not A (a bigger runner). A would have been a one-line change - runs-on: ubuntu-latest-8-cores and ProcessorCount - 2 scales itself to 6 - and it addresses the measured cause head-on. It was rejected on cost, not on merit: only the default ubuntu-latest/windows-latest/macos-latest tier is free and unlimited on a public repository, and every larger-runner tier is billed per minute regardless of repository visibility (GitHub Actions runner pricing). That is incompatible with the minimal-budget constraint in this record’s own Business Impact section. Sharding buys the same parallelism by adding runners rather than cores, and standard runners stay free, so it is $0 - the same reasoning #773 used.

Measured fixed per-shard cost, which is what decides how many shards are worth having (run 32337102365, visual-tests, 2026-08-20):

So roughly 2 min 45 s per shard is fixed and does not divide. At 4 shards that is ~3 min 50 s of test work each, for a ~6.5-7 min job and a ~9.5 min critical path once build (2 min 38 s) is added - under the 10 min target. Six shards would reach ~8 min end to end, and past that the fixed cost dominates entirely, so 4 is the point where the returns flatten rather than an arbitrary pick. The Aspire boot figure above had to be reconstructed from step timestamps and TUnit’s [slow] lines; AspireFixture now times its own boot and both workflows print it (their "Report Aspire stack startup cost" steps), so the next revision of this table is a direct reading rather than an inference.

How the shard split is expressed. TUnit 1.65.31 has no native sharding flag - its documented flag set is --list-tests/--treenode-filter/--maximum-parallel-tests/etc., with nothing that splits a run across machines - so a shard is a Microsoft.Testing.Platform --treenode-filter over class names, which was #2145’s stated fallback. The split is computed by .github/scripts/visual-test-shard.sh on every run rather than checked in: a hand-written list would silently stop running any newly added test class, since a class in no shard’s filter is simply never selected. Classes are packed longest-first into whichever shard is lightest, because sizes are extremely uneven - AccessibilityTests alone is 90 of the 542 cases while most classes are 1-5 - which lands the four shards at 136/136/135/135 cases. The script fails loudly if a file’s class name ever stops matching its file name, since that is the one way a class could fall out of every filter.

AssemblyParallelLimit was deliberately left alone. Each shard is still a 4-vCPU runner, so concurrency stays 2 and per-instant CPU contention per stack is unchanged - the win here is four stacks, not a busier one. Changing the parallel limit in the same commit would have confounded the one measurement this change exists to produce.

First sharded run (n=1, 2026-08-20, run 32345414446):

The Aspire boot instrumented above reported 63.0 s on shard 4, confirming the ~1 min the shard count was chosen from. Shard sizes came out at 136/136/135/135 test cases.

Wall-clock is therefore just over the < 10 min target rather than under it, and the residual is almost entirely fixed cost: ~2 min 45 s of build plus ~1 min 45 s of per-shard setup plus ~1 min of Aspire boot is ~5 min 30 s before a single test runs. More shards barely move that; the levers that would are the E2E rebalance (option C, issue #2148) and the build-artifact round trip this document’s 2026-07-28 note already flagged as re-measurable.

One failure in that run, on shard 4: JwtAudienceTests.Administration_AuthenticatedAdmin_LoadsOrganizationsWithoutAuthError, a 45 s timeout on the post-submit leg of a real Keycloak login. That is the contention flake this document has tracked since 2026-07-28, but sharding changes its distribution in a way worth recording: every shard now has a "first concurrent batch" against a cold Keycloak, where one 542-test session used to have exactly one, statistically absorbed by AccessibilityTests’ 90 cases - which is why the only `[Retry(2)] in the suite sits on exactly such a call site, with a comment naming that same cause. AuthHelper.LoginAsync’s own comment claimed callers carried that retry; only 1 of the 15 classes driving a real login actually did. The retry now lives inside the helper (scoped to the identity round trip, never to a product assertion, so #1321’s distinction holds) and `AspireFixture renders the login page once during boot so the custom Freemarker theme compiles inside the initializer’s budget rather than inside a test’s.

Second run, after the login fix (2026-08-20, run 32346711001): all four shards green, visual-tests (4) back to 135 tests / 0 failures, shards 6 min 41 s - 7 min 43 s, Aspire boot 64.9 s, critical path ~10 min 33 s. The same shape held on the release path (v1.0.0-rc.362, run 32347940443), which is the only thing that exercises `publish.yml’s copy of the matrix.

Still to re-measure. The above is n=2. Re-measure at n=20+ before treating ~10 min as the new figure, and treat the failure rate as the primary success criterion, not the wall-clock - runtime was only ever half of what made this debt expensive.

Mutation-score baseline recorded (issue #2147, 2026-08-20). Deliberately scoped to producing the number: no test changes, no threshold gate.

Why this and not line coverage. Line coverage cannot answer the question #2148 raises. A redundant end-to-end test and a load-bearing one look identical to it: delete either, and the lines they touched are still executed by something else, so the percentage barely moves. Mutation testing gives a falsifiable condition instead - record the score now, re-run it after each migration wave, and if the score does not drop, the coverage that was removed was redundant. Without a recorded number first, #2148 is unfalsifiable and should not start.

What was measured. Stryker.NET 4.16.0 mutating backend/src/Domain and backend/src/Application, one run per source project, both driven by Application.UnitTests (1017 tests, no Docker). .NET SDK 10.0.400, mutation-level: Standard, concurrency: 1 (see "The concurrency trap" below), on a 4-vCPU machine. Measured at commit c4a143b (ci: shard visual-tests across four runners and cut fixed waits (#2146)). The configuration is checked in at backend/tests/Application.UnitTests/stryker-config.json, so the run is reproducible by anyone.

"Generated" minus "scored" is 460 mutants Stryker excludes from the score itself: 333 ignored (blocks already covered), 126 that did not compile, 1 runtime error. The score is (killed + timeout) / (killed + timeout + survived + no coverage); there was exactly one timeout across both runs.

The concurrency trap - the most important finding here, because it decides whether the number is usable at all. The first pass ran at Stryker’s suggested concurrency: 2 and produced 71.60 % / 73.47 %. Repeating it at the same commit produced 70.68 % / 72.85 %. The mutant sets were identical both times, but individual verdicts churned: 47 of 872 Domain mutants and 99 of 1818 Application mutants changed status between two runs of unchanged code. Disabling Stryker’s bail and mix-mutants optimisations did not help (34 flips remained, at no cost in runtime), which ruled out mutant batching as the cause.

Dropping to concurrency: 1 did. Two runs per project at concurrency 1 were bit-for-bit identical - zero flips out of 872 and out of 1818. The determinism survives a dependency change too: re-running both projects after merging `main’s TUnit 1.65.31 → 1.65.38 bump (#2144) reproduced every single verdict, 0 flips out of 2690 mutants and the same 66.75 % / 68.49 %. So the number tracks the test suite, not the machine or the engine underneath it. Comparing the concurrency-2 runs against that deterministic baseline shows the error is one-directional:

Every parallel run invents kills and never loses one, and it invents a different set each time: across the four Domain runs the four spurious sets intersected in exactly zero mutants (75 distinct mutants were spuriously killed at least once). So concurrency inflates the score by roughly 5 points and jitters it by roughly 1. Stryker’s Microsoft.Testing.Platform runner is marked preview and warns as much on every run; this looks like its per-mutant attribution losing track of which test host reported which failure. concurrency: 1 is therefore load-bearing, not a performance setting - the whole value of this number is that a movement in it means something, and at concurrency 2 a 1-point drop is indistinguishable from running the tool twice.

Wall-clock: cheap enough to run per migration wave. 11 min 40 s for both projects sequentially, and the two projects are independent, so .github/workflows/mutation-tests.yml runs them as a matrix on separate runners - CI wall-clock is the longer leg (~8 min) plus job setup, not the sum. That is affordable several times per wave. It is still ~150x the fast-tests job whose 1017 tests it reuses (3 s), which is why it stays off the per-PR path.

Decided: it runs in CI, but manually and report-only. mutation-tests.yml is workflow_dispatch only, so it is never a required check and cannot block a merge, and thresholds.break is pinned to 0 so a low score exits 0 regardless. Same posture as the report-only coverage summary added to fast-tests in #1327: visibility is the point. Do not add a break threshold - a number whose whole purpose is to be free to move cannot double as a gate.

What 67.89 % is actually made of, because the headline is misleading read alone:

Domain and Application land within two points of each other on every one of those four slices (66.75/68.49, 80.39/81.52, 74.67/74.54, 85.89/87.37), which is a good sign the number measures a property of the test suite rather than of one layer.

The string survivors are not noise, and this is the single most actionable thing the run surfaced. Splitting the string-literal mutants by what the literal actually is:

Error codes are essentially unasserted: 6 kills out of 224. The cause is ResultFailureException(Error error) : Exception(error.Description) - assertions written as ThrowAsync<ResultFailureException>().WithMessage("not found") pin the human-readable prose and leave Error.Code untouched, even though Error.Code is what ResultFailureExceptionHandler puts on the wire as errorCode and what the frontend branches on. The contract that can be reworded freely is tested; the contract that cannot be changed without breaking clients is not. Those 174 survivors are 34 % of every survivor in both projects, so closing that one pattern alone would move the combined score from 67.9 % to roughly 75.7 %. Out of scope for #2147 (no test changes), worth its own issue.

How to read the number, and what a "good" score here would be. Stryker’s own built-in thresholds (high 80, low 60) are generic defaults, and a single global target is the wrong instrument for this codebase - 64 % of the surviving mutants are string swaps, whose kill rate says more about assertion style than about test quality. Useful expectations, in the order they should be reached for:

Caveats on the number.

Wave 1 - accessibility, migrated (issue #2148, 2026-08-20). The first of the waves #2148 calls for, taken in the order that issue suggested: the accessibility slice, because it was the single largest one (AccessibilityTests.cs alone held 90 of the 542 cases) and because vitest-axe at component level is a well-trodden swap rather than a new invention.

What moved. AccessibilityTests.cs went from 82 test methods / 85 cases to 14 methods / 17 cases, and from 3000 lines to 533. In its place, frontend/ gained a component-level a11y suite: 20 files, 99 cases, colocated as src/*/.a11y.test.tsx and run by the existing pnpm test gate in frontend-checks.yml. The new dependencies are vitest-axe, axe-core, @testing-library/react, @testing-library/dom, @testing-library/user-event and @testing-library/jest-dom; the shared harness lives in frontend/src/test/.

The gate is deliberately identical at both altitudes - fail on any serious/critical axe violation plus the same escalated moderate rules - so a scan that moved down did not also quietly get easier. What changed is scope, and it changed in both directions.

What could not move, and why the ~10 page-level smokes are the number they are. Three classes of check are unavailable in jsdom, and they are not cosmetic:

So what stayed is one scan per distinct layout and palette - public landing, browse, opportunity detail, signed-in volunteer profile, my-signups, org app dashboard, engagement management, administration (4 sections), static legal page - plus the five browser-only assertions above. What left is every scan whose only difference from another was which component happened to be mounted: 68 methods, each of which booted the Aspire stack, signed a user in, seeded data and drove a browser to reach a dialog or a dropdown panel that renders from props.

Residual coverage gap, stated rather than glossed. Contrast inside states only reachable behind a modal or an edit-mode toggle - the org-settings edit form, the dashboard widget colour dialogs, the cancel dialogs - is no longer checked automatically; those states keep their structural and semantic coverage at component level and lose their contrast coverage. Two things bound that: DateRangeContrastTests and EngagementStatusContrastTests remain as dedicated, deterministic contrast regressions, and contrast in this codebase is a property of a small set of design tokens defined once in global.css, with the floors documented in frontend/AGENTS.md. If the gap turns out to bite, the answer is a token-level contrast test, not 68 restored Aspire boots.

Mutation score after the wave: unchanged. Re-run at concurrency 1 against the same Stryker 4.16.0 configuration as the #2147 baseline: Domain 66.75 % (872 generated / 764 scored / 510 killed / 173 survived / 81 no coverage, 4 min 47 s), Application 68.49 % (1818 / 1466 / 1004 / 343 / 119, 8 min 45 s), combined 67.89 % - every one of those figures identical to the recorded baseline, mutant for mutant. That is the expected result rather than a surprising one, and worth saying plainly: this wave changed no file under backend/src, and the mutation run mutates Domain/Application driven by Application.UnitTests, which it also did not touch. The number is reported because #2148’s acceptance criteria ask for it after every wave and because a wave that did move backend coverage would be indistinguishable from this one without it - not because it is evidence about the a11y scans themselves. For this wave the falsifiable check is the one CI runs: the 14 surviving page scans still pass, and the 99 component cases assert the same rule set the deleted ones did.

Shard count deliberately not changed yet. #2148’s acceptance criteria call for shrinking the visual-tests shard count once the ~2 min 45 s of fixed per-shard cost starts dominating. It plausibly now does - the suite is down to 474 cases - but the numbers that would decide it come from a real run of this branch, not from an estimate, and this document has twice had to correct a figure that was reasoned about rather than measured. Re-read the shard timings from this change’s own dotnet.yml run before touching the matrix.

Next waves follow #2148’s own table - see the waves 2-5 note below for what they turned into.

Waves 2-5 - the rest of the rebalance, and what the target ratio actually turned out to be (issue #2148, 2026-08-20).

What moved. 84 more cases left VisualTests after wave 1’s 68, in four waves:

The measured shape, before and after. Counting every executable case in each project (Application.UnitTests + ArchitectureTests + frontend’s Vitest suite as "unit", `IntegrationTests as "integration", VisualTests as "E2E"):

So: the unit share hit its target exactly, and the E2E share did not come close. That is the finding, and it is worth stating plainly rather than filing as unfinished work.

Why ~2.5 % E2E is not reachable here. Classifying every one of the 391 surviving cases by what it actually depends on - not by which file it sits in:

Moving every one of the remaining ~198 - which would mean rewriting the two largest classes in the suite (VolunteerOpportunityTests, OrganizationTests) plus a long tail - lands E2E at roughly 190 cases, or ~7 %. The floor is set by the ~193 that need a browser, not by appetite. Reaching 50 would mean deleting layout, theme and real-auth regressions that have no lower-altitude equivalent - which is the exact failure #2147’s mutation baseline exists to prevent, just at a layer that baseline cannot see.

Recommendation: treat ~7 % as the realistic target and close #2148 against it, or reopen the question as "should layout regressions be covered by visual-diff snapshots instead of Playwright assertions", which is a different decision with a different cost.

The cheapest win was not the case count. HomePageOrgCtaTests called fixture.ResetAsync() before every test and carried [NotInParallel("visualtests-db")] so it could guarantee vera organized nothing. Three of its four cases moved, and with them the database reset and the serialization - that class no longer blocks every other DB-touching class in the suite. Three more classes (EngagementCheckInStatusCodeTests, MilestoneAchievementTests, LoginStreakConcurrencyTests) were paying for a Chromium context, a frontend and Playwright’s ~290 MiB browser download to make plain HTTP calls. Neither shows up as much in a case count and both cost real wall-clock.

Mutation score after waves 2-5: 66.75 % / 68.49 % / 67.89 %, bit-for-bit identical to the #2147 baseline. Re-run at the end of this branch on the same concurrency-1 Stryker 4.16.0 configuration: Domain 872 generated / 764 scored / 510 killed / 173 survived / 81 no-coverage; Application 1818 / 1466 / 1004 (1003 killed plus one timeout) / 343 / 119. That is the wanted result rather than a weak one, and it is worth being precise about why: no wave changed a file under backend/src - wave 5 changed only which project three test classes live in - and the run mutates Domain/Application driven by Application.UnitTests, which the waves also did not touch. A movement would have meant something quite specific: that the number does not measure what it claims to, so any drop would have been a tooling problem rather than lost coverage. What this confirms is narrow and should be read as such - it says the moved cases took no Application/Domain coverage with them, not that the Vitest and IntegrationTests replacements are as strong as the Playwright originals. Nothing in this repo measures that second thing today.

Shard count: still four, and now from a direct reading rather than an estimate. This branch’s own green run (32405319770, 2026-08-20, 391 cases) puts the four shards at 5 min 22 s - 6 min 39 s of wall clock, with the instrumented Aspire boot reading 53.6 s / 63.1 s / 68.3 s / 68.9 s. Splitting each shard into what scales and what does not: fixed ~2 min 36 s (1 min 12 s - 1 min 55 s of job setup, restore and image pulls, plus the 63.5 s mean Aspire boot inside the test run) and variable ~3 min 25 s (820 s of real test work over 391 cases, or ~2.1 s per case). That gives shard ~= 156 s + 2.1 s * (391 / N), which predicts a 6 min 02 s mean at N = 4 against that run’s observed 6 min 02 s.

Read that fit with care, because a second run of the identical suite arrived before this paragraph was a day old. Run 32407729992 (commit a0c972e, docs-only, so the same 391 cases) came in at a 6 min 30 s mean - 28 s above the model, not the exact match the first run suggested. What did reproduce is the number the decision actually turns on: the slowest shard was 6 min 39 s then 6 min 44 s, and the critical path 9 min 05 s then 9 min 06 s. The mean moves because the fast shards vary (run 1 spanned 5 min 22 s - 6 min 39 s, run 2 a tighter 6 min 09 s - 6 min 44 s); the ceiling barely does. So the model estimates the tail well and the middle loosely, which is the right way round for sizing a matrix - but the single-run exact fit was luck, and this document has been burned by exactly that before.

Extrapolating from the model, and holding the observed slowest-vs-mean spread: three shards would run ~7 min 10 s each for a ~10 min critical path, and five would run ~5 min 20 s for ~8 min 20 s, against the measured 9 min 05 s / 9 min 06 s at four. So dropping a runner costs roughly a minute of critical path and adding one buys roughly 45 s - three shards is not neutral, which is what the wave-1 estimate in this document assumed before the run existed to check it. Four stays. The threshold worth watching is the fixed share: at 43 % of a shard’s wall clock it is already the larger half of the residual, and removing another ~100 cases would push it past 50 %, which is the point where folding shards starts to pay. The matrix is worth touching once, at whatever the final case count turns out to be, rather than re-tuned per wave to land in the same place.

AssemblyParallelLimit’s `ProcessorCount - 2 is likewise unchanged. #2148 suggested revisiting it "once the suite is much smaller". It is 28 % smaller, not much smaller, and the contention it guards against (Playwright and axe-core sessions competing with the Aspire stack they all call into) is unchanged per-shard. Raising it is a controlled experiment against real CI failure rates, not a follow-on edit.

Waves 6-13 - the remaining pool cleared, and the answer to "how low does E2E go" (issue #2148, 2026-08-21).

The waves 2-5 note above closed with an estimate: ~198 cases were movable "in principle", which would land E2E at roughly 190 cases or ~7 %. That estimate has now been replaced by a measurement, and the estimate was close but arrived at from the wrong direction.

How the remaining pool was established. Rather than reasoning per class, every one of the then-surviving cases was read in full with its helper calls resolved, classified, and then handed to a second reviewer instructed to refute the classification and to default to "cannot move" under any uncertainty. That refutation pass is the part worth keeping: it rejected a third of the proposed moves, and several of the rejections were cases that would have passed in Vitest while asserting nothing - a queryByRole("button", { name: "Remove" }) that matches nothing whether or not a Remove button exists, because the accessible name is an aria-label reading "Remove Vera Volunteer". Playwright matches accessible names by substring and RTL matches exactly; a port that does not account for that difference is a green test with no subject.

What the pass produced. 134 cases confirmed movable - 120 to Vitest/RTL, 14 to IntegrationTests. All 134 have moved, in waves 6-13. Two further cases were deleted rather than moved, because IntegrationTests already covered exactly what they asserted (`EngagementReadRepositoryTests’ checked-in bucket rule and `EngagementTests’ gone-opportunity reclassification).

The measured final shape.

Every figure counts cases, not [Test] methods: a method carrying [Arguments] expands to one case per argument set, which is what CI runs and what visual-test-shard.sh balances.

So the unit layer overshot its target and E2E stopped at 8 %, not 2.5 %. Categorising the 232 survivors mechanically - by which browser-only API the case or its helpers actually reach, checked in order:

The first, third, fourth and fifth rows are the same four categories the waves 2-5 note called a floor, and they still are. What that note underestimated is the second row: cases whose subject is a journey rather than a rendering, which are not movable for a different reason than layout is - not "jsdom cannot measure this" but "there is nothing here for a component test to be about".

Recommendation, restated with the arithmetic behind it. ~8 % is the reachable target for this suite as it stands. Reaching 2.5 % would mean deleting layout, theme, real-auth and journey regressions that have no lower-altitude equivalent - the exact failure #2147’s mutation baseline exists to prevent, at a layer that baseline cannot see. Either adopt ~8 % and close #2148 against it, or reopen the question as "should layout regressions be visual-diff snapshots rather than Playwright assertions", which is a different decision with a different cost.

What the case count does not show. Fifteen whole classes are gone, several of which were paying a disproportionate share of the suite’s fixed cost: AvatarAndLogoDisplayTests forced [NotInParallel("visualtests-vera-avatar")] on its own upload case because its DELETE made vera’s avatar vanish mid-assertion, and EngagementReactivationTests, EngagementTimezoneTests and the VisualTests AdminReportsTests each booted a Chromium context and a frontend to make plain HTTP calls - EngagementTimezoneTests opened a page solely to scrape an OIDC token out of sessionStorage.

Three moved cases came out stronger than the originals, which is worth recording because the usual risk of this work is the opposite. SocialPreviewImageTests booted a browser, an Aspire stack and a frontend to read two hardcoded strings out of a static index.html - and despite its name never opened the image, so a declaration disagreeing with the file passed; as a Vitest case both are build inputs, so it now parses the PNG’s IHDR and actually compares. OrganizationSwitcher_SelectingAnOrgRow_…​ carried Skip.When(rowCount < 2) and silently did nothing unless the signed-in user happened to belong to two organizations; a two-org prop list removes the skip. MiniCalendar’s past-day guard is now asserted as behaviour (`onChange never fires) rather than only as the aria-disabled attribute - which matters, because that attribute deliberately leaves the button clickable so arrow keys can cross it, so the attribute alone was never the guard.

One finding filed rather than fixed. UndoCheckInEngagementEndpoint returns 409 for Engagement.AlreadyTerminated and Engagement.CheckInNotActive, both of which have user-facing translations, but declares only 400/401/403/404 - so NSwag generates no typed branch for 409 in either generated client and the rejection arrives as the non-generic ApiException. Not user-facing (the frontend reads errorCode off the body generically), but the OpenAPI document is inaccurate. Filed as afunto#2158; not fixed here because the one-line .ProducesProblem(…​) regenerates both API clients, which does not belong in a test-rebalance change.

Mutation score after waves 6-13: 66.75 % / 68.49 % / 67.89 %, still bit-for-bit identical to the #2147 baseline. Re-run on this branch’s final head at concurrency 1 on the same Stryker 4.16.0 configuration: Domain 872 generated / 764 scored / 510 killed / 173 survived / 81 no-coverage; Application 1818 / 1466 / 1004 (1003 killed plus one timeout) / 343 / 119. The reading is the same one the waves 2-5 note gave and should stay narrow: it says the moved cases took no Application/Domain coverage with them, not that the Vitest and IntegrationTests replacements are as strong as the Playwright originals. Nothing in this repo measures that second thing today.

Frontend coverage, which does move. vitest.config.ts’s `coverage.include was src/lib//.ts - correct while that suite was only the pure-function tests, and blind to everything #2148 moved down. Widened to src//.{ts,tsx} (excluding the tests themselves, the src/test/ harness, the NSwag-generated client and composition roots), the same measurement reads 7.22 % statements before, 60.94 % after - 4.45 % to 60.02 % branches, 7.32 % to 56.85 % functions. That is the number that actually moved, and it moved because coverage of components and pages now exists at a layer something measures.

A gap this document should name: nothing measures the moved tests' own strength. The mutation score above is Stryker.NET mutating Domain/Application under Application.UnitTests. It is silent about the Vitest suite by construction, so "identical score" says the moved cases took no backend coverage with them and nothing more.

StrykerJS (@stryker-mutator/core + @stryker-mutator/vitest-runner) closes exactly that gap, and #2148 adds it: frontend/stryker.config.json, pnpm mutation, and a mutation-tests-frontend job on the existing manual mutation-tests.yml. Findings worth recording before anyone changes its scope:

The cost is why the committed scope is src/lib/ and not src/. Instrumenting the whole frontend produces 14 851 mutants over 182 files - 5.5x the backend’s 2 690, against a suite that renders components in jsdom rather than calling pure functions. src/lib/** is 37 files / 897 mutants of pure logic: the tier that is both cheap to mutate and directly comparable to Domain/Application. The component tier stays on demand (pnpm mutation --mutate "src/components/Foo.tsx"), which is the useful granularity anyway - a component’s survivors name specific unasserted branches, a whole-app average names nothing.

The baseline, recorded so later waves have something to compare against. This is the forward-looking equivalent of #2147’s table: it says nothing about what this PR moved, and everything about whether the next change to src/lib weakens the suite covering it.

Wall-clock is from CI run 32479000847, the first real execution of mutation-tests.yml and of the mutation-tests-frontend job specifically; the same commit took 36 min 24 s on a contended 4-core sandbox. That run also re-confirmed the two .NET legs on a GitHub runner rather than locally: Domain 66.75 % in 4 min 23 s, Application 68.49 % in 7 min 34 s, both identical to the #2147 baseline. So the frontend leg alone costs more than twice the entire backend matrix, which is the single most useful fact for deciding where to run it.

concurrency: 2 is reproducible for StrykerJS, and this is worth stating explicitly because the backend concluded the opposite. The 80.95 % above was produced twice - once on a contended 4-core sandbox, once on a clean ubuntu-latest - with a bit-for-bit identical per-file table, at concurrency: 2 both times. Stryker.NET, by contrast, needs concurrency: 1 because its preview MTP runner invents 30-73 kills per run at 2 (see "The concurrency trap" above). These are different runners with different defects; nobody should propagate the .NET constraint to the JS config by analogy, and nobody should assume the JS result vindicates raising the .NET one. Two agreeing runs is evidence, not proof, but it is the evidence there is.

The 57-mutant gap between generated and scored is entirely ignoreStatic exclusions; there were no compile errors. At 80.95 % this tier scores materially better than the backend’s 66.75 % / 68.49 %, which is what one would expect of pure functions with colocated unit tests rather than evidence that the frontend is better tested - the tier was chosen for being the easy one. The survivors concentrate in four files: organizationFormSchema.ts (37 of 107), format.ts (28 of 145), imageCrop.ts (27 of 59) and imageUpload.ts (18 of 68). As on the backend, string-literal mutations lead the survivor mix (45 of 126), then conditional expressions (26) and equality operators (14) - so the same caution applies: read the survivors per file, do not chase the headline.

Two costs that are not obvious until you run it, and one estimate that did not hold. Every run pays a 3 min 21 s instrumented dry run before a single mutant is tested, against ~50 s for the same suite uninstrumented - fixed overhead larger than the whole checks / test job (1 min 39 s). And the run is slow in absolute terms: 36 min 24 s for the cheap tier, against ~12 minutes for the backend’s entire Domain + Application matrix. That gap is the whole reason the component tier stays on demand.

The estimate that did not hold is worth recording, because it was the basis for a configuration choice. Stryker warns that 111 static mutants (12 % of the 897) are estimated to take 69 % of the run, and recommends ignoreStatic. Taking that recommendation: the flag excluded 57 mutants rather than 111, and the run still took 36 minutes. The pass without it was killed at 35 minutes before finishing, so the saving is real but its size is unmeasured - what is measured is that it does not make the run cheap. The flag stays (module-scope initialisation is excluded from the denominator as a result, which is the trade), but nobody should plan against that 69 % as recoverable headroom.

Three practical notes: under pnpm the plugin must be listed explicitly in plugins or Stryker reports no test runner found; .stryker-tmp/ is a full copy of the project, so ESLint needs it in ignores or pnpm lint fails on its vendored coverage scripts after any local run; and the version is pinned for the same reason dotnet-stryker is - a new Stryker can add or drop mutators, and the score is only comparable across runs if the mutant set is.

Shard count and AssemblyParallelLimit: still four and still ProcessorCount - 2, and the reason is a correction to this document’s own framing.

Measured on this branch’s green run at 232 cases: build 2 min 12 s, shards 4 min 36 s / 5 min 21 s / 4 min 51 s / 5 min 04 s (mean 4 min 58 s), integration-tests 4 min 23 s, critical path 7 min 33 s - against 9 min 05 s at 391 cases and ~9 min 50 s at 474. The model (shard ~= 156 s + 2.1 s * (cases / N)) predicts 4 min 38 s at N = 4 against an observed 4 min 58 s mean, so it still under-predicts the middle by ~20 s and tracks the tail; unchanged in character from the two runs above.

The fixed share is now 52 % of a shard’s wall clock (156 s of the 298 s mean), so the 50 % threshold recorded above has indeed been crossed. It does not mean what this document said it meant. A high fixed share means adding shards has stopped paying - not that removing them starts to. Folding a shard moves its cases onto the survivors, which are running in parallel, so as long as variable work is positive the critical path always gets worse:

So folding to three costs ~40 s of critical path and saves one runner; folding to two costs ~2 minutes. The trade is runners against wall clock, and it never was a speed win - the earlier framing ("only then does folding shards start to pay") was wrong, and the arithmetic that produced it was reasoned about rather than measured, which is the third time this document has had to record that exact mistake. Four shards stay unless the runner budget becomes the binding constraint.

The #2159 residue, reconciled (issue #2162, 2026-08-23). #2152 closed #2159 against its headline number - 134 confirmed-movable cases - but #2159’s own summary table and its per-case detail section disagreed on the count: the table budgeted "25 further classes | 1 each", the detail section named roughly 44 distinct single-case classes. #2152 executed against the table and left 16 classes / 23 [Test] methods never moved and never re-refuted in writing. This note is that refutation pass, run the same way #2159’s own method describes: each class read in full with helpers resolved, then classified adversarially rather than trusted at face value.

The result is not "move all 16". Nine of the sixteen classes turned out to need the same adversarial second look #2152 gave everything else - some were correctly movable, some were wrongly classified as movable by #2159’s rushed detail pass, and two classes each split into a case that moves and a case that cannot.

The part that actually needed a decision, not just re-classification. #2159 flagged both SmokeTests.Login_AsVera_CompletesAndReturnsToHome and all of EngagementCoreJourneysTests as movable; the issue that carried this residue forward (#2162) pointed out that both read, by name, like exactly the critical journeys #2148 says must stay E2E - and that #2148 and #2159 had never been reconciled on this point. Reading the bodies rather than the names resolves it in both directions, not by splitting the difference:

#2148’s outstanding acceptance criterion - "critical user journeys explicitly listed and kept as real E2E" - is met by this list, not by the class name EngagementCoreJourneysTests that turned out not to be one. Grepping this repository for that phrase before #2162 returned nothing; these are the E2E cases actually carrying that weight, by what their bodies do rather than what their classes are called:

Read this table narrowly, the same way the mutation-score readings above ask to be read. "Search → apply → check in" as one continuous, named E2E case does not exist in this suite, and this list does not pretend otherwise - what exists instead is each step protected independently (apply via the real modal in the surviving OpportunityApplicationStateTests race-condition case; check-in via the real scanner above), each seeded through the API rather than clicked through from scratch. That is a deliberate cost-minimization already established by #2148/#2152 for every other survivor, not a gap this note is introducing.

The final shape. 328 cases have now moved out of VisualTests in total (310 from #2148/#2152, 18 more from this reconciliation), 2 were deleted outright as redundant (SmokeTests), leaving 214. visual-test-shard.sh repacked cleanly across all four shards with no orphaned class (54/54/53/53 cases, 18/19/18/18 classes) - confirming the "a class stops matching its file name" failure mode #2148 warned about did not happen here.