Add editor sample regression pack

This commit is contained in:
Rbanh committed 2026-07-13 01:52:58 -04:00
1 parent c55f34780f
commit 06010c5922
31 files changed
+3258 -110

No files matched your search

@@ -4,15 +4,17 @@ Date: 2026-07-12
![Three selected props settled on the committed placement floor](physics-placement-settled.png)
The native Wayland editor is shown with the committed
`assets/levels/physics_placement_showcase.scn.ron` fixture. Placement Prop A, B, and C were
The native Wayland editor is shown with the 2026-07-12 revision of
`assets/levels/physics_placement_showcase.scn.ron`. At that revision, Placement Prop A, B, and C were
multi-selected and released from three different heights through the hand/down-arrow action in the
existing horizontal viewport toolbar.
existing horizontal viewport toolbar. The fixture now exposes those stable actor IDs as Placement
Cuboid, Placement Sphere, and Placement Capsule; its distinct-shape rerun is tracked by the
[sample-pack evaluation](../sample-regression-pack/).
## Acceptance
- All three props used the hydrated Avian cuboid colliders and settled on Placement Floor after
1.5 simulated seconds.
- In the recorded revision, all three props used hydrated Avian cuboid colliders and settled on
Placement Floor after 1.5 simulated seconds.
- During preview the HUD reported selection count, simulated time, and Settled state; transform
gizmos and viewport picking did not compete with the modal tool.
- Escape restored every starting transform and left history empty.
@@ -25,4 +27,3 @@ existing horizontal viewport toolbar.
error after the stale-binding fix.
Packaged-runtime tests remain intentionally deferred by project-owner request.
@@ -1,8 +1,8 @@
# Production-Readiness Acceptance Matrix
**Matrix version:** 0.2
**Matrix version:** 0.3
**Last audit:** 2026-07-12
**Last audit:** 2026-07-13
**Release-candidate commit:** Not nominated
@@ -30,21 +30,21 @@ another commit, a dirty worktree, or an older package do not transfer to the can
| ID | Requirement | State | Current evidence and gap |
|----|-------------|-------|--------------------------|
| G1 | Project create/open/resume, scene authoring, autosave/recovery, hierarchy, prefab, and asset integrity pass | Partial | Project/recovery/session/multi-scene/prefab implementations are documented in [project launcher](../../project-launcher.md), [session recovery](../../session-recovery.md), [multi-scene composition](../../multi-scene-composition.md), and [prefab authoring](../../prefab-authoring.md). Collaborative file safety `#49` and non-blocking native dialogs `#52` passed live acceptance. Candidate-specific end-to-end reruns remain. |
| G2 | Brush, material, terrain, physics placement, animation, audio, navigation, PIE, and build/package samples pass | Fail | Animation, audio, navigation, renderer/material foundation `#51`, Material Library, targeted material drops, terrain `#22`-`#24`, physics placement `#25`, and build foundations are implemented with source/live fixtures. Brush acceptance `#37` remains incomplete. Packaged testing is owner-deferred. |
| G3 | Undo/redo/cancel invariants and helper cleanup cover every production mutation path | Partial | Source implementation and focused evidence are complete in [operator regression testing](../../operator-regression-testing.md) and the [operator-invariants evaluation](../operator-invariants/); Gitea `#33` is ready to close. A clean, exact-candidate rerun is still required for `Pass`. |
| G1 | Project create/open/resume, scene authoring, autosave/recovery, hierarchy, prefab, and asset integrity pass | Fail | Project/recovery/session/multi-scene/prefab implementations are documented in [project launcher](../../project-launcher.md), [session recovery](../../session-recovery.md), [multi-scene composition](../../multi-scene-composition.md), and [prefab authoring](../../prefab-authoring.md). Collaborative file safety `#49` and non-blocking native dialogs `#52` passed live acceptance, but `#55` records unguarded native dirty shutdown and a missing clean-history savepoint. Candidate-specific end-to-end reruns also remain. |
| G2 | Brush, material, terrain, physics placement, animation, audio, navigation, PIE, and build/package samples pass | Partial | The [five-area sample pack](../sample-regression-pack/) now provides deterministic Brush, Material, Terrain, Physics Placement, and Rendering fixtures with source validation. Animation, audio, navigation, PIE, and build foundations also exist, but the exact-candidate pack rerun, brush acceptance `#37`, and combined workflow evidence remain incomplete. Packaged testing is owner-deferred. |
| G3 | Undo/redo/cancel invariants and helper cleanup cover every production mutation path | Partial | Source implementation and focused evidence are complete in [operator regression testing](../../operator-regression-testing.md) and the [operator-invariants evaluation](../operator-invariants/); Gitea `#33` closed at `c55f347`. A clean, exact-candidate rerun is still required for `Pass`. |
| G4 | Representative project completes an eight-hour soak without unbounded memory/target growth or unrecoverable failure | Missing | No candidate soak log, resource timeline, failure ledger, or target-growth measurement exists. |
| G5 | Cold start, scene open/save, asset refresh, common manipulation, and package-build budgets are documented and measured | Missing | Gitea `#34` is open; no ratified budgets or candidate measurement record exists. |
| G6 | Headless content validation and CI are green from a clean checkout | Missing | [CI configuration](../../../../.github/workflows/ci.yml) exists and local source/headless checks have passed during feature work, but no clean-checkout candidate run is linked. The current Gitea server does not expose an Actions run endpoint for this repository. |
| G6 | Headless content validation and CI are green from a clean checkout | Missing | [CI configuration](../../../../.github/workflows/ci.yml) hydrates LFS and includes both project and sample content gates. Local implementation checks have passed, but no clean-checkout candidate run is linked. The current Gitea server does not expose an Actions run endpoint for this repository. |
| G7 | First-hour UX and recovery QA are signed off by someone other than the implementer | External | No independent sign-off exists. Gitea `#36` remains open; historical H1-H6 implementation-pass notes do not count. |
| G8 | Known limitations have severity/workaround and no P0 blocker remains | Fail | Limitations are distributed across feature docs rather than one candidate ledger. The remaining open P0 implementation blocker is `#32`; `#50` cannot close while it remains. Operator invariants `#33`, renderer foundation `#51`, roadmap audit `#35`, Material Library `#16`, targeted drops `#18`, collaborative safety `#49`, and dialog responsiveness `#52` are complete. Property-block application `#53` is P1; dynamic deformed Solari geometry `#54` is a documented P2 limitation with Forward/raster fallback. |
| G8 | Known limitations have severity/workaround and no P0 blocker remains | Fail | The sample-pack implementation removes the previously identified `#32` content gap, pending exact-commit native evidence and ticket closure, but guarded native shutdown and clean history savepoints remain the open P0 `#55`. A consolidated candidate limitations ledger is also missing. Deterministic imported-asset fingerprints `#56` and property-block application `#53` are P1; dynamic deformed Solari geometry `#54` is a documented P2 limitation with Forward/raster fallback. |
## Deliverables
| ID | Deliverable | State | Evidence or next action |
|----|-------------|-------|-------------------------|
| D1 | Versioned acceptance matrix under `docs/editor/evaluations/` | Pass | This file, version 0.2. |
| D2 | Release-candidate representative project and reproducible validation commands | Missing | Expand the committed project/regression pack through `#32`, including terrain and physics-placement samples, then nominate an exact commit. |
| D1 | Versioned acceptance matrix under `docs/editor/evaluations/` | Pass | This file, version 0.3. |
| D2 | Release-candidate representative project and reproducible validation commands | Partial | The committed [sample regression pack](../sample-regression-pack/) and commands exist. Nominate an exact commit, rerun from a clean LFS-hydrated checkout, and attach native results. |
| D3 | Signed milestone comment linking evidence, limitations, and exact commit | Missing | Post only after G1-G8 pass; no candidate exists yet. |
## Workflow Coverage
@@ -55,11 +55,13 @@ another commit, a dirty worktree, or an older package do not transfer to the can
| Scene save, autosave, recovery | Implemented | Not rerun |
| Hierarchy parenting and prefab structure | Implemented | Not rerun |
| Multi-scene composition | Implemented | Not rerun |
| Five-area editor sample pack | Implemented with manifest, native menu, typed scene gate, and headless validation | Source evidence recorded; exact-candidate native run pending |
| Brush blockout/edit/CSG | Implemented foundation; `#37` open | Not signed off |
| Material catalog and assignment | Renderer/material foundation `#51`, Material Library, and exact targeted drops accepted; optional property-block promotion is `#53` | Not rerun as one candidate |
| Terrain authoring | `#22`-`#24` source/live acceptance complete; M3 closed | Pass; packaged-runtime acceptance owner-deferred |
| Physics placement | `#25` source/live acceptance complete | Pass; packaged-runtime acceptance owner-deferred |
| Collider authoring and diagnostics | `#26` source/live acceptance complete | Pass; packaged-runtime acceptance owner-deferred |
| Rendering volumes and look development | Deterministic visible rendering fixture is in the sample pack | Not rerun as one candidate |
| Terrain authoring | `#22`-`#24` source/live acceptance complete; M3 closed | Feature-level live acceptance exists; not rerun as one candidate. Packaged-runtime acceptance owner-deferred. |
| Physics placement | `#25` source/live acceptance complete | Feature-level live acceptance exists; not rerun as one candidate. Packaged-runtime acceptance owner-deferred. |
| Collider authoring and diagnostics | `#26` source/live acceptance complete | Feature-level live acceptance exists; not rerun as one candidate. Packaged-runtime acceptance owner-deferred. |
| Animation | Implemented | Not rerun |
| Audio | Implemented | Not rerun |
| Navigation | Implemented | Not rerun |
@@ -81,6 +83,7 @@ cargo clippy --workspace --all-targets -- -D warnings
cargo clippy -p editor -p game --bins --features dev,hot-reload -- -D warnings
cargo test --workspace
cargo validate-levels --project .
cargo validate-samples --project .
cargo bake-navigation --project . --check
```
@@ -92,8 +95,9 @@ package hash verification, and packaged-runtime smoke protocol used for the cand
The final candidate must run for eight continuous hours against the representative project. The
record must sample process RSS, GPU memory, target-directory size, recovery generations, background
job counts, and error/warning totals at least every five minutes. Exercise scene switching, save and
recovery, asset refresh, material editing, brush/terrain/physics tools, animation/audio/navigation,
PIE transitions, and build UI cancellation without replacing the candidate during the run.
recovery, asset refresh, all five sample-pack scenes, material editing, brush/terrain/physics tools,
animation/audio/navigation, PIE transitions, and build UI cancellation without replacing the
candidate during the run.
A pass requires no unrecoverable editor failure, no lost authored work, no unbounded upward trend in
steady-state resource use, bounded recovery/cache behavior, and a triaged explanation for every
@@ -0,0 +1,55 @@
# Sample Regression Pack Evaluation
**Date:** 2026-07-13
**Issue:** [Gitea #32](https://git.spacetrainclubhouse.com/Falling-Metal-Interactive/Blacksite/issues/32)
**Implementation commit:** Pending commit
**Release-candidate commit:** Not nominated
**Implementation validation:** **Pass**
**Native visual acceptance:** **Pending**
This record separates source/fixture verification from release-candidate acceptance. The committed
five-area catalog and scenes are ready for native QA, but results from the implementation worktree
do not transfer to a later candidate commit.
## Source Evidence
| Check | Result | Evidence |
|-------|--------|----------|
| Manifest contract | Pass | Schema v1 contains one ordered entry for Brush, Material, Terrain, Physics Placement, and Rendering, with stable IDs and nonempty checks. |
| Authored scene fixtures | Pass | All five scene paths exist; actors use stable IDs and each scene contains its required authored feature. |
| Project validation | Pass | `cargo validate-levels --project .` reported 78 dependencies, five known nonblocking import/platform findings, and zero blocking errors on 2026-07-13. |
| Typed no-GPU deserialization | Pass | `cargo test -p editor every_manifest_sample_typed_deserializes_without_a_gpu --lib`: one passed, zero failed, 256 filtered out. |
| Sample-pack validation | Pass | `cargo validate-samples --project .`: five ordered samples, 83 dependencies, five known nonblocking findings, zero blocking errors. |
| Clean-checkout CI | Pending | The exact candidate must hydrate LFS and pass both content gates. Current Gitea does not expose an Actions run endpoint for this repository. |
## Native Protocol
Open every row through **File > Open Sample**, not through a typed filesystem path. Capture native
screenshots only after confirming the viewport is nonblank and correctly framed.
| Sample | Required native interaction and visual result | Result |
|--------|-----------------------------------------------|--------|
| Brush Blockout | Foundation, material-face tower, and red subtractive marker are distinct; enter brush edit and cancel one edit or clip preview without leaving helpers | Pending |
| Material Lab | Concrete cube, custom sphere, instance block, and emissive panel are visible; perform one exact primitive material drop and undo | Pending |
| Terrain Authoring | Hill chunks and two material layers render; perform and cancel one sculpt stroke and one paint stroke | Pending |
| Physics Placement | Cuboid, sphere, and capsule settle on the floor; cancel restores all three, then commit/undo behaves as one transaction | Pending |
| Rendering Lab | Red/cyan/amber anchors and emissive strip render; camera traversal visibly changes vignette, fog, and exposure ownership | Pending |
For each row, inspect the status strip, Diagnostics panel, and process log for missing references,
hydration errors, render errors, helper leaks, or operator failures. Attach native images to the Gitea
issue as ordinary uploaded attachments rather than embedding repository raw URLs. A committed
original may remain beside this record through Git LFS under the shared evaluation policy.
## Release Use
A future candidate may mark this evaluation fully passed only when the exact nominated commit has:
1. Passed typed scene deserialization, `validate-levels`, and `validate-samples` from a clean,
LFS-hydrated checkout.
2. Opened all five entries through the native menu and completed the interactions above.
3. Recorded the commit, machine profile, commands, exit codes, logs, and native attachments.