Add editor sample regression pack
This commit is contained in:
1 parent
c55f34780f
commit
06010c5922
31 files changed
+3258
-110
No files matched your search
@@ -4,15 +4,17 @@ Date: 2026-07-12
|
||||
|
||||

|
||||
|
||||
The native Wayland editor is shown with the committed
|
||||
`assets/levels/physics_placement_showcase.scn.ron` fixture. Placement Prop A, B, and C were
|
||||
The native Wayland editor is shown with the 2026-07-12 revision of
|
||||
`assets/levels/physics_placement_showcase.scn.ron`. At that revision, Placement Prop A, B, and C were
|
||||
multi-selected and released from three different heights through the hand/down-arrow action in the
|
||||
existing horizontal viewport toolbar.
|
||||
existing horizontal viewport toolbar. The fixture now exposes those stable actor IDs as Placement
|
||||
Cuboid, Placement Sphere, and Placement Capsule; its distinct-shape rerun is tracked by the
|
||||
[sample-pack evaluation](../sample-regression-pack/).
|
||||
|
||||
## Acceptance
|
||||
|
||||
- All three props used the hydrated Avian cuboid colliders and settled on Placement Floor after
|
||||
1.5 simulated seconds.
|
||||
- In the recorded revision, all three props used hydrated Avian cuboid colliders and settled on
|
||||
Placement Floor after 1.5 simulated seconds.
|
||||
- During preview the HUD reported selection count, simulated time, and Settled state; transform
|
||||
gizmos and viewport picking did not compete with the modal tool.
|
||||
- Escape restored every starting transform and left history empty.
|
||||
@@ -25,4 +27,3 @@ existing horizontal viewport toolbar.
|
||||
error after the stale-binding fix.
|
||||
|
||||
Packaged-runtime tests remain intentionally deferred by project-owner request.
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# Production-Readiness Acceptance Matrix
|
||||
|
||||
**Matrix version:** 0.2
|
||||
**Matrix version:** 0.3
|
||||
|
||||
**Last audit:** 2026-07-12
|
||||
**Last audit:** 2026-07-13
|
||||
|
||||
**Release-candidate commit:** Not nominated
|
||||
|
||||
@@ -30,21 +30,21 @@ another commit, a dirty worktree, or an older package do not transfer to the can
|
||||
|
||||
| ID | Requirement | State | Current evidence and gap |
|
||||
|----|-------------|-------|--------------------------|
|
||||
| G1 | Project create/open/resume, scene authoring, autosave/recovery, hierarchy, prefab, and asset integrity pass | Partial | Project/recovery/session/multi-scene/prefab implementations are documented in [project launcher](../../project-launcher.md), [session recovery](../../session-recovery.md), [multi-scene composition](../../multi-scene-composition.md), and [prefab authoring](../../prefab-authoring.md). Collaborative file safety `#49` and non-blocking native dialogs `#52` passed live acceptance. Candidate-specific end-to-end reruns remain. |
|
||||
| G2 | Brush, material, terrain, physics placement, animation, audio, navigation, PIE, and build/package samples pass | Fail | Animation, audio, navigation, renderer/material foundation `#51`, Material Library, targeted material drops, terrain `#22`-`#24`, physics placement `#25`, and build foundations are implemented with source/live fixtures. Brush acceptance `#37` remains incomplete. Packaged testing is owner-deferred. |
|
||||
| G3 | Undo/redo/cancel invariants and helper cleanup cover every production mutation path | Partial | Source implementation and focused evidence are complete in [operator regression testing](../../operator-regression-testing.md) and the [operator-invariants evaluation](../operator-invariants/); Gitea `#33` is ready to close. A clean, exact-candidate rerun is still required for `Pass`. |
|
||||
| G1 | Project create/open/resume, scene authoring, autosave/recovery, hierarchy, prefab, and asset integrity pass | Fail | Project/recovery/session/multi-scene/prefab implementations are documented in [project launcher](../../project-launcher.md), [session recovery](../../session-recovery.md), [multi-scene composition](../../multi-scene-composition.md), and [prefab authoring](../../prefab-authoring.md). Collaborative file safety `#49` and non-blocking native dialogs `#52` passed live acceptance, but `#55` records unguarded native dirty shutdown and a missing clean-history savepoint. Candidate-specific end-to-end reruns also remain. |
|
||||
| G2 | Brush, material, terrain, physics placement, animation, audio, navigation, PIE, and build/package samples pass | Partial | The [five-area sample pack](../sample-regression-pack/) now provides deterministic Brush, Material, Terrain, Physics Placement, and Rendering fixtures with source validation. Animation, audio, navigation, PIE, and build foundations also exist, but the exact-candidate pack rerun, brush acceptance `#37`, and combined workflow evidence remain incomplete. Packaged testing is owner-deferred. |
|
||||
| G3 | Undo/redo/cancel invariants and helper cleanup cover every production mutation path | Partial | Source implementation and focused evidence are complete in [operator regression testing](../../operator-regression-testing.md) and the [operator-invariants evaluation](../operator-invariants/); Gitea `#33` closed at `c55f347`. A clean, exact-candidate rerun is still required for `Pass`. |
|
||||
| G4 | Representative project completes an eight-hour soak without unbounded memory/target growth or unrecoverable failure | Missing | No candidate soak log, resource timeline, failure ledger, or target-growth measurement exists. |
|
||||
| G5 | Cold start, scene open/save, asset refresh, common manipulation, and package-build budgets are documented and measured | Missing | Gitea `#34` is open; no ratified budgets or candidate measurement record exists. |
|
||||
| G6 | Headless content validation and CI are green from a clean checkout | Missing | [CI configuration](../../../../.github/workflows/ci.yml) exists and local source/headless checks have passed during feature work, but no clean-checkout candidate run is linked. The current Gitea server does not expose an Actions run endpoint for this repository. |
|
||||
| G6 | Headless content validation and CI are green from a clean checkout | Missing | [CI configuration](../../../../.github/workflows/ci.yml) hydrates LFS and includes both project and sample content gates. Local implementation checks have passed, but no clean-checkout candidate run is linked. The current Gitea server does not expose an Actions run endpoint for this repository. |
|
||||
| G7 | First-hour UX and recovery QA are signed off by someone other than the implementer | External | No independent sign-off exists. Gitea `#36` remains open; historical H1-H6 implementation-pass notes do not count. |
|
||||
| G8 | Known limitations have severity/workaround and no P0 blocker remains | Fail | Limitations are distributed across feature docs rather than one candidate ledger. The remaining open P0 implementation blocker is `#32`; `#50` cannot close while it remains. Operator invariants `#33`, renderer foundation `#51`, roadmap audit `#35`, Material Library `#16`, targeted drops `#18`, collaborative safety `#49`, and dialog responsiveness `#52` are complete. Property-block application `#53` is P1; dynamic deformed Solari geometry `#54` is a documented P2 limitation with Forward/raster fallback. |
|
||||
| G8 | Known limitations have severity/workaround and no P0 blocker remains | Fail | The sample-pack implementation removes the previously identified `#32` content gap, pending exact-commit native evidence and ticket closure, but guarded native shutdown and clean history savepoints remain the open P0 `#55`. A consolidated candidate limitations ledger is also missing. Deterministic imported-asset fingerprints `#56` and property-block application `#53` are P1; dynamic deformed Solari geometry `#54` is a documented P2 limitation with Forward/raster fallback. |
|
||||
|
||||
## Deliverables
|
||||
|
||||
| ID | Deliverable | State | Evidence or next action |
|
||||
|----|-------------|-------|-------------------------|
|
||||
| D1 | Versioned acceptance matrix under `docs/editor/evaluations/` | Pass | This file, version 0.2. |
|
||||
| D2 | Release-candidate representative project and reproducible validation commands | Missing | Expand the committed project/regression pack through `#32`, including terrain and physics-placement samples, then nominate an exact commit. |
|
||||
| D1 | Versioned acceptance matrix under `docs/editor/evaluations/` | Pass | This file, version 0.3. |
|
||||
| D2 | Release-candidate representative project and reproducible validation commands | Partial | The committed [sample regression pack](../sample-regression-pack/) and commands exist. Nominate an exact commit, rerun from a clean LFS-hydrated checkout, and attach native results. |
|
||||
| D3 | Signed milestone comment linking evidence, limitations, and exact commit | Missing | Post only after G1-G8 pass; no candidate exists yet. |
|
||||
|
||||
## Workflow Coverage
|
||||
@@ -55,11 +55,13 @@ another commit, a dirty worktree, or an older package do not transfer to the can
|
||||
| Scene save, autosave, recovery | Implemented | Not rerun |
|
||||
| Hierarchy parenting and prefab structure | Implemented | Not rerun |
|
||||
| Multi-scene composition | Implemented | Not rerun |
|
||||
| Five-area editor sample pack | Implemented with manifest, native menu, typed scene gate, and headless validation | Source evidence recorded; exact-candidate native run pending |
|
||||
| Brush blockout/edit/CSG | Implemented foundation; `#37` open | Not signed off |
|
||||
| Material catalog and assignment | Renderer/material foundation `#51`, Material Library, and exact targeted drops accepted; optional property-block promotion is `#53` | Not rerun as one candidate |
|
||||
| Terrain authoring | `#22`-`#24` source/live acceptance complete; M3 closed | Pass; packaged-runtime acceptance owner-deferred |
|
||||
| Physics placement | `#25` source/live acceptance complete | Pass; packaged-runtime acceptance owner-deferred |
|
||||
| Collider authoring and diagnostics | `#26` source/live acceptance complete | Pass; packaged-runtime acceptance owner-deferred |
|
||||
| Rendering volumes and look development | Deterministic visible rendering fixture is in the sample pack | Not rerun as one candidate |
|
||||
| Terrain authoring | `#22`-`#24` source/live acceptance complete; M3 closed | Feature-level live acceptance exists; not rerun as one candidate. Packaged-runtime acceptance owner-deferred. |
|
||||
| Physics placement | `#25` source/live acceptance complete | Feature-level live acceptance exists; not rerun as one candidate. Packaged-runtime acceptance owner-deferred. |
|
||||
| Collider authoring and diagnostics | `#26` source/live acceptance complete | Feature-level live acceptance exists; not rerun as one candidate. Packaged-runtime acceptance owner-deferred. |
|
||||
| Animation | Implemented | Not rerun |
|
||||
| Audio | Implemented | Not rerun |
|
||||
| Navigation | Implemented | Not rerun |
|
||||
@@ -81,6 +83,7 @@ cargo clippy --workspace --all-targets -- -D warnings
|
||||
cargo clippy -p editor -p game --bins --features dev,hot-reload -- -D warnings
|
||||
cargo test --workspace
|
||||
cargo validate-levels --project .
|
||||
cargo validate-samples --project .
|
||||
cargo bake-navigation --project . --check
|
||||
```
|
||||
|
||||
@@ -92,8 +95,9 @@ package hash verification, and packaged-runtime smoke protocol used for the cand
|
||||
The final candidate must run for eight continuous hours against the representative project. The
|
||||
record must sample process RSS, GPU memory, target-directory size, recovery generations, background
|
||||
job counts, and error/warning totals at least every five minutes. Exercise scene switching, save and
|
||||
recovery, asset refresh, material editing, brush/terrain/physics tools, animation/audio/navigation,
|
||||
PIE transitions, and build UI cancellation without replacing the candidate during the run.
|
||||
recovery, asset refresh, all five sample-pack scenes, material editing, brush/terrain/physics tools,
|
||||
animation/audio/navigation, PIE transitions, and build UI cancellation without replacing the
|
||||
candidate during the run.
|
||||
|
||||
A pass requires no unrecoverable editor failure, no lost authored work, no unbounded upward trend in
|
||||
steady-state resource use, bounded recovery/cache behavior, and a triaged explanation for every
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
# Sample Regression Pack Evaluation
|
||||
|
||||
**Date:** 2026-07-13
|
||||
|
||||
**Issue:** [Gitea #32](https://git.spacetrainclubhouse.com/Falling-Metal-Interactive/Blacksite/issues/32)
|
||||
|
||||
**Implementation commit:** Pending commit
|
||||
|
||||
**Release-candidate commit:** Not nominated
|
||||
|
||||
**Implementation validation:** **Pass**
|
||||
|
||||
**Native visual acceptance:** **Pending**
|
||||
|
||||
This record separates source/fixture verification from release-candidate acceptance. The committed
|
||||
five-area catalog and scenes are ready for native QA, but results from the implementation worktree
|
||||
do not transfer to a later candidate commit.
|
||||
|
||||
## Source Evidence
|
||||
|
||||
| Check | Result | Evidence |
|
||||
|-------|--------|----------|
|
||||
| Manifest contract | Pass | Schema v1 contains one ordered entry for Brush, Material, Terrain, Physics Placement, and Rendering, with stable IDs and nonempty checks. |
|
||||
| Authored scene fixtures | Pass | All five scene paths exist; actors use stable IDs and each scene contains its required authored feature. |
|
||||
| Project validation | Pass | `cargo validate-levels --project .` reported 78 dependencies, five known nonblocking import/platform findings, and zero blocking errors on 2026-07-13. |
|
||||
| Typed no-GPU deserialization | Pass | `cargo test -p editor every_manifest_sample_typed_deserializes_without_a_gpu --lib`: one passed, zero failed, 256 filtered out. |
|
||||
| Sample-pack validation | Pass | `cargo validate-samples --project .`: five ordered samples, 83 dependencies, five known nonblocking findings, zero blocking errors. |
|
||||
| Clean-checkout CI | Pending | The exact candidate must hydrate LFS and pass both content gates. Current Gitea does not expose an Actions run endpoint for this repository. |
|
||||
|
||||
## Native Protocol
|
||||
|
||||
Open every row through **File > Open Sample**, not through a typed filesystem path. Capture native
|
||||
screenshots only after confirming the viewport is nonblank and correctly framed.
|
||||
|
||||
| Sample | Required native interaction and visual result | Result |
|
||||
|--------|-----------------------------------------------|--------|
|
||||
| Brush Blockout | Foundation, material-face tower, and red subtractive marker are distinct; enter brush edit and cancel one edit or clip preview without leaving helpers | Pending |
|
||||
| Material Lab | Concrete cube, custom sphere, instance block, and emissive panel are visible; perform one exact primitive material drop and undo | Pending |
|
||||
| Terrain Authoring | Hill chunks and two material layers render; perform and cancel one sculpt stroke and one paint stroke | Pending |
|
||||
| Physics Placement | Cuboid, sphere, and capsule settle on the floor; cancel restores all three, then commit/undo behaves as one transaction | Pending |
|
||||
| Rendering Lab | Red/cyan/amber anchors and emissive strip render; camera traversal visibly changes vignette, fog, and exposure ownership | Pending |
|
||||
|
||||
For each row, inspect the status strip, Diagnostics panel, and process log for missing references,
|
||||
hydration errors, render errors, helper leaks, or operator failures. Attach native images to the Gitea
|
||||
issue as ordinary uploaded attachments rather than embedding repository raw URLs. A committed
|
||||
original may remain beside this record through Git LFS under the shared evaluation policy.
|
||||
|
||||
## Release Use
|
||||
|
||||
A future candidate may mark this evaluation fully passed only when the exact nominated commit has:
|
||||
|
||||
1. Passed typed scene deserialization, `validate-levels`, and `validate-samples` from a clean,
|
||||
LFS-hydrated checkout.
|
||||
2. Opened all five entries through the native menu and completed the interactions above.
|
||||
3. Recorded the commit, machine profile, commands, exit codes, logs, and native attachments.
|
||||
Reference in new issue
Block a user