Record sample pack native acceptance
Some checks are pending
CI / Format, lint, test, build (push) Waiting to run

This commit is contained in:
Rbanh 2026-07-13 04:18:56 -04:00
parent d52cc2e3b7
commit 62b538999d
4 changed files with 37 additions and 21 deletions

View File

@ -93,7 +93,7 @@ Immutable-style log of significant decisions. Add a new numbered ADR when changi
| [editor/evaluations/collider-diagnostics/](editor/evaluations/collider-diagnostics/) | Live screenshot and acceptance results for collider overlays, diagnostics, shape history, and placement preflight | | [editor/evaluations/collider-diagnostics/](editor/evaluations/collider-diagnostics/) | Live screenshot and acceptance results for collider overlays, diagnostics, shape history, and placement preflight |
| [editor/evaluations/navigation-authoring/](editor/evaluations/navigation-authoring/) | Live screenshots and acceptance results for navigation authoring and runtime parity | | [editor/evaluations/navigation-authoring/](editor/evaluations/navigation-authoring/) | Live screenshots and acceptance results for navigation authoring and runtime parity |
| [editor/evaluations/operator-invariants/](editor/evaluations/operator-invariants/) | Source acceptance results for production operator lifecycle, rollback, cleanup, and undo/redo invariants | | [editor/evaluations/operator-invariants/](editor/evaluations/operator-invariants/) | Source acceptance results for production operator lifecycle, rollback, cleanup, and undo/redo invariants |
| [editor/evaluations/sample-regression-pack/](editor/evaluations/sample-regression-pack/) | Source evidence and native acceptance protocol for the editor sample regression pack | | [editor/evaluations/sample-regression-pack/](editor/evaluations/sample-regression-pack/) | Exact-implementation source and native acceptance evidence for the editor sample regression pack |
| [editor/evaluations/production-readiness/](editor/evaluations/production-readiness/) | Current versioned production gate, evidence matrix, candidate commands, soak protocol, and independent sign-off requirements | | [editor/evaluations/production-readiness/](editor/evaluations/production-readiness/) | Current versioned production gate, evidence matrix, candidate commands, soak protocol, and independent sign-off requirements |
## Working plans (not canonical long-term) ## Working plans (not canonical long-term)

View File

@ -41,7 +41,7 @@ Docs for the in-process egui editor (`crates/editor/`). Update this index when a
| [evaluations/collider-diagnostics/](evaluations/collider-diagnostics/) | Live screenshot and verification record for semantic collider overlays, health linking, shape switching, and placement reuse | | [evaluations/collider-diagnostics/](evaluations/collider-diagnostics/) | Live screenshot and verification record for semantic collider overlays, health linking, shape switching, and placement reuse |
| [evaluations/navigation-authoring/](evaluations/navigation-authoring/) | Live screenshots and verification record for navigation authoring and runtime parity | | [evaluations/navigation-authoring/](evaluations/navigation-authoring/) | Live screenshots and verification record for navigation authoring and runtime parity |
| [evaluations/operator-invariants/](evaluations/operator-invariants/) | Source acceptance record for production operator lifecycle, rollback, cleanup, and undo/redo invariants | | [evaluations/operator-invariants/](evaluations/operator-invariants/) | Source acceptance record for production operator lifecycle, rollback, cleanup, and undo/redo invariants |
| [evaluations/sample-regression-pack/](evaluations/sample-regression-pack/) | Source evidence and pending native protocol for the five-area editor regression pack | | [evaluations/sample-regression-pack/](evaluations/sample-regression-pack/) | Exact-implementation source and native acceptance evidence for the five-area editor regression pack |
| [evaluations/production-readiness/](evaluations/production-readiness/) | Current #50 release-candidate matrix and required clean-checkout, soak, performance, limitation, and independent-signoff evidence | | [evaluations/production-readiness/](evaluations/production-readiness/) | Current #50 release-candidate matrix and required clean-checkout, soak, performance, limitation, and independent-signoff evidence |
## Subsystems (code → doc) ## Subsystems (code → doc)

View File

@ -1,6 +1,6 @@
# Production-Readiness Acceptance Matrix # Production-Readiness Acceptance Matrix
**Matrix version:** 0.3 **Matrix version:** 0.4
**Last audit:** 2026-07-13 **Last audit:** 2026-07-13
@ -31,20 +31,20 @@ another commit, a dirty worktree, or an older package do not transfer to the can
| ID | Requirement | State | Current evidence and gap | | ID | Requirement | State | Current evidence and gap |
|----|-------------|-------|--------------------------| |----|-------------|-------|--------------------------|
| G1 | Project create/open/resume, scene authoring, autosave/recovery, hierarchy, prefab, and asset integrity pass | Fail | Project/recovery/session/multi-scene/prefab implementations are documented in [project launcher](../../project-launcher.md), [session recovery](../../session-recovery.md), [multi-scene composition](../../multi-scene-composition.md), and [prefab authoring](../../prefab-authoring.md). Collaborative file safety `#49` and non-blocking native dialogs `#52` passed live acceptance, but `#55` records unguarded native dirty shutdown and a missing clean-history savepoint. Candidate-specific end-to-end reruns also remain. | | G1 | Project create/open/resume, scene authoring, autosave/recovery, hierarchy, prefab, and asset integrity pass | Fail | Project/recovery/session/multi-scene/prefab implementations are documented in [project launcher](../../project-launcher.md), [session recovery](../../session-recovery.md), [multi-scene composition](../../multi-scene-composition.md), and [prefab authoring](../../prefab-authoring.md). Collaborative file safety `#49` and non-blocking native dialogs `#52` passed live acceptance, but `#55` records unguarded native dirty shutdown and a missing clean-history savepoint. Candidate-specific end-to-end reruns also remain. |
| G2 | Brush, material, terrain, physics placement, animation, audio, navigation, PIE, and build/package samples pass | Partial | The [five-area sample pack](../sample-regression-pack/) now provides deterministic Brush, Material, Terrain, Physics Placement, and Rendering fixtures with source validation. Animation, audio, navigation, PIE, and build foundations also exist, but the exact-candidate pack rerun, brush acceptance `#37`, and combined workflow evidence remain incomplete. Packaged testing is owner-deferred. | | G2 | Brush, material, terrain, physics placement, animation, audio, navigation, PIE, and build/package samples pass | Partial | The [five-area sample pack](../sample-regression-pack/) passed source and native acceptance on exact implementation commit `d52cc2e`, including Brush, Material, Terrain, Physics Placement, and Rendering. No release candidate is nominated, brush acceptance `#37` remains open, and combined animation/audio/navigation/PIE/build evidence is incomplete. Packaged testing is owner-deferred. |
| G3 | Undo/redo/cancel invariants and helper cleanup cover every production mutation path | Partial | Source implementation and focused evidence are complete in [operator regression testing](../../operator-regression-testing.md) and the [operator-invariants evaluation](../operator-invariants/); Gitea `#33` closed at `c55f347`. A clean, exact-candidate rerun is still required for `Pass`. | | G3 | Undo/redo/cancel invariants and helper cleanup cover every production mutation path | Partial | Source implementation and focused evidence are complete in [operator regression testing](../../operator-regression-testing.md) and the [operator-invariants evaluation](../operator-invariants/); Gitea `#33` closed at `c55f347`. A clean, exact-candidate rerun is still required for `Pass`. |
| G4 | Representative project completes an eight-hour soak without unbounded memory/target growth or unrecoverable failure | Missing | No candidate soak log, resource timeline, failure ledger, or target-growth measurement exists. | | G4 | Representative project completes an eight-hour soak without unbounded memory/target growth or unrecoverable failure | Missing | No candidate soak log, resource timeline, failure ledger, or target-growth measurement exists. |
| G5 | Cold start, scene open/save, asset refresh, common manipulation, and package-build budgets are documented and measured | Missing | Gitea `#34` is open; no ratified budgets or candidate measurement record exists. | | G5 | Cold start, scene open/save, asset refresh, common manipulation, and package-build budgets are documented and measured | Missing | Gitea `#34` is open; no ratified budgets or candidate measurement record exists. |
| G6 | Headless content validation and CI are green from a clean checkout | Missing | [CI configuration](../../../../.github/workflows/ci.yml) hydrates LFS and includes both project and sample content gates. Local implementation checks have passed, but no clean-checkout candidate run is linked. The current Gitea server does not expose an Actions run endpoint for this repository. | | G6 | Headless content validation and CI are green from a clean checkout | Missing | [CI configuration](../../../../.github/workflows/ci.yml) hydrates LFS and includes both project and sample content gates. Local implementation checks have passed, but no clean-checkout candidate run is linked. The current Gitea server does not expose an Actions run endpoint for this repository. |
| G7 | First-hour UX and recovery QA are signed off by someone other than the implementer | External | No independent sign-off exists. Gitea `#36` remains open; historical H1-H6 implementation-pass notes do not count. | | G7 | First-hour UX and recovery QA are signed off by someone other than the implementer | External | No independent sign-off exists. Gitea `#36` remains open; historical H1-H6 implementation-pass notes do not count. |
| G8 | Known limitations have severity/workaround and no P0 blocker remains | Fail | The sample-pack implementation removes the previously identified `#32` content gap, pending exact-commit native evidence and ticket closure, but guarded native shutdown and clean history savepoints remain the open P0 `#55`. A consolidated candidate limitations ledger is also missing. Deterministic imported-asset fingerprints `#56` and property-block application `#53` are P1; dynamic deformed Solari geometry `#54` is a documented P2 limitation with Forward/raster fallback. | | G8 | Known limitations have severity/workaround and no P0 blocker remains | Fail | Sample-pack gap `#32` has exact-implementation source/native evidence at `d52cc2e`, but guarded native shutdown and clean history savepoints remain the open P0 `#55`. A consolidated candidate limitations ledger is also missing. Deterministic imported-asset fingerprints `#56` and property-block application `#53` are P1; dynamic deformed Solari geometry `#54` is a documented P2 limitation with Forward/raster fallback. |
## Deliverables ## Deliverables
| ID | Deliverable | State | Evidence or next action | | ID | Deliverable | State | Evidence or next action |
|----|-------------|-------|-------------------------| |----|-------------|-------|-------------------------|
| D1 | Versioned acceptance matrix under `docs/editor/evaluations/` | Pass | This file, version 0.3. | | D1 | Versioned acceptance matrix under `docs/editor/evaluations/` | Pass | This file, version 0.4. |
| D2 | Release-candidate representative project and reproducible validation commands | Partial | The committed [sample regression pack](../sample-regression-pack/) and commands exist. Nominate an exact commit, rerun from a clean LFS-hydrated checkout, and attach native results. | | D2 | Release-candidate representative project and reproducible validation commands | Partial | The committed [sample regression pack](../sample-regression-pack/), commands, and exact-implementation native evidence exist. Nominate a release candidate and rerun from a clean LFS-hydrated checkout. |
| D3 | Signed milestone comment linking evidence, limitations, and exact commit | Missing | Post only after G1-G8 pass; no candidate exists yet. | | D3 | Signed milestone comment linking evidence, limitations, and exact commit | Missing | Post only after G1-G8 pass; no candidate exists yet. |
## Workflow Coverage ## Workflow Coverage
@ -55,7 +55,7 @@ another commit, a dirty worktree, or an older package do not transfer to the can
| Scene save, autosave, recovery | Implemented | Not rerun | | Scene save, autosave, recovery | Implemented | Not rerun |
| Hierarchy parenting and prefab structure | Implemented | Not rerun | | Hierarchy parenting and prefab structure | Implemented | Not rerun |
| Multi-scene composition | Implemented | Not rerun | | Multi-scene composition | Implemented | Not rerun |
| Five-area editor sample pack | Implemented with manifest, native menu, typed scene gate, and headless validation | Source evidence recorded; exact-candidate native run pending | | Five-area editor sample pack | Implemented with manifest, native menu, typed scene gate, and headless validation | Exact-implementation source/native acceptance passed at `d52cc2e`; not rerun as one release candidate |
| Brush blockout/edit/CSG | Implemented foundation; `#37` open | Not signed off | | Brush blockout/edit/CSG | Implemented foundation; `#37` open | Not signed off |
| Material catalog and assignment | Renderer/material foundation `#51`, Material Library, and exact targeted drops accepted; optional property-block promotion is `#53` | Not rerun as one candidate | | Material catalog and assignment | Renderer/material foundation `#51`, Material Library, and exact targeted drops accepted; optional property-block promotion is `#53` | Not rerun as one candidate |
| Rendering volumes and look development | Deterministic visible rendering fixture is in the sample pack | Not rerun as one candidate | | Rendering volumes and look development | Deterministic visible rendering fixture is in the sample pack | Not rerun as one candidate |

View File

@ -4,17 +4,17 @@
**Issue:** [Gitea #32](https://git.spacetrainclubhouse.com/Falling-Metal-Interactive/Blacksite/issues/32) **Issue:** [Gitea #32](https://git.spacetrainclubhouse.com/Falling-Metal-Interactive/Blacksite/issues/32)
**Implementation commit:** Pending commit **Implementation commit:** `d52cc2e3b72c908d192ed9e9f49306c25367599b`
**Release-candidate commit:** Not nominated **Release-candidate commit:** Not nominated
**Implementation validation:** **Pass** **Implementation validation:** **Pass**
**Native visual acceptance:** **Pending** **Native visual acceptance:** **Pass**
This record separates source/fixture verification from release-candidate acceptance. The committed This record separates source/fixture verification from release-candidate acceptance. The exact
five-area catalog and scenes are ready for native QA, but results from the implementation worktree implementation commit above passed the five-area native protocol on a clean tracked worktree. These
do not transfer to a later candidate commit. results do not transfer to a later release-candidate commit.
## Source Evidence ## Source Evidence
@ -22,9 +22,9 @@ do not transfer to a later candidate commit.
|-------|--------|----------| |-------|--------|----------|
| Manifest contract | Pass | Schema v1 contains one ordered entry for Brush, Material, Terrain, Physics Placement, and Rendering, with stable IDs and nonempty checks. | | Manifest contract | Pass | Schema v1 contains one ordered entry for Brush, Material, Terrain, Physics Placement, and Rendering, with stable IDs and nonempty checks. |
| Authored scene fixtures | Pass | All five scene paths exist; actors use stable IDs and each scene contains its required authored feature. | | Authored scene fixtures | Pass | All five scene paths exist; actors use stable IDs and each scene contains its required authored feature. |
| Project validation | Pass | `cargo validate-levels --project .` reported 78 dependencies, five known nonblocking import/platform findings, and zero blocking errors on 2026-07-13. | | Project validation | Pass | `cargo validate-levels --project .` reported 81 dependencies, five known nonblocking import/platform findings, and zero blocking errors on 2026-07-13. |
| Typed no-GPU deserialization | Pass | `cargo test -p editor every_manifest_sample_typed_deserializes_without_a_gpu --lib`: one passed, zero failed, 256 filtered out. | | Typed no-GPU deserialization | Pass | `cargo test -p editor every_manifest_sample_typed_deserializes_without_a_gpu --lib`: one passed, zero failed, 257 filtered out. |
| Sample-pack validation | Pass | `cargo validate-samples --project .`: five ordered samples, 83 dependencies, five known nonblocking findings, zero blocking errors. | | Sample-pack validation | Pass | `cargo validate-samples --project .`: five ordered samples, 86 dependencies, five known nonblocking findings, zero blocking errors. |
| Clean-checkout CI | Pending | The exact candidate must hydrate LFS and pass both content gates. Current Gitea does not expose an Actions run endpoint for this repository. | | Clean-checkout CI | Pending | The exact candidate must hydrate LFS and pass both content gates. Current Gitea does not expose an Actions run endpoint for this repository. |
## Native Protocol ## Native Protocol
@ -34,17 +34,33 @@ screenshots only after confirming the viewport is nonblank and correctly framed.
| Sample | Required native interaction and visual result | Result | | Sample | Required native interaction and visual result | Result |
|--------|-----------------------------------------------|--------| |--------|-----------------------------------------------|--------|
| Brush Blockout | Foundation, material-face tower, and red subtractive marker are distinct; enter brush edit and cancel one edit or clip preview without leaving helpers | Pending | | Brush Blockout | Foundation, material-face tower, and red subtractive marker are distinct; enter brush edit and cancel one edit or clip preview without leaving helpers | Pass - selected one tower face in Brush Clip, observed the magenta preview, then canceled with no history or helper leak. |
| Material Lab | Concrete cube, custom sphere, instance block, and emissive panel are visible; perform one exact primitive material drop and undo | Pending | | Material Lab | Concrete cube, custom sphere, instance block, and emissive panel are visible; perform one exact primitive material drop and undo | Pass - dropped `surface_tint_instance` on `Concrete Reference / Surface`; one history entry was created and Ctrl+Z restored the exact prior material. |
| Terrain Authoring | Hill chunks and two material layers render; perform and cancel one sculpt stroke and one paint stroke | Pending | | Terrain Authoring | Hill chunks and two material layers render; perform and cancel one sculpt stroke and one paint stroke | Pass - Concrete/Surface Tint rendered visibly under active Solari through the forward raster boundary; both stroke cancels restored data and left history clean. |
| Physics Placement | Cuboid, sphere, and capsule settle on the floor; cancel restores all three, then commit/undo behaves as one transaction | Pending | | Physics Placement | Cuboid, sphere, and capsule settle on the floor; cancel restores all three, then commit/undo behaves as one transaction | Pass - three props settled in 2.3 seconds; cancel restored the airborne transforms, and the second run committed one `Move Selection` entry that Ctrl+Z restored. |
| Rendering Lab | Red/cyan/amber anchors and emissive strip render; camera traversal visibly changes vignette, fog, and exposure ownership | Pending | | Rendering Lab | Red/cyan/amber anchors and emissive strip render; camera traversal visibly changes vignette, fog, and exposure ownership | Pass - HUD ownership reached Foggy courtyard 73%/2 overrides, Cave mouth 100%/2 overrides, and Vignette demo 68% with the expected visible changes. |
For each row, inspect the status strip, Diagnostics panel, and process log for missing references, For each row, inspect the status strip, Diagnostics panel, and process log for missing references,
hydration errors, render errors, helper leaks, or operator failures. Attach native images to the Gitea hydration errors, render errors, helper leaks, or operator failures. Attach native images to the Gitea
issue as ordinary uploaded attachments rather than embedding repository raw URLs. A committed issue as ordinary uploaded attachments rather than embedding repository raw URLs. A committed
original may remain beside this record through Git LFS under the shared evaluation policy. original may remain beside this record through Git LFS under the shared evaluation policy.
## Native Environment And Artifacts
- Arch Linux `7.1.3-arch1-1`, Hyprland `0.55.4`, native Wayland.
- NVIDIA GeForce RTX 3080 Ti, driver `610.43.03`.
- Bevy `0.19.0`; the viewport reported the effective `Solari` path throughout the pass.
- Exact binary: `target/debug/editor --project .`, built from the implementation commit above.
- Process log: no warnings or errors across all five menu opens and interactions.
- [Gitea issue comment #1057](https://git.spacetrainclubhouse.com/Falling-Metal-Interactive/Blacksite/issues/32#issuecomment-1057)
embeds full-resolution 3426x1384 PNG uploads, not repository/LFS raw links:
`blacksite-brush-d52cc2e.png`
(`sha256:3853bb51a70320b95af2cc48a1609251defc22705fb5f27fd3b1ab434f71877f`),
`blacksite-terrain-solari-d52cc2e.png`
(`sha256:8ee4e5d5c40436a5719fbec34deaef10f5bcb79748b052268781c29468e1ef7b`), and
`blacksite-rendering-vignette-d52cc2e.png`
(`sha256:67a09743f86439adf98ad2cf9b9d91041f89954aea2b8f29c29bbf10ead8c000`).
## Release Use ## Release Use
A future candidate may mark this evaluation fully passed only when the exact nominated commit has: A future candidate may mark this evaluation fully passed only when the exact nominated commit has: