Define production readiness acceptance gate

This commit is contained in:
Rbanh committed 2026-07-12 11:29:17 -04:00
1 parent 5a82a9ee29
commit f2ad88fd03
11 files changed
+241

No files matched your search

+3
View File
@@ -79,6 +79,7 @@ Immutable-style log of significant decisions. Add a new numbered ADR when changi
| [editor/collaborative-file-safety.md](editor/collaborative-file-safety.md) | Guarded authored writes, compact Git/read-only status, conflict recovery, and ownership providers |
| [editor/evaluations/material-renderer-foundation/](editor/evaluations/material-renderer-foundation/) | Live screenshots and acceptance results for the renderer/material/component foundation |
| [editor/evaluations/navigation-authoring/](editor/evaluations/navigation-authoring/) | Live screenshots and acceptance results for navigation authoring and runtime parity |
| [editor/evaluations/production-readiness/](editor/evaluations/production-readiness/) | Current versioned production gate, evidence matrix, candidate commands, soak protocol, and independent sign-off requirements |
## Working plans (not canonical long-term)
@@ -98,6 +99,8 @@ Detailed milestone and feature plans live in [`.cursor/plans/`](../.cursor/plans
| `navigation_authoring_*.plan.md` | M7 navigation bounds, deterministic bake, diagnostics, preview, and runtime query API |
| `renderer_material_component_foundation_*.plan.md` | Renderer/material slots, skinned pose lifecycle, Surface ABI, and component foundation |
| `source_control_collaboration_safety_*.plan.md` | Exact authored-file guards, observational Git status, conflict recovery, and provider contract |
| `production_readiness_acceptance_*.plan.md` | Release-candidate evidence matrix, blocker sequence, clean-checkout checks, soak, budgets, and independent sign-off |
| `material_library_and_targeted_drop_*.plan.md` | Dedicated Material Library, exact viewport slot/primitive/brush targeting, hover preview, cancel, and grouped history |
## Crate responsibilities (quick reference)
+1
View File
@@ -26,6 +26,7 @@ Docs for the in-process egui editor (`crates/editor/`). Update this index when a
| [collaborative-file-safety.md](collaborative-file-safety.md) | Exact authored-file revisions, Git/read-only status, conflict recovery, and optional ownership providers |
| [evaluations/material-renderer-foundation/](evaluations/material-renderer-foundation/) | Live screenshots and verification record for the renderer/material/component foundation |
| [evaluations/navigation-authoring/](evaluations/navigation-authoring/) | Live screenshots and verification record for navigation authoring and runtime parity |
| [evaluations/production-readiness/](evaluations/production-readiness/) | Current #50 release-candidate matrix and required clean-checkout, soak, performance, limitation, and independent-signoff evidence |
## Subsystems (code → doc)
+2
View File
@@ -1,5 +1,7 @@
# H1 Gate Sign-Off — Game Development Ready
> Historical baseline only. Current production authority: [production-readiness matrix](production-readiness/).
**Date:** 2026-05-30
**Evaluator:** implementation pass
**Result:** Pass (implementation complete; manual dogfood recommended)
+2
View File
@@ -1,5 +1,7 @@
# H2 Gate Sign-Off — Content Pipeline Maturity
> Historical baseline only. Current production authority: [production-readiness matrix](production-readiness/).
**Date:** 2026-05-30
**Result:** Pass with waivers
+2
View File
@@ -1,5 +1,7 @@
# H3 Gate Sign-Off — Gameplay Authoring
> Historical baseline only. Current production authority: [production-readiness matrix](production-readiness/).
**Date:** 2026-05-30
**Result:** Pass
+2
View File
@@ -1,5 +1,7 @@
# H4 Gate Sign-Off — Multiplayer Tooling
> Historical baseline only. Current production authority: [production-readiness matrix](production-readiness/).
**Date:** 2026-05-30
**Result:** Pass with waivers
+2
View File
@@ -1,5 +1,7 @@
# H5 Gate Sign-Off — Extensibility
> Historical baseline only. Current production authority: [production-readiness matrix](production-readiness/).
**Date:** 2026-05-30
**Result:** Pass
+2
View File
@@ -1,5 +1,7 @@
# H6 Gate Sign-Off — Production / Framework 1.0
> Historical baseline only. Current production authority: [production-readiness matrix](production-readiness/).
**Date:** 2026-05-30
**Result:** Pass with waivers
@@ -0,0 +1,104 @@
# Production-Readiness Acceptance Matrix
**Matrix version:** 0.1
**Last audit:** 2026-07-12
**Release-candidate commit:** Not nominated
**Overall result:** **Not ready**
This is the current authority for Gitea
[`#50`](https://git.spacetrainclubhouse.com/Falling-Metal-Interactive/Blacksite/issues/50).
The older H1-H6 files are historical implementation snapshots and do not satisfy this gate.
## Evidence Rules
| State | Meaning |
|-------|---------|
| Pass | Candidate-specific evidence proves the full row. |
| Partial | Some implementation/evidence exists, but the full row is not proven. |
| Deferred | The project owner has explicitly postponed this check; it is not a pass. |
| Missing | Required implementation or candidate evidence does not exist. |
| Fail | Current authoritative evidence contradicts the gate. |
| External | Requires a person other than the implementer. |
Acceptance evidence must name the exact candidate commit and commands or protocol used. Results from
another commit, a dirty worktree, or an older package do not transfer to the candidate.
## Exit Gate
| ID | Requirement | State | Current evidence and gap |
|----|-------------|-------|--------------------------|
| G1 | Project create/open/resume, scene authoring, autosave/recovery, hierarchy, prefab, and asset integrity pass | Partial | Project/recovery/session/multi-scene/prefab implementations are documented in [project launcher](../../project-launcher.md), [session recovery](../../session-recovery.md), [multi-scene composition](../../multi-scene-composition.md), and [prefab authoring](../../prefab-authoring.md). Candidate-specific end-to-end reruns and the open asset-integrity work remain. |
| G2 | Brush, material, terrain, physics placement, animation, audio, navigation, PIE, and build/package samples pass | Fail | Animation, audio, navigation, and build foundations are implemented. Material/drag-drop tickets `#16`, `#18`, and `#51` remain open; terrain `#22`-`#24` and physics placement `#25` are not complete; brush acceptance `#37` remains open. Packaged testing is owner-deferred. |
| G3 | Undo/redo/cancel invariants and helper cleanup cover every production mutation path | Partial | The reusable harness and current tool coverage are documented in [operator regression testing](../../operator-regression-testing.md). Gitea `#33` remains open for terrain-stroke and physics-settle fixtures after those tools exist. |
| G4 | Representative project completes an eight-hour soak without unbounded memory/target growth or unrecoverable failure | Missing | No candidate soak log, resource timeline, failure ledger, or target-growth measurement exists. |
| G5 | Cold start, scene open/save, asset refresh, common manipulation, and package-build budgets are documented and measured | Missing | Gitea `#34` is open; no ratified budgets or candidate measurement record exists. |
| G6 | Headless content validation and CI are green from a clean checkout | Missing | [CI configuration](../../../../.github/workflows/ci.yml) exists and local source/headless checks have passed during feature work, but no clean-checkout candidate run is linked. The current Gitea server does not expose an Actions run endpoint for this repository. |
| G7 | First-hour UX and recovery QA are signed off by someone other than the implementer | External | No independent sign-off exists. Gitea `#36` remains open; historical H1-H6 implementation-pass notes do not count. |
| G8 | Known limitations have severity/workaround and no P0 blocker remains | Fail | Limitations are distributed across feature docs rather than one candidate ledger. Open P0 work includes `#16`, `#18`, `#32`, `#33`, `#35`, and `#51`; `#50` cannot close while a gate-relevant P0 remains. |
## Deliverables
| ID | Deliverable | State | Evidence or next action |
|----|-------------|-------|-------------------------|
| D1 | Versioned acceptance matrix under `docs/editor/evaluations/` | Pass | This file, version 0.1. |
| D2 | Release-candidate representative project and reproducible validation commands | Missing | Expand the committed project/regression pack through `#32`, including terrain and physics-placement samples, then nominate an exact commit. |
| D3 | Signed milestone comment linking evidence, limitations, and exact commit | Missing | Post only after G1-G8 pass; no candidate exists yet. |
## Workflow Coverage
| Workflow | Implementation status | Candidate acceptance |
|----------|-----------------------|----------------------|
| Project create/open/resume | Implemented | Not rerun |
| Scene save, autosave, recovery | Implemented | Not rerun |
| Hierarchy parenting and prefab structure | Implemented | Not rerun |
| Multi-scene composition | Implemented | Not rerun |
| Brush blockout/edit/CSG | Implemented foundation; `#37` open | Not signed off |
| Material catalog and assignment | Partial; `#16`, `#18`, `#51` open | Blocked |
| Terrain authoring | `#22`-`#24` open | Blocked |
| Physics placement | `#25` open | Blocked |
| Animation | Implemented | Not rerun |
| Audio | Implemented | Not rerun |
| Navigation | Implemented | Not rerun |
| PIE | Implemented | Not rerun |
| Build/package/runtime | Implemented foundation | Deferred by owner |
| Collaborative file safety | Local source/headless acceptance; `#49` open | Awaiting publication and live visual QA |
## Candidate Validation Commands
Run these from a fresh clone of the nominated commit with Git LFS content present. Record wall time,
machine profile, exit code, and artifact hashes. Packaged commands stay disabled until the project
owner re-enables them.
```bash
cargo fmt --all -- --check
cargo check --workspace --all-targets
cargo clippy --workspace --all-targets -- -D warnings
cargo clippy -p editor -p game --bins --features dev,hot-reload -- -D warnings
cargo test --workspace
cargo validate-levels --project .
cargo bake-navigation --project . --check
```
When packaged acceptance is re-enabled, append the exact `package-project` profile/target commands,
package hash verification, and packaged-runtime smoke protocol used for the candidate.
## Soak Protocol
The final candidate must run for eight continuous hours against the representative project. The
record must sample process RSS, GPU memory, target-directory size, recovery generations, background
job counts, and error/warning totals at least every five minutes. Exercise scene switching, save and
recovery, asset refresh, material editing, brush/terrain/physics tools, animation/audio/navigation,
PIE transitions, and build UI cancellation without replacing the candidate during the run.
A pass requires no unrecoverable editor failure, no lost authored work, no unbounded upward trend in
steady-state resource use, bounded recovery/cache behavior, and a triaged explanation for every
error. Numeric thresholds remain blocked on the performance-budget work in `#34`.
## Independent Sign-Off
The evaluator must not be the implementation agent. They record the candidate commit, machine,
elapsed first-hour workflow, recovery scenario, defects, severity, and go/no-go decision. Waivers
must include a workaround and owner ticket; an open P0 cannot be waived.