Blacksite/docs/editor/evaluations/production-readiness
Rbanh 9a67a25a15
Some checks are pending
CI / Format, lint, test, build (push) Waiting to run
Clarify renderer foundation follow-ups
2026-07-12 23:01:02 -04:00
..
README.md Clarify renderer foundation follow-ups 2026-07-12 23:01:02 -04:00

Production-Readiness Acceptance Matrix

Matrix version: 0.2

Last audit: 2026-07-12

Release-candidate commit: Not nominated

Overall result: Not ready

This is the current authority for Gitea #50. The older H1-H6 files are historical implementation snapshots and do not satisfy this gate.

Evidence Rules

State Meaning
Pass Candidate-specific evidence proves the full row.
Partial Some implementation/evidence exists, but the full row is not proven.
Deferred The project owner has explicitly postponed this check; it is not a pass.
Missing Required implementation or candidate evidence does not exist.
Fail Current authoritative evidence contradicts the gate.
External Requires a person other than the implementer.

Acceptance evidence must name the exact candidate commit and commands or protocol used. Results from another commit, a dirty worktree, or an older package do not transfer to the candidate.

Exit Gate

ID Requirement State Current evidence and gap
G1 Project create/open/resume, scene authoring, autosave/recovery, hierarchy, prefab, and asset integrity pass Partial Project/recovery/session/multi-scene/prefab implementations are documented in project launcher, session recovery, multi-scene composition, and prefab authoring. Collaborative file safety #49 and non-blocking native dialogs #52 passed live acceptance. Candidate-specific end-to-end reruns remain.
G2 Brush, material, terrain, physics placement, animation, audio, navigation, PIE, and build/package samples pass Fail Animation, audio, navigation, renderer/material foundation #51, Material Library, targeted material drops, terrain #22-#24, physics placement #25, and build foundations are implemented with source/live fixtures. Brush acceptance #37 remains incomplete. Packaged testing is owner-deferred.
G3 Undo/redo/cancel invariants and helper cleanup cover every production mutation path Partial The reusable harness and current tool coverage are documented in operator regression testing. Terrain stroke and physics settle/cancel/undo fixtures now exist; Gitea #33 remains open for full mutation-path coverage.
G4 Representative project completes an eight-hour soak without unbounded memory/target growth or unrecoverable failure Missing No candidate soak log, resource timeline, failure ledger, or target-growth measurement exists.
G5 Cold start, scene open/save, asset refresh, common manipulation, and package-build budgets are documented and measured Missing Gitea #34 is open; no ratified budgets or candidate measurement record exists.
G6 Headless content validation and CI are green from a clean checkout Missing CI configuration exists and local source/headless checks have passed during feature work, but no clean-checkout candidate run is linked. The current Gitea server does not expose an Actions run endpoint for this repository.
G7 First-hour UX and recovery QA are signed off by someone other than the implementer External No independent sign-off exists. Gitea #36 remains open; historical H1-H6 implementation-pass notes do not count.
G8 Known limitations have severity/workaround and no P0 blocker remains Fail Limitations are distributed across feature docs rather than one candidate ledger. Open P0 work includes #32 and #33; #50 cannot close while a gate-relevant P0 remains. Renderer foundation #51, roadmap audit #35, Material Library #16, targeted drops #18, collaborative safety #49, and dialog responsiveness #52 are closed. Property-block application #53 is P1; dynamic deformed Solari geometry #54 is a documented P2 limitation with Forward/raster fallback.

Deliverables

ID Deliverable State Evidence or next action
D1 Versioned acceptance matrix under docs/editor/evaluations/ Pass This file, version 0.2.
D2 Release-candidate representative project and reproducible validation commands Missing Expand the committed project/regression pack through #32, including terrain and physics-placement samples, then nominate an exact commit.
D3 Signed milestone comment linking evidence, limitations, and exact commit Missing Post only after G1-G8 pass; no candidate exists yet.

Workflow Coverage

Workflow Implementation status Candidate acceptance
Project create/open/resume Implemented Not rerun
Scene save, autosave, recovery Implemented Not rerun
Hierarchy parenting and prefab structure Implemented Not rerun
Multi-scene composition Implemented Not rerun
Brush blockout/edit/CSG Implemented foundation; #37 open Not signed off
Material catalog and assignment Renderer/material foundation #51, Material Library, and exact targeted drops accepted; optional property-block promotion is #53 Not rerun as one candidate
Terrain authoring #22-#24 source/live acceptance complete; M3 closed Pass; packaged-runtime acceptance owner-deferred
Physics placement #25 source/live acceptance complete Pass; packaged-runtime acceptance owner-deferred
Collider authoring and diagnostics #26 source/live acceptance complete Pass; packaged-runtime acceptance owner-deferred
Animation Implemented Not rerun
Audio Implemented Not rerun
Navigation Implemented Not rerun
PIE Implemented Not rerun
Build/package/runtime Implemented foundation Deferred by owner
Collaborative file safety Implemented; #49 closed with live external-change acceptance Not rerun as one candidate
Native file/folder dialogs Implemented; #52 closed with live 12-second compositor-responsiveness hold Not rerun as one candidate

Candidate Validation Commands

Run these from a fresh clone of the nominated commit with Git LFS content present. Record wall time, machine profile, exit code, and artifact hashes. Packaged commands stay disabled until the project owner re-enables them.

cargo fmt --all -- --check
cargo check --workspace --all-targets
cargo clippy --workspace --all-targets -- -D warnings
cargo clippy -p editor -p game --bins --features dev,hot-reload -- -D warnings
cargo test --workspace
cargo validate-levels --project .
cargo bake-navigation --project . --check

When packaged acceptance is re-enabled, append the exact package-project profile/target commands, package hash verification, and packaged-runtime smoke protocol used for the candidate.

Soak Protocol

The final candidate must run for eight continuous hours against the representative project. The record must sample process RSS, GPU memory, target-directory size, recovery generations, background job counts, and error/warning totals at least every five minutes. Exercise scene switching, save and recovery, asset refresh, material editing, brush/terrain/physics tools, animation/audio/navigation, PIE transitions, and build UI cancellation without replacing the candidate during the run.

A pass requires no unrecoverable editor failure, no lost authored work, no unbounded upward trend in steady-state resource use, bounded recovery/cache behavior, and a triaged explanation for every error. Numeric thresholds remain blocked on the performance-budget work in #34.

Independent Sign-Off

The evaluator must not be the implementation agent. They record the candidate commit, machine, elapsed first-hour workflow, recovery scenario, defects, severity, and go/no-go decision. Waivers must include a workaround and owner ticket; an open P0 cannot be waived.