Phase 5: ship layout QA budgets, import hardening, and fixture pack
CI / test (push) Waiting to run

This commit is contained in:
Rbanh committed 2026-02-19 14:00:05 -05:00
1 parent 31a47346ea
commit 8b6c9593e1
17 files changed
+446 -57

No files matched your search

+29
View File
@@ -0,0 +1,29 @@
# Fixture Pack
Schemeta uses deterministic fixture circuits for compile/layout/readability QA.
## Core fixtures
1. `examples/esp32-audio.json`
- Baseline audio pipeline (MCU -> DAC -> amp).
2. `examples/dense-analog.json`
- High-density analog topology used for overlap/crossing stress.
3. `examples/i2c-sensor-hub.json`
- Shared bus topology with pull-ups and interrupt branches.
4. `examples/power-tree.json`
- Multi-domain power-distribution with digital branch nets.
## Usage
1. Manual import in UI:
- `Import JSON` then select fixture file.
2. API:
- `POST /compile` or `POST /analyze` with fixture payload.
3. Automated regression:
- `npm run test:ui` uses `dense-analog` for layout budget checks.
## Notes
1. Fixtures should stay deterministic (stable ordering and explicit naming).
2. Avoid avoidable warnings in default/demo fixtures.
3. Add a fixture whenever a routing/layout bug is fixed to prevent regressions.
+49
View File
@@ -0,0 +1,49 @@
# Phase 5 Execution Plan
Milestone: `Phase 5 - Layout Robustness and JSON Import UX Hardening`
## Goals
1. Make drag/re-layout behavior measurable and regression-resistant.
2. Improve JSON import/apply reliability with validation-first messaging.
3. Ship richer fixture coverage and enforce readability budgets in CI.
## Workstreams
1. QA harness and metrics artifact
- Extend `tests/ui-regression-runner.js` to capture:
- initial compile metrics
- post-drag metrics
- post-drag auto-tidy metrics
- dense fixture metrics
- Emit machine-readable artifact:
- `output/playwright/ui-metrics-report.json`
2. Layout/readability quality budgets
- Enforce thresholds in UI regression runner with env overrides:
- `UI_SAMPLE_MAX_*`
- `UI_DRAG_MAX_*`
- `UI_TIDY_MAX_*`
- `UI_DENSE_MAX_*`
- Keep defaults tuned to current deterministic behavior.
3. JSON import UX hardening
- Validate with `/analyze` before applying/importing JSON.
- Block apply/import when validation has errors.
- Surface first warning/error and path for faster correction.
- Keep deterministic change summary after successful apply.
4. Fixture pack expansion
- Add representative fixtures beyond the baseline audio example:
- `examples/dense-analog.json`
- `examples/i2c-sensor-hub.json`
- `examples/power-tree.json`
- Maintain warning-clean default sample (`frontend/sample.schemeta.json`).
## Done Definition
1. `npm test` passes.
2. `npm run test:ui` passes with quality budgets.
3. `output/playwright/ui-metrics-report.json` is generated on every UI run.
4. Import/apply validation behavior is deterministic and user-visible.
5. Phase 5 issues `#24-#29` are closed and milestone is closed.
+8 -4
View File
@@ -14,10 +14,13 @@ This document defines measurable release gates for Schemeta.
- `npm run test:ui` passes.
2. Visual regression
- No unexpected screenshot diffs in `tests/baselines/ui`.
- Dense analog fixture remains under threshold:
- crossings = `0`
- overlaps = `0`
- detour <= `2.5`
- UI budget thresholds (defaults in `tests/ui-regression-runner.js`) are met:
- sample: crossings <= `1`, overlaps <= `1`, detour <= `3.2`
- drag: crossings <= `3`, overlaps <= `3`, detour <= `3.5`
- drag+tidy: crossings <= `2`, overlaps <= `2`, detour <= `2.0`
- dense fixture: crossings <= `2`, overlaps <= `2`, detour <= `3.0`
- Machine-readable report generated:
- `output/playwright/ui-metrics-report.json`
3. Interaction reliability
- Selection/deselection/isolate/reset flow verified.
- Undo/redo parity verified for component, pin, net, and symbol edits.
@@ -48,6 +51,7 @@ This document defines measurable release gates for Schemeta.
- syntax checks
- `npm test`
- `npm run test:ui`
- verify `output/playwright/ui-metrics-report.json` was produced
- Release candidate required:
- checklist completion in `docs/release-checklist.md`
- intentional baseline updates reviewed and approved
+3 -1
View File
@@ -4,6 +4,7 @@ Use this checklist before cutting a release tag.
Reference docs:
- `docs/quality-gates.md`
- `docs/phase4-execution-plan.md`
- `docs/phase5-execution-plan.md`
## Pre-merge
@@ -30,7 +31,8 @@ Reference docs:
- [ ] Visual baselines updated intentionally (`tests/baselines/ui`) and screenshot diff checks pass.
- [ ] Labels remain legible at common zoom levels.
- [ ] No major overlap/crossing regressions vs previous release baseline.
- [ ] Dense analog fixture meets gate thresholds (`crossings=0`, `overlaps=0`, detour target).
- [ ] UI metrics report generated (`output/playwright/ui-metrics-report.json`).
- [ ] Sample/drag/tidy/dense fixture all meet the thresholds in `docs/quality-gates.md`.
## Security / Operations