schemeta/docs/quality-gates.md
Rbanh cd007ba431
Some checks are pending
CI / test (push) Waiting to run
Improve layout fallback flow and stabilize QA baselines
2026-02-19 18:33:14 -05:00

2.0 KiB

Quality Gates

This document defines measurable release gates for Schemeta.

Release Stages

  • Beta: feature complete, controlled rollout quality.
  • GA: production-grade quality, accessibility, and operational readiness.

Beta Gates

  1. Functional correctness
    • npm test passes.
    • npm run test:ui passes.
  2. Visual regression
    • No unexpected screenshot diffs in tests/baselines/ui.
    • UI budget thresholds (defaults in tests/ui-regression-runner.js) are met:
      • sample: crossings <= 1, overlaps <= 1, detour <= 3.6
      • drag: crossings <= 3, overlaps <= 3, detour <= 3.5
      • drag+tidy: crossings <= 2, overlaps <= 2, detour <= 4.5
      • dense fixture: crossings <= 2, overlaps <= 2, detour <= 4.2
    • Machine-readable report generated:
      • output/playwright/ui-metrics-report.json
  3. Interaction reliability
    • Selection/deselection/isolate/reset flow verified.
    • Undo/redo parity verified for component, pin, net, and symbol edits.
  4. API/MCP compatibility
    • api_version/schema_version values present and documented.
    • Compile/analyze envelopes remain backward compatible (additive policy).

GA Gates

  1. Accessibility
    • Core workflows keyboard-operable.
    • WCAG AA contrast on primary controls/text.
  2. Responsive UX
    • Usable at 1280x720 and 1440x900 without clipping critical controls.
  3. Schematic readability
    • No component-wire overlaps from auto layout/routing.
    • Label overlap count near-zero on benchmark fixtures.
  4. Operational readiness
    • Rate limiting + auth + health checks validated.
    • Error telemetry includes structured codes and context.
  5. Data durability
    • Save/load/restore and JSON roundtrip validated.
    • MCP UI bundle + tool endpoints verified end-to-end.

CI Mapping

  • Always required:
    • syntax checks
    • npm test
    • npm run test:ui
    • verify output/playwright/ui-metrics-report.json was produced
  • Release candidate required:
    • checklist completion in docs/release-checklist.md
    • intentional baseline updates reviewed and approved