diff --git a/.cursor/plans/deterministic_asset_fingerprints_2026-07-13.plan.md b/.cursor/plans/deterministic_asset_fingerprints_2026-07-13.plan.md new file mode 100644 index 0000000..13bf4d1 --- /dev/null +++ b/.cursor/plans/deterministic_asset_fingerprints_2026-07-13.plan.md @@ -0,0 +1,35 @@ +# Deterministic Imported-Asset Fingerprints + +## Scope + +Implement Gitea #56 so imported-source identity is content-addressed and editor/validator runs do +not rewrite project artifacts because checkout timestamps or serialization whitespace changed. + +## Contract + +- Persist one shared fingerprint for imported model, texture, and audio sources: byte length plus a + BLAKE3 digest of the exact source bytes. +- Do not persist or compare source mtimes as semantic identity. Legacy timestamp fields remain + readable but are ignored. +- Static-mesh and animation manifests use the same content identity as the asset registry. +- Preserve stable registry UUIDs across metadata-only drift and uniquely identifiable source moves. +- Before writing a registry or generated manifest, compare the parsed semantic document; preserve + the existing bytes when the result is equivalent. +- Project validators stay read-only and validate both length and digest, including same-size edits. + +## Implementation + +1. Add the shared fingerprint type and migrate animation/static-mesh contracts. +2. Persist fingerprints for model, texture, and audio registry records and generalize move recovery. +3. Make registry and manifest publication semantic and byte-preserving. +4. Extend static-mesh and registry validation plus focused migration/read-only tests. +5. Update ADRs, asset-pipeline docs, CI clean-worktree assertion, and committed generated artifacts. + +## Verification + +- Focused editor and scene tests for mtime-only drift, same-size changes, legacy manifests, + byte-preserving equivalent refresh, imported source kinds, and read-only validation. +- Formatting, diff check, clippy, and workspace all-target tests. +- Native editor startup and asset refresh with before/after Git status and source timestamp drift. +- `cargo validate-levels --project .` and `cargo validate-samples --project .`, followed by a clean + worktree assertion. Packaged-runtime tests remain deferred by owner direction. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 0c46e15..1ea051e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -63,6 +63,11 @@ jobs: - name: Validate editor sample pack run: cargo validate-samples + - name: Assert validation leaves checkout clean + run: | + git diff --exit-code + test -z "$(git ls-files --others --exclude-standard)" + - name: Clippy launch feature matrix run: cargo clippy -p editor -p game --bins --features dev,hot-reload -- -D warnings diff --git a/README.md b/README.md index 840c5b1..e90b385 100644 --- a/README.md +++ b/README.md @@ -377,6 +377,7 @@ The `.vscode/` folder is preconfigured: - [ADR 0017: Normalized Static Mesh Assets](docs/adr/0017-normalized-static-mesh-assets.md) - [ADR 0034: Registry-driven Authoring Components](docs/adr/0034-registry-driven-authoring-components.md) - [ADR 0037: Collaborative Authored-File Safety](docs/adr/0037-collaborative-authored-file-safety.md) +- [ADR 0043: Content-Addressed Import Fingerprints](docs/adr/0043-content-addressed-import-fingerprints.md) ## Project Layout @@ -428,6 +429,9 @@ crates/ - [x] Production operator invariants across palette dispatch, assets/material drops, brush/terrain/physics modal tools, grouping/lighting, and the transform finalizer: stable terminal status, exact cancel/failure rollback, helper cleanup, grouped history, and repeated undo/redo projections ([testing contract](docs/editor/operator-regression-testing.md), [evaluation](docs/editor/evaluations/operator-invariants/), [Gitea #33](https://git.spacetrainclubhouse.com/Falling-Metal-Interactive/Blacksite/issues/33)) - [x] Verify: `cargo fmt --check` / `cargo check --workspace` / `cargo clippy --workspace` / strict foundation clippy / `cargo test -p sim` - [x] Stable asset registry with UUIDs + import settings in asset browser details +- [x] Deterministic imported-source fingerprints for models, textures, and audio; BLAKE3-backed + static/animation manifests; byte-preserving equivalent refresh; and read-only validator checkout + assertions ([ADR 0043](docs/adr/0043-content-addressed-import-fingerprints.md), [Gitea #56](https://git.spacetrainclubhouse.com/Falling-Metal-Interactive/Blacksite/issues/56)) - [x] Asset Browser expandable model subasset shelves, independent mesh/material/texture thumbnails, staged import/material details with shader-schema parameters, context actions, and trash-first file removal - [x] Audio clip catalog/import foundation for Ogg, WAV, MP3, and FLAC with dedicated filtering, file details, and stable runtime-resolvable asset references - [x] Audio source/listener authoring, non-dirty spatial audition, viewport icons/range gizmos, stable buses, PIE/runtime parity, device diagnostics, and shared release validation ([ADR 0030](docs/adr/0030-audio-authoring-and-bus-schema.md); production acceptance completed in [Gitea #47](https://git.spacetrainclubhouse.com/Falling-Metal-Interactive/Blacksite/issues/47)) diff --git a/assets/.index/registry.ron b/assets/.index/registry.ron index 7b190f8..92e25b4 100644 --- a/assets/.index/registry.ron +++ b/assets/.index/registry.ron @@ -4,6 +4,10 @@ path: "assets/audio/editor_audition_tone.ogg", label: "editor_audition_tone", kind_tag: "AudioClip", + source_fingerprint: Some(( + byte_len: 4450, + content_hash: "8ef56184ab5b5cd135e45df152f8a95737023420947f3d08c9b3e3605e55a349", + )), import_settings: ( scale: 1.0, generate_collider: true, @@ -22,6 +26,10 @@ path: "assets/audio/Puzzle Lobby.mp3", label: "Puzzle Lobby", kind_tag: "AudioClip", + source_fingerprint: Some(( + byte_len: 1680820, + content_hash: "3b5a401dbaccfdbda540c60070ce259fb506d7da548f45008eb2b3fa720b4b57", + )), import_settings: ( scale: 1.0, generate_collider: true, @@ -76,6 +84,10 @@ path: "assets/models/painted_wooden_chair_02_2k.fbx", label: "painted_wooden_chair_02_2k", kind_tag: "Model", + source_fingerprint: Some(( + byte_len: 59964, + content_hash: "b12973a62dcb44589e380ea833eade726ae98a86c81084c842ee3801866d6a46", + )), import_settings: ( scale: 1.0, generate_collider: true, @@ -94,6 +106,10 @@ path: "assets/models/RobotExpressive.glb", label: "RobotExpressive", kind_tag: "Model", + source_fingerprint: Some(( + byte_len: 463988, + content_hash: "5869cb813e6a6093eda7af2824a9c13a582750e0443e88ce7c991a1f83415521", + )), import_settings: ( scale: 1.0, generate_collider: true, @@ -112,6 +128,10 @@ path: "assets/models/robot_expressive.glb", label: "robot_expressive", kind_tag: "Model", + source_fingerprint: Some(( + byte_len: 463988, + content_hash: "5869cb813e6a6093eda7af2824a9c13a582750e0443e88ce7c991a1f83415521", + )), import_settings: ( scale: 1.0, generate_collider: true, @@ -130,6 +150,10 @@ path: "assets/textures/metal_stool_01_nor_gl_2k.jpg", label: "metal_stool_01_nor_gl_2k", kind_tag: "Texture", + source_fingerprint: Some(( + byte_len: 2779150, + content_hash: "8c0aa4cf4270c9bf7217fba40b2b48878972ee59cff851cb8dfa57a90f4f373b", + )), import_settings: ( scale: 1.0, generate_collider: true, @@ -148,6 +172,10 @@ path: "assets/textures/metal_stool_01_arm_2k.jpg", label: "metal_stool_01_arm_2k", kind_tag: "Texture", + source_fingerprint: Some(( + byte_len: 3152060, + content_hash: "31b2249ed7b50d3a021312a1e299167897513710ee8ded75c5c5e56aad592e10", + )), import_settings: ( scale: 1.0, generate_collider: true, @@ -166,6 +194,10 @@ path: "assets/textures/metal_stool_01_diff_2k.jpg", label: "metal_stool_01_diff_2k", kind_tag: "Texture", + source_fingerprint: Some(( + byte_len: 2603921, + content_hash: "e0b9dee5b09f1968e7c6ea82eb9a157d303c25d0600913b8b9c809adf50b73c8", + )), import_settings: ( scale: 1.0, generate_collider: true, diff --git a/assets/meshes/generated/113f74df-e39c-41d4-9b5b-e48efe541f7f.static_mesh.ron b/assets/meshes/generated/113f74df-e39c-41d4-9b5b-e48efe541f7f.static_mesh.ron index 303ecca..a836a4e 100644 --- a/assets/meshes/generated/113f74df-e39c-41d4-9b5b-e48efe541f7f.static_mesh.ron +++ b/assets/meshes/generated/113f74df-e39c-41d4-9b5b-e48efe541f7f.static_mesh.ron @@ -1,5 +1,5 @@ ( - schema_version: 3, + schema_version: 4, asset_id: "113f74df-e39c-41d4-9b5b-e48efe541f7f", label: "RobotExpressive", source: ( @@ -7,7 +7,7 @@ format: "glb", fingerprint: ( byte_len: 463988, - modified_unix_secs: 1783799272, + content_hash: "5869cb813e6a6093eda7af2824a9c13a582750e0443e88ce7c991a1f83415521", ), dependencies: [], ), diff --git a/assets/meshes/generated/3f63f359-45eb-4cb2-8970-71921cbd7bd0.static_mesh.ron b/assets/meshes/generated/3f63f359-45eb-4cb2-8970-71921cbd7bd0.static_mesh.ron index bbf92f0..4363ddd 100644 --- a/assets/meshes/generated/3f63f359-45eb-4cb2-8970-71921cbd7bd0.static_mesh.ron +++ b/assets/meshes/generated/3f63f359-45eb-4cb2-8970-71921cbd7bd0.static_mesh.ron @@ -1,5 +1,5 @@ ( - schema_version: 3, + schema_version: 4, asset_id: "3f63f359-45eb-4cb2-8970-71921cbd7bd0", label: "robot_expressive", source: ( @@ -7,7 +7,7 @@ format: "glb", fingerprint: ( byte_len: 463988, - modified_unix_secs: 1783750957, + content_hash: "5869cb813e6a6093eda7af2824a9c13a582750e0443e88ce7c991a1f83415521", ), dependencies: [], ), diff --git a/assets/meshes/generated/b98ef565-3500-49e7-9935-f685fa9b2594.static_mesh.ron b/assets/meshes/generated/b98ef565-3500-49e7-9935-f685fa9b2594.static_mesh.ron index 683980a..530c1ee 100644 --- a/assets/meshes/generated/b98ef565-3500-49e7-9935-f685fa9b2594.static_mesh.ron +++ b/assets/meshes/generated/b98ef565-3500-49e7-9935-f685fa9b2594.static_mesh.ron @@ -1,5 +1,5 @@ ( - schema_version: 3, + schema_version: 4, asset_id: "b98ef565-3500-49e7-9935-f685fa9b2594", label: "painted_wooden_chair_02_2k", source: ( @@ -7,7 +7,7 @@ format: "fbx", fingerprint: ( byte_len: 59964, - modified_unix_secs: 1780713434, + content_hash: "b12973a62dcb44589e380ea833eade726ae98a86c81084c842ee3801866d6a46", ), dependencies: [], ), diff --git a/crates/editor/src/assets/animation.rs b/crates/editor/src/assets/animation.rs index d1b236d..acf10a7 100644 --- a/crates/editor/src/assets/animation.rs +++ b/crates/editor/src/assets/animation.rs @@ -3,7 +3,6 @@ use std::collections::BTreeSet; use std::fs; use std::path::Path; -use std::time::UNIX_EPOCH; use serde::Deserialize; use shared::{ @@ -14,6 +13,7 @@ use shared::{ }; use crate::asset_db::AssetRecord; +use crate::assets::fingerprint::write_pretty_ron_if_changed; pub fn animation_manifest_path(asset_id: &str) -> String { format!("{ANIMATION_ARTIFACT_DIR}/{asset_id}.animation.ron") @@ -32,15 +32,9 @@ pub fn refresh_animation_artifact(record: &mut AssetRecord) -> Result Result pub fn build_animation_manifest(record: &AssetRecord) -> Result { let bytes = fs::read(&record.path) .map_err(|error| format!("could not read {}: {error}", record.path))?; - let fingerprint = source_fingerprint(&record.path, &bytes)?; + let fingerprint = source_fingerprint(&bytes); let format = source_format(&record.path)?; match format.as_str() { "gltf" | "glb" => build_gltf_manifest(record, format, fingerprint, &bytes), @@ -582,19 +576,8 @@ fn fbx_matrix_bytes(matrix: &ufbx::Matrix) -> Vec { .collect() } -fn source_fingerprint(path: &str, bytes: &[u8]) -> Result { - let metadata = fs::metadata(path).map_err(|error| format!("could not stat {path}: {error}"))?; - let modified_unix_secs = metadata - .modified() - .ok() - .and_then(|time| time.duration_since(UNIX_EPOCH).ok()) - .map(|duration| duration.as_secs()) - .unwrap_or_default(); - Ok(AnimationSourceFingerprint { - byte_len: metadata.len(), - modified_unix_secs, - content_hash: blake3::hash(bytes).to_hex().to_string(), - }) +fn source_fingerprint(bytes: &[u8]) -> AnimationSourceFingerprint { + AnimationSourceFingerprint::from_bytes(bytes) } fn source_format(path: &str) -> Result { @@ -626,6 +609,7 @@ mod tests { path: path.to_string_lossy().replace('\\', "/"), label: "Animated Hero".into(), kind_tag: "Model".into(), + source_fingerprint: None, import_settings: ImportSettings::default(), dependencies: Vec::new(), } @@ -754,6 +738,30 @@ mod tests { fs::remove_dir_all(root).unwrap(); } + #[test] + fn legacy_mtime_metadata_does_not_rewrite_equivalent_animation_manifest() { + let root = + std::env::temp_dir().join(format!("blacksite-animation-legacy-{}", Uuid::new_v4())); + let path = write_animated_gltf(&root); + let manifest = build_animation_manifest(&fixture_record(&path)).unwrap(); + let canonical = + ron::ser::to_string_pretty(&manifest, ron::ser::PrettyConfig::default()).unwrap(); + let marker = format!("byte_len: {},", manifest.source.fingerprint.byte_len); + let legacy = canonical.replacen( + &marker, + &format!("{marker}\n modified_unix_secs: 123456,"), + 1, + ); + assert_ne!(legacy, canonical); + let artifact = root.join("legacy.animation.ron"); + fs::write(&artifact, &legacy).unwrap(); + + assert!(!write_pretty_ron_if_changed(&artifact, &manifest).unwrap()); + assert_eq!(fs::read_to_string(&artifact).unwrap(), legacy); + + fs::remove_dir_all(root).unwrap(); + } + #[test] fn explicit_default_clip_is_stable_and_stale_ids_are_diagnostic() { let root = diff --git a/crates/editor/src/assets/asset_db.rs b/crates/editor/src/assets/asset_db.rs index 63d1afb..d5cea6e 100644 --- a/crates/editor/src/assets/asset_db.rs +++ b/crates/editor/src/assets/asset_db.rs @@ -2,9 +2,12 @@ use bevy::prelude::*; use serde::{Deserialize, Serialize}; +use shared::AssetSourceFingerprint; use std::collections::{HashMap, HashSet}; use uuid::Uuid; +use crate::assets::fingerprint::{fingerprint_file, write_pretty_ron_if_changed}; + /// Stable asset identity for dependency tracking and prefab references. #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, Hash)] pub struct AssetId(pub Uuid); @@ -93,6 +96,8 @@ pub struct AssetRecord { pub path: String, pub label: String, pub kind_tag: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub source_fingerprint: Option, #[serde(default)] pub import_settings: ImportSettings, #[serde(default)] @@ -122,9 +127,10 @@ fn sync_registry_from_browser( return; } - let mut existing: HashMap = registry - .records - .drain(..) + let previous_records = std::mem::take(&mut registry.records); + let mut existing: HashMap = previous_records + .iter() + .cloned() .map(|record| (record.path.clone(), record)) .collect(); let current_paths = assets @@ -146,12 +152,30 @@ fn sync_registry_from_browser( seen_paths.insert(path.clone(), ()); let kind_tag = format!("{:?}", asset.kind); + let source_fingerprint = if imported_source_kind(&kind_tag) { + match fingerprint_file(&path) { + Ok(fingerprint) => Some(fingerprint), + Err(error) => { + warn!("Imported source fingerprint failed for {path}: {error}"); + None + } + } + } else { + None + }; let mut record = if let Some(mut prior) = existing.remove(&path).or_else(|| { - take_uniquely_moved_model_record(&mut existing, ¤t_paths, &path, &kind_tag) + source_fingerprint.as_ref().and_then(|fingerprint| { + take_uniquely_moved_import_record( + &mut existing, + ¤t_paths, + &kind_tag, + fingerprint, + ) + }) }) { if prior.path != path { info!( - "Asset registry preserved model identity {} across move {} -> {}", + "Asset registry preserved imported source identity {} across move {} -> {}", prior.id.as_string(), prior.path, path @@ -159,18 +183,26 @@ fn sync_registry_from_browser( prior.path = path.clone(); } prior.label = asset.label.clone(); - prior.kind_tag = kind_tag; + prior.kind_tag = kind_tag.clone(); prior } else { AssetRecord { id: AssetId::new(), path: path.clone(), label: asset.label.clone(), - kind_tag, + kind_tag: kind_tag.clone(), + source_fingerprint: None, import_settings: ImportSettings::default(), dependencies: Vec::new(), } }; + if imported_source_kind(&kind_tag) { + if let Some(source_fingerprint) = source_fingerprint { + record.source_fingerprint = Some(source_fingerprint); + } + } else { + record.source_fingerprint = None; + } if record.kind_tag == "Model" { if let Err(error) = super::refresh_model_artifacts(&mut record) { warn!( @@ -183,7 +215,7 @@ fn sync_registry_from_browser( next_records.push(record); } - let changed = registry.records != next_records; + let changed = previous_records != next_records; registry.records = next_records; if changed { registry.index_dirty = true; @@ -198,26 +230,28 @@ fn sync_registry_from_browser( } } -fn take_uniquely_moved_model_record( +fn imported_source_kind(kind_tag: &str) -> bool { + matches!(kind_tag, "Model" | "Texture" | "AudioClip") +} + +fn take_uniquely_moved_import_record( existing: &mut HashMap, current_paths: &HashSet, - new_path: &str, kind_tag: &str, + fingerprint: &AssetSourceFingerprint, ) -> Option { - if kind_tag != "Model" { + if !imported_source_kind(kind_tag) { return None; } - let bytes = std::fs::read(new_path).ok()?; - let content_hash = blake3::hash(&bytes).to_hex().to_string(); let candidates = existing .iter() .filter_map(|(old_path, record)| { - if current_paths.contains(old_path) || record.kind_tag != "Model" { + if current_paths.contains(old_path) || record.kind_tag != kind_tag { return None; } - let manifest_path = record.import_settings.animation_manifest_path.as_deref()?; - let manifest = super::animation::load_animation_manifest(manifest_path).ok()?; - (manifest.source.fingerprint.content_hash == content_hash).then(|| old_path.clone()) + recorded_source_fingerprint(record) + .is_some_and(|prior| prior == *fingerprint) + .then(|| old_path.clone()) }) .collect::>(); if candidates.len() != 1 { @@ -226,6 +260,15 @@ fn take_uniquely_moved_model_record( existing.remove(&candidates[0]) } +fn recorded_source_fingerprint(record: &AssetRecord) -> Option { + record.source_fingerprint.clone().or_else(|| { + let manifest_path = record.import_settings.animation_manifest_path.as_deref()?; + super::animation::load_animation_manifest(manifest_path) + .ok() + .map(|manifest| manifest.source.fingerprint) + }) +} + pub fn find_asset_by_path(registry: &AssetRegistry, path: &str) -> Option { registry .records @@ -262,11 +305,16 @@ pub fn ensure_asset_record( if let Some(record) = find_asset_by_path(registry, &path) { return Ok(record); } + let kind_tag = kind_tag.into(); + let source_fingerprint = imported_source_kind(&kind_tag) + .then(|| fingerprint_file(&path).ok()) + .flatten(); let record = AssetRecord { id: AssetId::new(), path, label: label.into(), - kind_tag: kind_tag.into(), + kind_tag, + source_fingerprint, import_settings: ImportSettings::default(), dependencies: Vec::new(), }; @@ -292,6 +340,9 @@ pub fn update_import_settings( else { return false; }; + if record.import_settings == settings { + return false; + } record.import_settings = settings; registry.index_dirty = true; true @@ -302,13 +353,7 @@ pub fn registry_index_path() -> &'static str { } pub fn save_registry(registry: &AssetRegistry) -> Result<(), String> { - let path = registry_index_path(); - if let Some(parent) = std::path::Path::new(path).parent() { - std::fs::create_dir_all(parent).map_err(|e| e.to_string())?; - } - let text = ron::ser::to_string_pretty(®istry.records, ron::ser::PrettyConfig::default()) - .map_err(|e| e.to_string())?; - std::fs::write(path, text).map_err(|e| e.to_string()) + write_pretty_ron_if_changed(registry_index_path(), ®istry.records).map(|_| ()) } pub fn load_registry() -> AssetRegistry { @@ -333,6 +378,63 @@ mod tests { ANIMATION_MANIFEST_SCHEMA_VERSION, }; + #[test] + fn imported_source_policy_covers_model_texture_and_audio_only() { + for kind in ["Model", "Texture", "AudioClip"] { + assert!(imported_source_kind(kind), "missing imported kind {kind}"); + } + for kind in ["Material", "Level", "Prefab", "PostProcessEffect"] { + assert!( + !imported_source_kind(kind), + "authored kind {kind} was hashed" + ); + } + } + + #[test] + fn legacy_registry_record_loads_without_forcing_a_new_identity() { + let source = r#"( + id: ("00000000-0000-0000-0000-000000000000"), + path: "assets/textures/legacy.png", + label: "Legacy", + kind_tag: "Texture", + )"#; + let record: AssetRecord = ron::from_str(source).unwrap(); + + assert_eq!(record.id, AssetId(Uuid::nil())); + assert!(record.source_fingerprint.is_none()); + assert_eq!(record.import_settings, ImportSettings::default()); + } + + #[test] + fn content_match_preserves_every_imported_source_kind_across_move() { + let fingerprint = AssetSourceFingerprint::from_bytes(b"stable imported bytes"); + for kind in ["Model", "Texture", "AudioClip"] { + let record = AssetRecord { + id: AssetId::new(), + path: format!("assets/old/{kind}"), + label: kind.into(), + kind_tag: kind.into(), + source_fingerprint: Some(fingerprint.clone()), + import_settings: ImportSettings::default(), + dependencies: Vec::new(), + }; + let expected_id = record.id.clone(); + let mut existing = HashMap::from([(record.path.clone(), record)]); + + let moved = take_uniquely_moved_import_record( + &mut existing, + &HashSet::new(), + kind, + &fingerprint, + ) + .expect("unique content identity should preserve the record"); + + assert_eq!(moved.id, expected_id); + assert!(existing.is_empty()); + } + } + fn moved_model_fixture(bytes: &[u8]) -> (std::path::PathBuf, std::path::PathBuf, AssetRecord) { let key = Uuid::new_v4(); let source_path = std::env::temp_dir().join(format!("blacksite-moved-model-{key}.glb")); @@ -351,7 +453,6 @@ mod tests { format: "glb".into(), fingerprint: AnimationSourceFingerprint { byte_len: bytes.len() as u64, - modified_unix_secs: 0, content_hash: blake3::hash(bytes).to_hex().to_string(), }, dependencies: Vec::new(), @@ -371,6 +472,7 @@ mod tests { path: old_path, label: "Moved Model".into(), kind_tag: "Model".into(), + source_fingerprint: None, import_settings: ImportSettings { animation_manifest_path: Some(manifest_path.to_string_lossy().into_owned()), ..Default::default() @@ -385,12 +487,13 @@ mod tests { let (source_path, manifest_path, record) = moved_model_fixture(b"stable model bytes"); let expected_id = record.id.clone(); let mut existing = HashMap::from([(record.path.clone(), record)]); + let fingerprint = fingerprint_file(&source_path).unwrap(); - let moved = take_uniquely_moved_model_record( + let moved = take_uniquely_moved_import_record( &mut existing, &HashSet::new(), - &source_path.to_string_lossy(), "Model", + &fingerprint, ) .expect("unique moved model should retain its registry record"); @@ -406,13 +509,10 @@ mod tests { let old_path = record.path.clone(); let mut existing = HashMap::from([(old_path.clone(), record)]); let current_paths = HashSet::from([old_path]); + let fingerprint = fingerprint_file(&source_path).unwrap(); - let moved = take_uniquely_moved_model_record( - &mut existing, - ¤t_paths, - &source_path.to_string_lossy(), - "Model", - ); + let moved = + take_uniquely_moved_import_record(&mut existing, ¤t_paths, "Model", &fingerprint); assert!(moved.is_none()); assert_eq!(existing.len(), 1); diff --git a/crates/editor/src/assets/catalog.rs b/crates/editor/src/assets/catalog.rs index baf8803..05205a2 100644 --- a/crates/editor/src/assets/catalog.rs +++ b/crates/editor/src/assets/catalog.rs @@ -1548,6 +1548,7 @@ mod tests { path: asset.path.clone().unwrap(), label: asset.label.clone(), kind_tag: "AudioClip".into(), + source_fingerprint: None, import_settings: Default::default(), dependencies: Vec::new(), }], @@ -1630,7 +1631,6 @@ mod tests { format: "glb".into(), fingerprint: shared::AnimationSourceFingerprint { byte_len: 1, - modified_unix_secs: 0, content_hash: "fixture".into(), }, dependencies: Vec::new(), @@ -1663,6 +1663,7 @@ mod tests { path: "assets/models/robot.glb".into(), label: "Robot".into(), kind_tag: "Model".into(), + source_fingerprint: None, import_settings: crate::asset_db::ImportSettings { animation_manifest_path: Some(manifest_path.to_string_lossy().into_owned()), default_animation_clip_id: Some(clip_id.clone()), diff --git a/crates/editor/src/assets/fingerprint.rs b/crates/editor/src/assets/fingerprint.rs new file mode 100644 index 0000000..e2a0330 --- /dev/null +++ b/crates/editor/src/assets/fingerprint.rs @@ -0,0 +1,114 @@ +use std::fs; +use std::path::Path; + +use serde::de::DeserializeOwned; +use serde::Serialize; +use shared::AssetSourceFingerprint; + +pub(crate) fn fingerprint_file(path: impl AsRef) -> Result { + let path = path.as_ref(); + let bytes = fs::read(path) + .map_err(|error| format!("could not read imported source {}: {error}", path.display()))?; + Ok(AssetSourceFingerprint::from_bytes(&bytes)) +} + +/// Writes canonical pretty RON only when the parsed document changes semantically. +/// +/// Equivalent existing bytes, including custom formatting and final-newline policy, stay intact. +pub(crate) fn write_pretty_ron_if_changed( + path: impl AsRef, + value: &T, +) -> Result +where + T: DeserializeOwned + PartialEq + Serialize, +{ + let path = path.as_ref(); + if fs::read_to_string(path) + .ok() + .and_then(|text| ron::from_str::(&text).ok()) + .is_some_and(|existing| existing == *value) + { + return Ok(false); + } + + let text = ron::ser::to_string_pretty(value, ron::ser::PrettyConfig::default()) + .map_err(|error| format!("could not serialize RON: {error}"))?; + if fs::read(path).ok().as_deref() == Some(text.as_bytes()) { + return Ok(false); + } + if let Some(parent) = path.parent() { + fs::create_dir_all(parent) + .map_err(|error| format!("could not create {}: {error}", parent.display()))?; + } + fs::write(path, text) + .map_err(|error| format!("could not write {}: {error}", path.display()))?; + Ok(true) +} + +#[cfg(test)] +mod tests { + use super::*; + use serde::Deserialize; + use std::fs::{File, FileTimes}; + use std::time::{Duration, SystemTime}; + use uuid::Uuid; + + #[derive(Debug, Deserialize, PartialEq, Serialize)] + struct Fixture { + count: u32, + label: String, + } + + fn fixture_path(name: &str) -> std::path::PathBuf { + std::env::temp_dir().join(format!( + "blacksite-fingerprint-{name}-{}.ron", + Uuid::new_v4() + )) + } + + #[test] + fn metadata_only_drift_does_not_change_content_identity() { + let path = fixture_path("mtime"); + fs::write(&path, b"stable source bytes").unwrap(); + let before = fingerprint_file(&path).unwrap(); + File::options() + .write(true) + .open(&path) + .unwrap() + .set_times( + FileTimes::new().set_modified(SystemTime::now() + Duration::from_secs(86_400)), + ) + .unwrap(); + + assert_eq!(fingerprint_file(&path).unwrap(), before); + fs::remove_file(path).unwrap(); + } + + #[test] + fn same_size_byte_change_updates_content_identity() { + let path = fixture_path("same-size"); + fs::write(&path, b"source-a").unwrap(); + let before = fingerprint_file(&path).unwrap(); + fs::write(&path, b"source-b").unwrap(); + let after = fingerprint_file(&path).unwrap(); + + assert_eq!(before.byte_len, after.byte_len); + assert_ne!(before.content_hash, after.content_hash); + fs::remove_file(path).unwrap(); + } + + #[test] + fn equivalent_ron_preserves_exact_existing_bytes() { + let path = fixture_path("semantic"); + let existing = b"( label: \"stable\", count: 7, )\n\n"; + fs::write(&path, existing).unwrap(); + let value = Fixture { + count: 7, + label: "stable".into(), + }; + + assert!(!write_pretty_ron_if_changed(&path, &value).unwrap()); + assert_eq!(fs::read(&path).unwrap(), existing); + fs::remove_file(path).unwrap(); + } +} diff --git a/crates/editor/src/assets/mod.rs b/crates/editor/src/assets/mod.rs index 57da715..48ca69d 100644 --- a/crates/editor/src/assets/mod.rs +++ b/crates/editor/src/assets/mod.rs @@ -3,6 +3,7 @@ pub mod animation; pub mod asset_db; mod catalog; +mod fingerprint; mod import; pub mod materials; pub mod operators; diff --git a/crates/editor/src/assets/operators.rs b/crates/editor/src/assets/operators.rs index acce012..1f4129d 100644 --- a/crates/editor/src/assets/operators.rs +++ b/crates/editor/src/assets/operators.rs @@ -228,7 +228,6 @@ mod tests { format: "glb".into(), fingerprint: AnimationSourceFingerprint { byte_len: 1, - modified_unix_secs: 0, content_hash: "fixture".into(), }, dependencies: Vec::new(), @@ -265,6 +264,7 @@ mod tests { path: model_path.clone(), label: "Operator Fixture".into(), kind_tag: "Model".into(), + source_fingerprint: None, import_settings: ImportSettings { animation_manifest_path: Some(path.to_string_lossy().into_owned()), ..Default::default() @@ -352,6 +352,7 @@ mod tests { path: path.clone(), label: "Impact".into(), kind_tag: "AudioClip".into(), + source_fingerprint: None, import_settings: Default::default(), dependencies: Vec::new(), }], @@ -502,6 +503,7 @@ mod tests { path: material_path.clone(), label: "Concrete".into(), kind_tag: "Material".into(), + source_fingerprint: None, import_settings: Default::default(), dependencies: Vec::new(), }], diff --git a/crates/editor/src/assets/static_mesh.rs b/crates/editor/src/assets/static_mesh.rs index 61aed72..c64a6b7 100644 --- a/crates/editor/src/assets/static_mesh.rs +++ b/crates/editor/src/assets/static_mesh.rs @@ -2,7 +2,6 @@ use std::fs; use std::path::Path; -use std::time::UNIX_EPOCH; use bevy::gltf::GltfAssetLabel; use bevy::prelude::*; @@ -15,11 +14,13 @@ use crate::asset_db::{ AssetRecord, ImportSettings, MaterialImportPolicy, ModelHierarchyMode, ModelPlacementMode, }; use shared::{ - ComponentInstanceId, EditorAssetRef, MaterialRef, RendererMaterialSet, RendererMaterialSlot, - StaticMeshRenderer, StaticMeshRendererEntry, + AssetSourceFingerprint, ComponentInstanceId, EditorAssetRef, MaterialRef, RendererMaterialSet, + RendererMaterialSlot, StaticMeshRenderer, StaticMeshRendererEntry, }; -pub const STATIC_MESH_MANIFEST_SCHEMA: u32 = 3; +use crate::assets::fingerprint::{fingerprint_file, write_pretty_ron_if_changed}; + +pub const STATIC_MESH_MANIFEST_SCHEMA: u32 = 4; pub const STATIC_MESH_ARTIFACT_DIR: &str = "assets/meshes/generated"; #[derive(Debug, Clone, Serialize, Deserialize, PartialEq)] @@ -42,11 +43,7 @@ pub struct StaticMeshSource { pub dependencies: Vec, } -#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)] -pub struct StaticMeshSourceFingerprint { - pub byte_len: u64, - pub modified_unix_secs: u64, -} +pub type StaticMeshSourceFingerprint = AssetSourceFingerprint; #[derive(Debug, Clone, Serialize, Deserialize, PartialEq)] pub struct StaticMeshImportSnapshot { @@ -134,16 +131,9 @@ pub fn refresh_static_mesh_artifact( manifest.source.dependencies.dedup(); record.dependencies = manifest.source.dependencies.clone(); - let text = ron::ser::to_string_pretty(&manifest, ron::ser::PrettyConfig::default()) - .map_err(|err| format!("could not serialize static mesh manifest: {err}"))?; - if let Some(parent) = Path::new(&path).parent() { - fs::create_dir_all(parent) - .map_err(|err| format!("could not create {}: {err}", parent.display()))?; - } - let should_write = fs::read_to_string(&path).ok().as_deref() != Some(text.as_str()); - if should_write { - fs::write(&path, text) - .map_err(|err| format!("could not write static mesh manifest {path}: {err}"))?; + if write_pretty_ron_if_changed(&path, &manifest) + .map_err(|error| format!("could not publish static mesh manifest {path}: {error}"))? + { info!( "Static mesh manifest refreshed: source={} artifact={} parts={}", record.path, @@ -287,7 +277,7 @@ fn part_effective_material_id(part: &StaticMeshPart) -> Option { fn build_static_mesh_manifest(record: &AssetRecord) -> Result { let format = source_format(&record.path)?; - let fingerprint = source_fingerprint(&record.path)?; + let fingerprint = fingerprint_file(&record.path)?; let import = StaticMeshImportSnapshot { scale: record.import_settings.scale, generate_collider: record.import_settings.generate_collider, @@ -631,20 +621,6 @@ fn source_format(path: &str) -> Result { .ok_or_else(|| format!("asset path `{path}` has no extension")) } -fn source_fingerprint(path: &str) -> Result { - let metadata = fs::metadata(path).map_err(|err| format!("could not stat {path}: {err}"))?; - let modified_unix_secs = metadata - .modified() - .ok() - .and_then(|time| time.duration_since(UNIX_EPOCH).ok()) - .map(|duration| duration.as_secs()) - .unwrap_or_default(); - Ok(StaticMeshSourceFingerprint { - byte_len: metadata.len(), - modified_unix_secs, - }) -} - fn resolve_dependency(source_path: &str, uri: &str) -> String { if uri.starts_with("data:") || uri.contains("://") { return uri.to_string(); @@ -672,7 +648,7 @@ mod tests { format: "glb".into(), fingerprint: StaticMeshSourceFingerprint { byte_len: 42, - modified_unix_secs: 10, + content_hash: "a".repeat(64), }, dependencies: Vec::new(), }, @@ -741,6 +717,55 @@ mod tests { ); } + #[test] + fn schema_v3_manifest_without_hash_migrates_to_content_fingerprint() { + let root = std::env::temp_dir().join(format!( + "blacksite-static-mesh-legacy-{}", + uuid::Uuid::new_v4() + )); + fs::create_dir_all(&root).unwrap(); + let path = root.join("legacy.static_mesh.ron"); + let expected = test_manifest(); + let canonical = + ron::ser::to_string_pretty(&expected, ron::ser::PrettyConfig::default()).unwrap(); + let legacy = canonical + .replacen("schema_version: 4", "schema_version: 3", 1) + .lines() + .filter(|line| !line.contains("content_hash:")) + .collect::>() + .join("\n"); + fs::write(&path, legacy).unwrap(); + + assert!(write_pretty_ron_if_changed(&path, &expected).unwrap()); + assert_eq!( + load_static_mesh_manifest(&path.to_string_lossy()).unwrap(), + expected + ); + + fs::remove_dir_all(root).unwrap(); + } + + #[test] + fn equivalent_static_manifest_preserves_existing_formatting_and_newline() { + let root = std::env::temp_dir().join(format!( + "blacksite-static-mesh-semantic-{}", + uuid::Uuid::new_v4() + )); + fs::create_dir_all(&root).unwrap(); + let path = root.join("stable.static_mesh.ron"); + let manifest = test_manifest(); + let exact = format!( + "{}\n\n", + ron::ser::to_string_pretty(&manifest, ron::ser::PrettyConfig::default()).unwrap() + ); + fs::write(&path, &exact).unwrap(); + + assert!(!write_pretty_ron_if_changed(&path, &manifest).unwrap()); + assert_eq!(fs::read_to_string(&path).unwrap(), exact); + + fs::remove_dir_all(root).unwrap(); + } + #[test] fn renderer_from_manifest_can_ignore_source_materials() { let manifest = test_manifest(); @@ -785,6 +810,7 @@ mod tests { path, label: "Robot Expressive".into(), kind_tag: "Model".into(), + source_fingerprint: None, import_settings: ImportSettings::default(), dependencies: Vec::new(), }; diff --git a/crates/editor/src/ui/animation_inspector.rs b/crates/editor/src/ui/animation_inspector.rs index 46db655..4c19765 100644 --- a/crates/editor/src/ui/animation_inspector.rs +++ b/crates/editor/src/ui/animation_inspector.rs @@ -884,7 +884,6 @@ mod tests { format: "glb".into(), fingerprint: AnimationSourceFingerprint { byte_len: 42, - modified_unix_secs: 1, content_hash: "hash".into(), }, dependencies: Vec::new(), diff --git a/crates/game/src/animation.rs b/crates/game/src/animation.rs index 5ae3d2c..200cac7 100644 --- a/crates/game/src/animation.rs +++ b/crates/game/src/animation.rs @@ -1267,7 +1267,6 @@ mod tests { format: "glb".into(), fingerprint: AnimationSourceFingerprint { byte_len: 1, - modified_unix_secs: 0, content_hash: "fixture".into(), }, dependencies: Vec::new(), diff --git a/crates/scene/src/project_validation.rs b/crates/scene/src/project_validation.rs index cd1c86f..0d05e34 100644 --- a/crates/scene/src/project_validation.rs +++ b/crates/scene/src/project_validation.rs @@ -9,12 +9,13 @@ use shared::{ animation_clip_source_index, animation_clip_sub_asset_id, animation_skeleton_source_index, animation_skeleton_sub_asset_id, brush_math::{validate_brush, BrushDiagnosticSeverity}, - AnimationControllerDesc, AnimationDiagnosticSeverity, AnimationManifest, AudioListenerDesc, - AudioSourceDesc, BrushDesc, ColliderDesc, ColliderShapeDesc, EditorAssetRef, MaterialAsset, - MaterialDesc, MaterialInstanceAsset, MaterialOverride, ModelRef, PostProcessEffectAsset, - PostProcessVolumeDesc, PrefabInstance, PrefabRef, RendererMaterialSet, ShaderSchemaAsset, - SkinnedMeshRenderer, StaticMeshRenderer, TerrainDesc, ANIMATION_MANIFEST_SCHEMA_VERSION, - AUDIO_CLIP_SUB_ASSET_ID, COMPONENT_ANIMATION_CONTROLLER_DESC, COMPONENT_SKINNED_MESH_RENDERER, + AnimationControllerDesc, AnimationDiagnosticSeverity, AnimationManifest, + AssetSourceFingerprint, AudioListenerDesc, AudioSourceDesc, BrushDesc, ColliderDesc, + ColliderShapeDesc, EditorAssetRef, MaterialAsset, MaterialDesc, MaterialInstanceAsset, + MaterialOverride, ModelRef, PostProcessEffectAsset, PostProcessVolumeDesc, PrefabInstance, + PrefabRef, RendererMaterialSet, ShaderSchemaAsset, SkinnedMeshRenderer, StaticMeshRenderer, + TerrainDesc, ANIMATION_MANIFEST_SCHEMA_VERSION, AUDIO_CLIP_SUB_ASSET_ID, + COMPONENT_ANIMATION_CONTROLLER_DESC, COMPONENT_SKINNED_MESH_RENDERER, NAVIGATION_GENERATED_ARTIFACT_DIRECTORY, }; @@ -322,6 +323,8 @@ struct RegistryAssetRecord { path: String, kind_tag: String, #[serde(default)] + source_fingerprint: Option, + #[serde(default)] import_settings: RegistryImportSettings, #[serde(default)] dependencies: Vec, @@ -482,10 +485,96 @@ fn validate_asset_registry( repair: "Convert the clip to Ogg Vorbis, WAV, MP3, or FLAC and reimport it.".into(), }); } + if matches!(record.kind_tag.as_str(), "Model" | "Texture" | "AudioClip") { + validate_content_fingerprint( + project_root, + report, + "assets/.index/registry.ron", + &record.path, + record.source_fingerprint.as_ref(), + "registry.source_fingerprint_missing", + "registry.source_fingerprint_stale", + "registered imported source", + ); + } } animation_catalog } +#[allow(clippy::too_many_arguments)] +fn validate_content_fingerprint( + project_root: &Path, + report: &mut ProjectValidationReport, + owner_path: &str, + source_path: &str, + fingerprint: Option<&AssetSourceFingerprint>, + missing_code: &str, + stale_code: &str, + source_label: &str, +) { + let Some(fingerprint) = fingerprint.filter(|fingerprint| fingerprint.is_complete()) else { + report.findings.push(ProjectValidationFinding { + severity: ValidationSeverity::Error, + code: missing_code.into(), + source_path: owner_path.into(), + owner_actor_id: None, + reference: Some(source_path.into()), + message: format!("{source_label} has no complete BLAKE3 content fingerprint"), + repair: "Refresh the asset registry or reimport the source with this editor version." + .into(), + }); + return; + }; + let Some(path) = resolve_reference(project_root, source_path) else { + return; + }; + let Ok(metadata) = std::fs::metadata(&path) else { + return; + }; + if metadata.len() != fingerprint.byte_len { + report.findings.push(ProjectValidationFinding { + severity: ValidationSeverity::Error, + code: stale_code.into(), + source_path: owner_path.into(), + owner_actor_id: None, + reference: Some(source_path.into()), + message: format!( + "{source_label} size changed from {} to {} bytes", + fingerprint.byte_len, + metadata.len() + ), + repair: "Refresh or reimport the changed source before validation or packaging.".into(), + }); + return; + } + match blake3_file_hash(&path) { + Ok(content_hash) if content_hash != fingerprint.content_hash => { + report.findings.push(ProjectValidationFinding { + severity: ValidationSeverity::Error, + code: stale_code.into(), + source_path: owner_path.into(), + owner_actor_id: None, + reference: Some(source_path.into()), + message: format!( + "{source_label} content changed even though its byte length is unchanged" + ), + repair: "Refresh or reimport the changed source before validation or packaging." + .into(), + }); + } + Ok(_) => {} + Err(error) => report.findings.push(ProjectValidationFinding { + severity: ValidationSeverity::Error, + code: stale_code.into(), + source_path: owner_path.into(), + owner_actor_id: None, + reference: Some(source_path.into()), + message: format!("could not hash {source_label}: {error}"), + repair: "Restore readable source content and rerun validation.".into(), + }), + } +} + fn reference_is_runtime_package_asset(reference: &str) -> bool { let relative = Path::new(reference.split('#').next().unwrap_or(reference)); if relative.is_absolute() @@ -709,10 +798,12 @@ struct StaticMeshManifestView { struct StaticMeshSourceView { path: String, #[serde(default)] + fingerprint: AssetSourceFingerprint, + #[serde(default)] dependencies: Vec, } -const STATIC_MESH_MANIFEST_SCHEMA_VERSION: u32 = 3; +const STATIC_MESH_MANIFEST_SCHEMA_VERSION: u32 = 4; fn validate_static_mesh_manifests(project_root: &Path, report: &mut ProjectValidationReport) { for path in ron_files( @@ -754,6 +845,16 @@ fn validate_static_mesh_manifests(project_root: &Path, report: &mut ProjectValid "model", &manifest.source.path, ); + validate_content_fingerprint( + project_root, + report, + &source, + &manifest.source.path, + Some(&manifest.source.fingerprint), + "static_mesh.manifest_fingerprint_missing", + "static_mesh.manifest_stale", + "static mesh source", + ); for dependency in manifest.source.dependencies { add_reference( project_root, @@ -975,12 +1076,7 @@ fn validate_animation_manifest_contract( } let fingerprint = &manifest.source.fingerprint; - let hash_is_present = fingerprint.content_hash.len() == 64 - && fingerprint - .content_hash - .bytes() - .all(|byte| byte.is_ascii_hexdigit()); - if fingerprint.byte_len == 0 || fingerprint.modified_unix_secs == 0 || !hash_is_present { + if fingerprint.byte_len == 0 || !fingerprint.is_complete() { report.findings.push(ProjectValidationFinding { severity: ValidationSeverity::Error, code: "animation.manifest_fingerprint_missing".into(), @@ -3071,7 +3167,7 @@ mod tests { } #[test] - fn static_mesh_manifest_schema_v3_is_current() { + fn static_mesh_manifest_schema_v4_is_current() { let root = fixture_root(); let source_path = "assets/models/static.glb"; std::fs::create_dir_all(root.join("assets/models")).unwrap(); @@ -3081,7 +3177,8 @@ mod tests { std::fs::write( artifact_dir.join("static-id.static_mesh.ron"), format!( - "(schema_version:{STATIC_MESH_MANIFEST_SCHEMA_VERSION},asset_id:\"static-id\",source:(path:\"{source_path}\",dependencies:[]),warnings:[])" + "(schema_version:{STATIC_MESH_MANIFEST_SCHEMA_VERSION},asset_id:\"static-id\",source:(path:\"{source_path}\",fingerprint:(byte_len:5,content_hash:\"{}\"),dependencies:[]),warnings:[])", + blake3::hash(b"model").to_hex() ), ) .unwrap(); @@ -3095,6 +3192,70 @@ mod tests { std::fs::remove_dir_all(root).unwrap(); } + #[test] + fn same_size_static_mesh_source_change_invalidates_content_hash() { + let root = fixture_root(); + let source_path = "assets/models/static.glb"; + std::fs::create_dir_all(root.join("assets/models")).unwrap(); + std::fs::write(root.join(source_path), b"model").unwrap(); + let fingerprint = AssetSourceFingerprint::from_bytes(b"model"); + let artifact = root.join("assets/meshes/generated/static-id.static_mesh.ron"); + std::fs::create_dir_all(artifact.parent().unwrap()).unwrap(); + std::fs::write( + &artifact, + format!( + "(schema_version:{STATIC_MESH_MANIFEST_SCHEMA_VERSION},asset_id:\"static-id\",source:(path:\"{source_path}\",fingerprint:(byte_len:{},content_hash:\"{}\"),dependencies:[]),warnings:[])", + fingerprint.byte_len, fingerprint.content_hash + ), + ) + .unwrap(); + std::fs::write(root.join(source_path), b"modem").unwrap(); + + let report = validate_project(&root); + + let finding = report + .findings + .iter() + .find(|finding| finding.code == "static_mesh.manifest_stale") + .expect("same-size model edit must stale the static mesh manifest"); + assert!(finding.message.contains("byte length is unchanged")); + assert!(finding.repair.contains("Refresh") || finding.repair.contains("reimport")); + std::fs::remove_dir_all(root).unwrap(); + } + + #[test] + fn registry_detects_same_size_changes_for_all_imported_source_kinds() { + for (kind, relative) in [ + ("Model", "assets/models/source.glb"), + ("Texture", "assets/textures/source.png"), + ("AudioClip", "assets/audio/source.ogg"), + ] { + let root = fixture_root(); + let path = root.join(relative); + std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + std::fs::write(&path, b"source-a").unwrap(); + let fingerprint = AssetSourceFingerprint::from_bytes(b"source-a"); + std::fs::write( + root.join("assets/.index/registry.ron"), + format!( + "[(id:(\"00000000-0000-0000-0000-000000000001\"),path:\"{relative}\",kind_tag:\"{kind}\",source_fingerprint:Some((byte_len:{},content_hash:\"{}\")),dependencies:[])]", + fingerprint.byte_len, fingerprint.content_hash + ), + ) + .unwrap(); + std::fs::write(&path, b"source-b").unwrap(); + + let report = validate_project(&root); + + assert!(report.findings.iter().any(|finding| { + finding.code == "registry.source_fingerprint_stale" + && finding.reference.as_deref() == Some(relative) + && finding.message.contains("byte length is unchanged") + })); + std::fs::remove_dir_all(root).unwrap(); + } + } + fn write_manifest(root: &Path, version: u32) { std::fs::write( root.join("assets/project.ron"), @@ -3128,7 +3289,6 @@ mod tests { format: source_format.into(), fingerprint: AnimationSourceFingerprint { byte_len: 5, - modified_unix_secs: 1, content_hash: blake3::hash(b"model").to_hex().to_string(), }, dependencies: Vec::new(), @@ -3188,10 +3348,12 @@ mod tests { let manifest = animation_manifest_path .map(|path| format!("Some({path:?})")) .unwrap_or_else(|| "None".into()); + let fingerprint = AssetSourceFingerprint::from_bytes(b"model"); std::fs::write( root.join("assets/.index/registry.ron"), format!( - "[(id:(\"{ANIMATION_ASSET_ID}\"),path:\"{source_path}\",kind_tag:\"Model\",import_settings:(static_mesh_manifest_path:None,animation_manifest_path:{manifest}),dependencies:[])]" + "[(id:(\"{ANIMATION_ASSET_ID}\"),path:\"{source_path}\",kind_tag:\"Model\",source_fingerprint:Some((byte_len:{},content_hash:\"{}\")),import_settings:(static_mesh_manifest_path:None,animation_manifest_path:{manifest}),dependencies:[])]", + fingerprint.byte_len, fingerprint.content_hash ), ) .unwrap(); @@ -3205,10 +3367,12 @@ mod tests { let source = root.join(source_path); std::fs::create_dir_all(source.parent().unwrap()).unwrap(); std::fs::write(source, b"model").unwrap(); + let fingerprint = AssetSourceFingerprint::from_bytes(b"model"); std::fs::write( root.join("assets/.index/registry.ron"), format!( - "[(id:(\"{ANIMATION_ASSET_ID}\"),path:\"{source_path}\",kind_tag:\"Model\",import_settings:(static_mesh_manifest_path:None,animation_manifest_path:Some(\"{ANIMATION_MANIFEST_PATH}\"),default_animation_clip_id:Some(\"{default_clip_id}\")),dependencies:[])]" + "[(id:(\"{ANIMATION_ASSET_ID}\"),path:\"{source_path}\",kind_tag:\"Model\",source_fingerprint:Some((byte_len:{},content_hash:\"{}\")),import_settings:(static_mesh_manifest_path:None,animation_manifest_path:Some(\"{ANIMATION_MANIFEST_PATH}\"),default_animation_clip_id:Some(\"{default_clip_id}\")),dependencies:[])]", + fingerprint.byte_len, fingerprint.content_hash ), ) .unwrap(); @@ -3632,6 +3796,49 @@ mod tests { std::fs::remove_dir_all(root).unwrap(); } + #[test] + fn project_validation_preserves_registry_and_generated_artifact_bytes() { + let root = fixture_root(); + let source_path = "assets/models/hero.glb"; + let manifest = animation_fixture_manifest(source_path, "glb", "rig-signature", true); + write_animation_source_and_registry(&root, source_path, Some(ANIMATION_MANIFEST_PATH)); + write_animation_manifest(&root, &manifest); + write_animation_scene(&root, &animation_fixture_controller(&manifest)); + + let static_path = root.join("assets/meshes/generated/model-id.static_mesh.ron"); + std::fs::create_dir_all(static_path.parent().unwrap()).unwrap(); + std::fs::write( + &static_path, + format!( + "( schema_version: {STATIC_MESH_MANIFEST_SCHEMA_VERSION}, asset_id: \"model-id\", source: ( path: \"{source_path}\", fingerprint: ( byte_len: {}, content_hash: \"{}\", ), dependencies: [], ), warnings: [], )\n\n", + manifest.source.fingerprint.byte_len, + manifest.source.fingerprint.content_hash + ), + ) + .unwrap(); + let paths = [ + root.join("assets/.index/registry.ron"), + root.join(ANIMATION_MANIFEST_PATH), + static_path, + ]; + let before = paths + .iter() + .map(std::fs::read) + .collect::, _>>() + .unwrap(); + + let _ = validate_project(&root); + let _ = validate_project(&root); + + let after = paths + .iter() + .map(std::fs::read) + .collect::, _>>() + .unwrap(); + assert_eq!(after, before); + std::fs::remove_dir_all(root).unwrap(); + } + #[test] fn missing_animation_manifest_is_reported_globally_and_on_the_actor_property() { let root = fixture_root(); diff --git a/crates/shared/src/animation.rs b/crates/shared/src/animation.rs index 2e8b5d5..17036e4 100644 --- a/crates/shared/src/animation.rs +++ b/crates/shared/src/animation.rs @@ -266,13 +266,8 @@ pub struct AnimationManifestSource { pub dependencies: Vec, } -#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] -pub struct AnimationSourceFingerprint { - pub byte_len: u64, - pub modified_unix_secs: u64, - /// BLAKE3 hash of the source file bytes. - pub content_hash: String, -} +/// Backward-compatible name for the shared imported-source fingerprint contract. +pub type AnimationSourceFingerprint = crate::AssetSourceFingerprint; #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] pub struct AnimationSkeletonRecord { diff --git a/crates/shared/src/asset_fingerprint.rs b/crates/shared/src/asset_fingerprint.rs new file mode 100644 index 0000000..59b5635 --- /dev/null +++ b/crates/shared/src/asset_fingerprint.rs @@ -0,0 +1,50 @@ +use serde::{Deserialize, Serialize}; + +/// Stable identity for the exact bytes of an imported project source. +/// +/// Filesystem timestamps are deliberately excluded: Git and LFS do not preserve checkout mtimes, +/// so they cannot participate in a portable authored-data contract. +#[derive(Debug, Clone, Default, PartialEq, Eq, Hash, Serialize, Deserialize)] +pub struct AssetSourceFingerprint { + pub byte_len: u64, + #[serde(default)] + pub content_hash: String, +} + +impl AssetSourceFingerprint { + pub fn from_bytes(bytes: &[u8]) -> Self { + Self { + byte_len: bytes.len() as u64, + content_hash: blake3::hash(bytes).to_hex().to_string(), + } + } + + pub fn is_complete(&self) -> bool { + self.content_hash.len() == 64 + && self + .content_hash + .bytes() + .all(|byte| byte.is_ascii_digit() || (b'a'..=b'f').contains(&byte)) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn exact_bytes_define_source_identity() { + let first = AssetSourceFingerprint::from_bytes(b"asset-a"); + let same = AssetSourceFingerprint::from_bytes(b"asset-a"); + let same_size_change = AssetSourceFingerprint::from_bytes(b"asset-b"); + + assert_eq!(first, same); + assert_ne!(first, same_size_change); + assert_eq!(first.byte_len, same_size_change.byte_len); + assert!(first.is_complete()); + + let mut noncanonical = first; + noncanonical.content_hash.make_ascii_uppercase(); + assert!(!noncanonical.is_complete()); + } +} diff --git a/crates/shared/src/lib.rs b/crates/shared/src/lib.rs index c86f81c..360d6fe 100644 --- a/crates/shared/src/lib.rs +++ b/crates/shared/src/lib.rs @@ -6,6 +6,7 @@ mod actor; mod animation; +mod asset_fingerprint; pub mod brush_math; mod components; mod hydration; @@ -18,6 +19,7 @@ mod rendering_profile_asset; pub use actor::{infer_actor_kind, validate_actor, ActorValidationError}; pub use animation::*; +pub use asset_fingerprint::AssetSourceFingerprint; pub use components::*; pub use hydration::{ cascade_config_from_rendering, flush_level_object_hydration, diff --git a/docs/README.md b/docs/README.md index a6c902e..103d135 100644 --- a/docs/README.md +++ b/docs/README.md @@ -57,6 +57,7 @@ Immutable-style log of significant decisions. Add a new numbered ADR when changi | [0040](adr/0040-terrain-material-layer-weights.md) | Four-channel terrain material layers, compact normalized weights, and raster transport | | [0041](adr/0041-transactional-editor-physics-placement.md) | Paused editor physics ownership and transactional gravity placement | | [0042](adr/0042-guarded-editor-shutdown-and-document-savepoints.md) | Guarded native editor exit and canonical per-document clean checkpoints | +| [0043](adr/0043-content-addressed-import-fingerprints.md) | Content-addressed imported-source identity and byte-preserving artifact publication | ## Editor framework diff --git a/docs/adr/0017-normalized-static-mesh-assets.md b/docs/adr/0017-normalized-static-mesh-assets.md index 8555ae5..2b0325c 100644 --- a/docs/adr/0017-normalized-static-mesh-assets.md +++ b/docs/adr/0017-normalized-static-mesh-assets.md @@ -13,7 +13,12 @@ retaining explicit, separate paths for skinned geometry and generic full-scene d ## Decision -Model imports now generate deterministic project artifacts under `assets/meshes/generated/*.static_mesh.ron`. The artifact records stable part IDs, source path, format, source fingerprint, import settings, mesh/material subasset labels, source node/material metadata, dependency paths, and warnings for metadata that is not played by the static mesh path. +Model imports now generate deterministic project artifacts under +`assets/meshes/generated/*.static_mesh.ron`. The artifact records stable part IDs, source path, +format, the content-addressed source fingerprint owned by +[ADR 0043](0043-content-addressed-import-fingerprints.md), import settings, mesh/material subasset +labels, source node/material metadata, dependency paths, and warnings for metadata that is not +played by the static mesh path. The saved scene authoring component is `StaticMeshRenderer`. Its slots reference imported mesh/material assets by `EditorAssetRef` (`asset_id` + `sub_asset_id`) rather than source file paths. Hydration resolves those refs through the generated artifact, spawns runtime child `Mesh3d` parts, and strips/regenerates them like other hydrated ECS. Built-in primitives remain `Primitive`, and explicit full-scene placement remains `ImportedModel + ModelRef`. @@ -35,4 +40,6 @@ Hierarchy mode is also stored per asset. `SingleActor` places all mesh parts int - Static mesh actors can expose renderer slots, source labels, shadows, collider policy, and material overrides in the inspector. - Skinned parts are marked in artifact schema v2 and excluded from static renderer construction and hydration. Animation playback uses the dedicated contract in ADRs 0031 and 0033. +- Static-mesh schema v4 uses byte length plus BLAKE3 rather than checkout mtime, so a source touch is + a no-op while a same-size byte edit is detected. - Generated artifacts are project files but are hidden from the Asset Browser catalog to avoid treating them as levels/prefabs. diff --git a/docs/adr/0028-authoritative-project-content-validation.md b/docs/adr/0028-authoritative-project-content-validation.md index f554b11..9bdb78d 100644 --- a/docs/adr/0028-authoritative-project-content-validation.md +++ b/docs/adr/0028-authoritative-project-content-validation.md @@ -42,6 +42,12 @@ startup entrypoint. Unknown newer scene schemas are rejected rather than down-st requirements are explicit findings; for example, Solari projects record the required forward-path fallback QA when ray tracing is unavailable. +Imported model, texture, and audio registry records, animation manifests, and static-mesh manifests +share the content-addressed fingerprint contract in +[ADR 0043](0043-content-addressed-import-fingerprints.md). Validation hashes source bytes and catches +same-size changes without consulting mtimes. The validation entry point is read-only; CI asserts that +running both level and sample validation leaves the checkout unchanged. + Terrain descriptors participate in the same actor-owned dependency pass. Their authored dimensions, height and weight grids, chunk settings, base material, and material layers are validated before hydration so a sample or shipping scene cannot hide an invalid or unresolved terrain reference. @@ -58,3 +64,5 @@ hydration so a sample or shipping scene cannot hide an invalid or unresolved ter editor UI and headless tooling consume the same scene ordering and validation result. - CI must hydrate Git LFS before validation because a pointer file is intentionally treated as missing runtime content. +- Validators report stale or incomplete import fingerprints with repair guidance but never rewrite + the registry or generated artifacts. diff --git a/docs/adr/0043-content-addressed-import-fingerprints.md b/docs/adr/0043-content-addressed-import-fingerprints.md new file mode 100644 index 0000000..fc2b506 --- /dev/null +++ b/docs/adr/0043-content-addressed-import-fingerprints.md @@ -0,0 +1,55 @@ +# ADR 0043: Content-Addressed Import Fingerprints + +## Status + +Accepted + +## Context + +Generated model artifacts and the project asset registry must be stable across contributors, CI, +Git worktrees, and LFS checkouts. Git does not preserve source-file modification times. Persisting a +checkout mtime as source identity therefore caused the editor to rewrite otherwise equivalent +animation and static-mesh manifests during startup. Static-mesh manifests also used only byte length +and mtime, so a same-size source edit could evade the persisted identity contract. + +Formatting-only publication was a second source of noise. Re-serializing an equivalent RON +document could normalize its final newline or layout even when import semantics had not changed. +That made clean-checkout release evidence unreliable and obscured intentional asset edits. + +## Decision + +`shared::AssetSourceFingerprint` is the persisted imported-source identity. It contains the exact +source byte length and lowercase BLAKE3 digest. Filesystem timestamps are not serialized or compared +as semantic identity; an importer may use them only as an in-memory scan hint before confirming +content. + +The asset registry stores this fingerprint for every imported model, texture, and audio source. +Content identity also drives unique move reconciliation, so metadata-only drift cannot replace a +stable registry UUID. Ambiguous copies still receive a new identity. + +Animation and static-mesh manifests use the same fingerprint. Animation schema v3 already contained +a content hash, so its schema remains v3 and legacy `modified_unix_secs` fields are accepted as +ignored unknown metadata. Static-mesh schema v4 replaces its length-plus-mtime record with length +plus BLAKE3. Existing schema-v3 static artifacts are regenerated once; unrelated assets are not +reimported. + +Registry and generated-manifest writers parse the existing RON and compare the semantic value before +publication. Equivalent documents preserve their exact existing bytes, including layout and final +newlines. Changed documents publish canonical pretty RON. + +Authoritative project validation remains read-only. It validates registry, animation, and +static-mesh fingerprints by length and digest, including same-size source edits. CI runs both +headless validators and then asserts that tracked and untracked checkout state is unchanged. + +## Consequences + +- A source touch or fresh checkout mtime cannot dirty the project. +- Same-size source edits invalidate the affected registry/manifests deterministically. +- Model, texture, and audio IDs survive uniquely identifiable moves by content. +- Initial migration adds fingerprints to imported registry records and upgrades static manifests to + schema v4. Legacy animation files can remain byte-identical indefinitely because their timestamp + field is ignored. +- Startup hashing reads imported source bytes. Future scan caches may use metadata to avoid work, + but they must verify content before persisting a changed identity. +- External glTF/FBX dependency paths remain explicit manifest dependencies; extending fingerprints + to a dependency graph requires a separate versioned contract. diff --git a/docs/editor/animation-authoring.md b/docs/editor/animation-authoring.md index 2afe794..58c1849 100644 --- a/docs/editor/animation-authoring.md +++ b/docs/editor/animation-authoring.md @@ -68,10 +68,11 @@ serialize into authored scenes. `SkinnedMeshRenderer`, skeleton references, and clip references carry the model registry UUID. Generated artifacts are keyed by that UUID; source paths are cached resolution and repair hints. When one -model is moved or renamed on disk, catalog refresh reconciles it by its animation-manifest content -hash, preserves the UUID, and regenerates both manifests with the new source path. An identical copy -is not treated as a move while the original still exists. Ambiguous content matches intentionally -receive a new identity rather than silently binding the wrong asset. +model is moved or renamed on disk, catalog refresh reconciles it by the registry's shared +content-addressed source fingerprint, preserves the UUID, and regenerates both manifests with the +new source path. The same registry policy covers textures and audio clips. An identical copy is not +treated as a move while the original still exists. Ambiguous content matches intentionally receive +a new identity rather than silently binding the wrong asset. Scene schema v3 migrates legacy animated actors that paired `ModelRef` with `AnimationControllerDesc` to the dedicated skinned renderer. Generic `ImportedModel + ModelRef` @@ -83,6 +84,12 @@ used as identity. Animation-manifest schema v1 artifacts must be reimported to g signatures; schema v3 adds the explicit default-clip ID. Schema v2 deserializes as Imported rest pose and never receives an inferred default. +Source identity uses exact byte length plus BLAKE3 and never checkout mtime. Existing animation +schema-v3 files may retain a legacy `modified_unix_secs` field in their exact bytes; the current +reader ignores it, and an equivalent refresh preserves the whole file unchanged. Static-mesh schema +v4 uses the same content fingerprint. See +[ADR 0043](../adr/0043-content-addressed-import-fingerprints.md). + ## Validation And Format Boundary Interactive diagnostics and `cargo validate-levels` resolve registry UUIDs, generated manifests, diff --git a/docs/editor/architecture.md b/docs/editor/architecture.md index ca3d3c6..719d51f 100644 --- a/docs/editor/architecture.md +++ b/docs/editor/architecture.md @@ -224,7 +224,11 @@ the Edit-to-Play boundary restore the complete runtime snapshot. See - **Processing:** the asset registry generates normalized model manifests under `assets/meshes/generated/`. The manifests store stable part IDs, glTF/FBX mesh/material subasset labels, whether each part is skin-bound, source metadata, dependencies, and import settings. - These artifacts are hidden from the Asset Browser catalog. + Model, texture, and audio registry records use byte length plus BLAKE3 source identity; model + manifests use the same policy. Semantic RON comparison preserves exact existing bytes for an + equivalent refresh, so checkout mtimes and formatting do not dirty project content. See + [ADR 0043](../adr/0043-content-addressed-import-fingerprints.md). These artifacts are hidden from + the Asset Browser catalog. - **Browser subassets:** model rows can expand into a content shelf backed by the generated manifest. Unrigged mesh subassets place as independent `StaticMeshRenderer` actors. A skinned subasset places its owning source through `SkinnedMeshRenderer`, retaining joints and inverse bind diff --git a/docs/editor/release-notes.md b/docs/editor/release-notes.md index 9fabf4c..4b0cf0c 100644 --- a/docs/editor/release-notes.md +++ b/docs/editor/release-notes.md @@ -2,7 +2,8 @@ ## Shipped -- Stable asset registry (`assets/.index/registry.ron`) with import settings +- Stable asset registry (`assets/.index/registry.ron`) with import settings and content-addressed + model/texture/audio source fingerprints - Prefab instances (`PrefabInstance`) + save-as-prefab workflow - Unsaved-scene guards on New/Open/Recent - Hierarchy additive select, reparent/unparent undo, multi-entity gizmo delta