New
Feature
Dataset Versioning & Schema Diff (Hive/Delta)
ILUM now tracks version history for datasets and shows field-level diffs so you can audit changes and catch breaking schema updates quickly.
What’s new
- Version history timeline with event labels (e.g., CREATE, DROP) and timestamps.
- Schema change summary per version: counts of additions / removals / modifications.
- Field-level diff chips (e.g., FIELD REMOVED product_id (long), FIELD ADDED price (long)).
- Format & engine tags surfaced with each version (e.g., PARQUET, DELTA).
- Compare action to diff any two versions side-by-side.
- Works for tables registered in the Hive Metastore (including Delta Lake tables).
Why it’s useful
- Immediate visibility into breaking changes before downstream jobs run.
- Clean audit trail for governance reviews and incident retros.
- Faster root-cause analysis when KPIs or pipelines regress after a change.
Where to find it
Open a dataset → Versions tab.
