New Feature
15 days ago

New object-storage backends: RustFS & Rook-Ceph

Ilum's S3-compatible storage layer is now fully pluggable. MinIO remains the default, now joined by two new backends: RustFS and Rook-Ceph. All three sit behind a single stable address. You can change the storage backend underneath the whole platform without reconfiguring a single service. You can even migrate from one backend to another while everything else keeps running.
  • RustFS: a lightweight, open-source (Apache-2.0), Rust-based S3-compatible store, available as an opt-in choice. RustFS is planned to become the default in 6.8.0.
  • Rook-Ceph (Ceph RGW): point Ilum at your own pre-deployed Ceph cluster as the storage backend, with safeguards that prevent accidentally enabling two backends at once.
  • One address, every consumer: Spark, Trino, Nessie, Jupyter, MLflow, Airflow, Kestra and Langfuse all reach storage through the same address. Swapping backends needs no per-service changes.
  • One place for credentials: all backends share a single credentials source. Rotate keys in one place.
  • Guided MinIO to RustFS migration: a built-in migration copies your buckets across, with a preview run first. Your source data is never deleted unless you explicitly choose to.
Why it matters: object storage is the one dependency every other module shares, which usually makes it the hardest thing to change. Putting it behind a stable address lets you pick the store that fits your environment (lightweight RustFS, your own Ceph or classic MinIO) and migrate between them as a routine, reversible step.
RustFS console

Links: Object storage guide, Migrate between providers
Available in version: 6.7.2