New
Release
Version 6.3.0 Live ! 🎉😊
Ilum 6.3.0 is here, a robust update designed to supercharge your data workflows and elevate your Spark experience. This release brings a host of new features, performance enhancements, and essential integrations that empower advanced data processing and governance for Spark users.
New Features in This Release
New Features in This Release
1. Introducing Gitea Integration: Enhancing Version Control Inside Ilum
We’re excited to announce that Gitea—a lightweight, self-hosted Git repository manager—is now built into Ilum. Manage version control for Jupyter notebooks, Spark job scripts, Airflow DAGs, and more, all without leaving the platform. This seamless integration simplifies collaboration for data engineers, data scientists, and DevOps teams.
We’re excited to announce that Gitea—a lightweight, self-hosted Git repository manager—is now built into Ilum. Manage version control for Jupyter notebooks, Spark job scripts, Airflow DAGs, and more, all without leaving the platform. This seamless integration simplifies collaboration for data engineers, data scientists, and DevOps teams.
2. Internal Security System Improved
Our new internal security system strengthens authorization and authentication. User data is securely stored in a dedicated database, with Role-Based Access Control (RBAC) enabling you to organize users into roles and groups, ensuring every action is properly permissioned.
Our new internal security system strengthens authorization and authentication. User data is securely stored in a dedicated database, with Role-Based Access Control (RBAC) enabling you to organize users into roles and groups, ensuring every action is properly permissioned.
3. Enhanced SQL UI with Notebook-Style Execution
Experience a more interactive SQL Editor that now supports notebook-style operations. Write and execute SQL queries in structured cells—just like in Jupyter Notebook—so you can explore and analyze big data more efficiently.
Experience a more interactive SQL Editor that now supports notebook-style operations. Write and execute SQL queries in structured cells—just like in Jupyter Notebook—so you can explore and analyze big data more efficiently.
Other Improvements
-
Changed ilum-ui Service Type:
Due to issues with kubectl port-forward, we now expose a NodePort by default. -
Security Configuration Enhancements:
Security-related settings in ilum-core have been moved from the config map to a dedicated Kubernetes Secret, enhancing the protection of sensitive data. -
Ilum Submit Configurations for Spark SQL Engines:
Launch Spark SQL engines via the Ilum Web Application or JDBC endpoint with automatic configuration application from the selected cluster—eliminating manual configuration steps. -
Embedded Git Repository:
With Gitea integrated as a module, Ilum now provides a built-in Git server for seamless version control. -
Kafka Address Configuration for ilum-core:
Added the ability to set a dedicated Kafka address for the ilum-core pod, separate from the global Kafka configuration for Spark jobs.
Happy Data Processing! 🚀✨