Release
Version 6.1.0
- Data Lineage Support Added: Enhanced data management with new lineage tracking capabilities.
- Base Storage Support for HDFS/Azure/GCS: Expanded storage options to include HDFS, Azure Blob Storage, and Google Cloud Storage.
- LDAP/OAuth2 Support: Strengthened security with added support for LDAP and OAuth2 authentication methods.
- YARN Web UI Access: Improved monitoring and management with access to the YARN web user interface.
- Prometheus Metrics in Spark Jobs: Enabled Prometheus metrics for detailed monitoring within Spark jobs.
- Hadoop Config in Config Maps: Simplified configuration by allowing Hadoop settings to be passed to Spark jobs via Kubernetes config maps.
- Cluster Storages Spark Config to Spark Jobs: Streamlined job configuration with the ability to pass cluster storage settings directly to Spark jobs.
- Detecting Spark Driver Pod Failure: Increased reliability with new mechanisms to detect Spark driver pod failures.
- Fetching Metrics from the History Server: Enhanced analytics and troubleshooting with the ability to fetch metrics from the Spark history server.
- Tags for Jobs and Groups: Improved organization and management with tagging capabilities for Jobs and Groups.
- Exposing YARN Jobs for Ilum's Prometheus: Expanded monitoring options by making YARN jobs visible to Ilum's Prometheus setup.
- Tons of Small Improvements: Implemented numerous minor enhancements across the platform to boost performance and user experience.
Release
Version 6.0.0
- UI v3
- Embedded Jupyter Notebook
- Embedded Airflow
- Encapsulate the spark-submit processes within separate kubernetes pod
- Tons of small improvements
Release
Version 5.2.0
Release
Version 5.1.0
- Now featuring an accessible Spark UI link for actively running jobs, offering seamless monitoring and management.
- Fully compatible with Ilum REST API v1.1, enabling advanced functionality.
- Streamlined license attachment during Helm installation, eliminating the need for manual intervention.
- Enhanced data filtering capabilities with the ability to list all existing spark cluster names.
- Added a robust health check endpoint for the Ilum service, ensuring system stability and reliability.
- Refined user interface with subtle yet impactful improvements, elevating the overall user experience.
Release
Version 5.0.2
- Enhanced user interface with subtle refinements for an improved user experience.
- Extended ilum-job-api to maintain seamless backward compatibility with Java 8, ensuring support for legacy systems.
Release
Version 5.0.1
- Introduced versatile, multi-platform Docker images for ilum-core, expanding compatibility and usability across various systems.
- Revitalized Bitnami Helm repository with essential updates to dependencies, ensuring seamless access and functionality.
- Implemented subtle user interface enhancements for a more polished and user-friendly experience.
Release
Version 5.0 Live NOW! 🎉😊
Introducing Ilum 5.0.
We're excited to share with you one of our biggest updates yet, packed with our most popular feature request, significant performance improvements, a revamped UI, plus plenty of additional improvements, tweaks and fixes. With this update, you can expect to see an enhanced user experience and improved performance.
Watch the video!
Some of the new features:
We're excited to share with you one of our biggest updates yet, packed with our most popular feature request, significant performance improvements, a revamped UI, plus plenty of additional improvements, tweaks and fixes. With this update, you can expect to see an enhanced user experience and improved performance.
Watch the video!
Some of the new features:
- Apache Zeppelin integration
- Jupyter integration
- New UI
- OpenAPI support
- Introduced the ability to configure gRPC communication, enabling more efficient and flexible data exchange.
- Transitioned the default Spark Docker image to docker.ilum.cloud/ilum-spark:3.3.0, providing users with the latest and most optimized version.
- Shifted the default communication mode to gRPC, emphasizing speed and performance.
- Expanded offerings with the addition of Helm charts for Ilum Livy Proxy, Jupyter SparkMagic, and Zeppelin, streamlining deployment and management processes.
- Introduced Ilum Table Format, a cutting-edge storage format designed for efficient data processing and optimized query performance, bringing innovation and enhanced capabilities to the platform.
