New Feature
6 days ago

Savings Opportunities - right-sizing CPU & memory for Spark on Kubernetes

ILUM now recommends vCPU and memory reductions per job based on real utilization, with clear cost impact estimates.

What’s new
  • Savings Opportunities panel
    Detects under-utilization (CPU %, memory %) and proposes an optimal resource set (vCPUs, memory, executors). Shows the exact delta, e.g. “reduce from 12 cores / 24 GB → 10 cores / 7 GB.”
  • Configuration Comparison
    Side-by-side Current → Optimal for:
    • vCPUs (per job or preset)
    • Memory (executor/container)
    • Executors (kept constant when parallelism requires it)
  • Cost impact math
    Displays hourly and projected monthly savings from your rate card; highlights the driver (CPU vs memory). Uses observed runtime + price inputs; all figures are reproducible.
  • Actionable output
    • Copy the exact settings (Spark/K8s)
    • Apply to virtual-cluster preset or open a change PR for the pipeline
    • Notes include headroom assumptions and the utilization window used
How it decides
  • Aggregates recent runs’ CPU active time, memory peaks, and task parallelism.
  • Picks the smallest config that avoids spill/GC pressure and preserves recorded concurrency.
  • Enforces guardrails (min cores/memory, executor floor) and won’t downsize below observed peaks.
Where to find it
Job → Analytics → Savings Opportunities

Notes
  • Estimates depend on your price model (per-vCPU/per-GB). Update rates to keep projections accurate.