Charmed Apache Spark: New Stable Release Now Available!

announcements and community
1

We are excited to announce a new official stable release of Charmed Apache Spark (Rev4), now available on the 3.4/stable and 3.5/stable channels.

Building on our recent candidate release, this update delivers a production-ready, fully open-source data lake experience. It brings together significant performance enhancements—including GPU and ARM support—alongside critical security patches and component upgrades to ensure a seamless analytics experience on Kubernetes.

What’s New?

This release introduces several major features and updates across the Charmed Spark ecosystem:

  • GPU Support: You can now leverage NVIDIA GPUs to accelerate your Spark workloads. Check out our How-to use GPU guide to get started.

  • Experimental ARM Support: We’ve introduced support for ARM64 architectures (available on candidate risks), allowing you to optimize costs and performance on ARM-based infrastructure.

  • Component Upgrades:

    • Apache Spark: Updated to versions 3.4.4 and 3.5.7.

    • Apache Kyuubi: Updated to 1.10.3, featuring a reworked health check system and improved metrics dashboards.

    • NVIDIA Spark-Rapids: Updated to 25.12.0.

  • Enhanced Storage & Integration:

    • Validation for MicroCeph as a lightweight S3-compliant alternative to MinIO.

    • Support custom certificate for S3-compliant storage.

    • Improved Spark Integration Hub experience with centralized Spark configuration and secret management.

  • Infrastructure & Security:

    • Full compatibility with Juju 3.6.13+.

    • Integration with oauth2proxy for the Spark History Server.

    • Canonical security-maintained OCI images updated for better reliability and vulnerability management.

Get Started

You can find the updated charms on Charmhub.io or deploy them directly using our comprehensive deployment guide.

Acknowledgements

We would like to thank the Apache Spark and Apache Kyuubi communities. Your continuous work makes it possible for us to deliver a robust, open-source solution for processing data at scale.

Let’s build the future of open-source data processing together!

3 Likes