We are pleased to announce the candidate releases for Charmed Apache Spark under the track 4.0.
This release brings the support for Apache Spark 4.0 to Charmed Apache Spark. Apache Spark 4.0 introduces production-ready client-server architecture with Spark Connect, default ANSI SQL compliance for higher data integrity, an optimized VARIANT data type for up to 8x faster JSON processing, and advanced Python-native APIs for simplified data source and streaming development, and many more. The full changelog for Spark 4.0.2 can be found here.
| Charm / Artifact | Channel / Tag |
|---|---|
Rock Images: charmed-spark, charmed-spark-kyuubi, charmed-spark-gpu, charmed-spark-jupyterlab |
4.0-22.04_candidate |
spark-client snap |
4.0/candidate |
spark-history-server-k8s charm |
4/candidate |
kyuubi-k8s charm |
4.0/candidate |
It is to be noted that there is no new release for spark-integration-hub-k8s charm under track 4.0 because spark-integration-hub-k8s from 3/stable is fully compatible with Apache Spark 4.
As always, we invite data scientists, ML engineers, and developers to explore the new candidate release and share valuable feedback with us!
This candidate release gives you early access to our processing solution on top of Apache Spark 4.0, along with an opportunity to influence the future of Charmed Apache Spark. Your feedback is crucial to improving our products:
- Follow our step-by-step tutorial to test the new release
- Reach out in Matrix with your questions about Charmed Apache Spark’s capabilities
- Report any bugs or issues you encounter
- Provide suggestions for improving our products
Please note: This candidate release is not intended for production use and may involve risks. We encourage you to save your work regularly and proceed with caution. Canonical’s Processing team is always ready to assist you with any feedback or issues you might have.
We plan to allow a couple of weeks for testing the current candidate release before promoting to stable. Looking forward to receiving valuable feedback from you. Good day!