Review `kyuubi-k8s` for listing

The Apache Kyuubi K8s charm is the component of the Charmed Spark solution that enables users to query dataset stored in object storage (e.g. S3 or Azure Blob storage) using SQL dialect and leveraging on the Spark parallel processing capabilities to do the computation and aggregation, providing querying capabilities on top of large datasets, useful for Big Data analytics.

The Apache Kyuubi K8s charm supports the integration with existing charms, among which:

  • Spark Integration Hub charm in order to provide and configure the Spark service account to be used to run Spark engines.
  • Various data stores charm for:
    • storing raw data in object storage backend (s3-integrator or azure-storage)
    • postgresql K8s for storing metadata about dataset organization (postgresql that is used as a Hive Metastore or data catalogue)
    • postgresql K8s for storing the user data (username and encrypted password)
    • zookeeper to enable Kyuubi engine coordination
  • Observability components
  • tls certifcates operators (e.g. self-signed-certificates) to enable to expose encrypted endpoints

The Apache Kyuubi K8s charm deploys and operates a deployment of the Apache Kyuubi rock. The Apache Kyuubi K8s is part of the Charmed Spark Solution.

Documentation Links

The Apache Kyuubi operator is part of the Charmed Apache Spark solution. Please refer to the Kyuubi tutorial or the Kyuubi sections in the solution docs for more information about how to deploy it and configure it as part of the solution.

More information about how the Kyuubi server fit in the Charmed Spark solution can be found in the explanation section. The bundled solution can be easily deployed using terraform as well.

Metadata links

Codebase

PR for code review

CI Links

Further testing when part of the solution can also be found in the bundle repository:

Non-blocking issues to fix:

  • The CONTRIBUTING.md has a 404 link for dev setup (because of Juju doc rearrangement, I think).
  • Please add an icon to improve the look of the Charmhub page.
  • Could you add an optional: false to the relations that are required? It’s the default, but the current ask that that it’s explicitly added to all the relations so that it’s clear that it’s required rather than “didn’t look at whether it’s required” (which is obvious in this case looking at the other relations, but wouldn’t be to an automated tool).
Review item Objective Review criteria Review comment
Intended functionality Despite all the items for publication readiness, the charm must work. Charm does what it is meant to do - ideally done in a demo. :heavy_check_mark:
Charmhub.io charm detail page A complete and consistent appearance of the charm is required for a quality impression of the charm collection. The overall appearance looks good, which means:
* The name complies with the naming guidelines.
* The publisher is identified.
* The links are provided.
* The documentation looks reasonable.
:heavy_check_mark:
Source repository Generally, the source code for charms must be accessible by the community for transparency and collaboration. It is not entirely mandatory to have the charm published as OSS for review, but the repository must be accessible from the persons working on the review request. :heavy_check_mark:
Coding conventions The source code of the charm is accessible in the sense of approachability. Consistent source code style and formatting are also considered a sign of being committed to quality. The implemented checks for coding conventions are reasonable and implemented in the regular CI/CD implementation. :heavy_check_mark:
Release automation implementation An implementation for automated releasing to charmhub.io improves the ability to provide updates covering vulnerabilities quickly. Release automation for unstable channels to enable testing when new versions of the charm code or the workload become available. :heavy_check_mark:
Installation test implemented (could be part of the integration test) In particular, for the charm review, assuring a reasonable test suite is essential to allow for automated releases in future. With this test, for every build, it is ensured that the installation is successful. An implementation for checking the installation is present. The implementation should also check for successful installation as part of the automation, and the workload behaves as expected. At the time of review, the test runs successfully. :heavy_check_mark:
Installation test results Availability of test results is mandatory for a working collaborative project. URL to test results from CI/CD automation. :heavy_check_mark:
Integration tests implemented In particular for the review of charms, assuring a reasonable test suite is important to allow for automated releases in future. With this test, for every build, it is ensured that the integration with other charms is successful. An implementation for testing the required integrations (if applicable) is present. The implementation should also check for successful integration as part of he automation and the workload behaves as expected.
At the time of review, the test runs successfully.
:heavy_check_mark:
Integration test results Availability of test results is mandatory for a working collaborative project. URL to test results from CI/CD automation. :heavy_check_mark:
Documentation for usage The documentation for using the charm should be separate from the documentation for developing or contributing to the charm. URL to this documentation is present. :heavy_check_mark:
Documentation for contributing The documentation for contributing to the charm should be separate from the documentation for developing or using the charm. URL to this documentation is present. :heavy_check_mark:
Licensing statement For the charm shared, OSS or not, the licensing terms of the charm are clarified (which also implies an identified authorship of the charm). URL to the ruling licensing statement is present. :heavy_check_mark:

@adityagoel28 this one is good for listing, thanks!