The Apache Kyuubi K8s charm is the component of the Charmed Spark solution that enables users to query dataset stored in object storage (e.g. S3 or Azure Blob storage) using SQL dialect and leveraging on the Spark parallel processing capabilities to do the computation and aggregation, providing querying capabilities on top of large datasets, useful for Big Data analytics.
The Apache Kyuubi K8s charm supports the integration with existing charms, among which:
Spark Integration Hub charm in order to provide and configure the Spark service account to be used to run Spark engines.
More information about how the Kyuubi server fit in the Charmed Spark solution can be found in the explanation section. The bundled solution can be easily deployed using terraform as well.
The CONTRIBUTING.md has a 404 link for dev setup (because of Juju doc rearrangement, I think).
Please add an icon to improve the look of the Charmhub page.
Could you add an optional: false to the relations that are required? Itās the default, but the current ask that that itās explicitly added to all the relations so that itās clear that itās required rather than ādidnāt look at whether itās requiredā (which is obvious in this case looking at the other relations, but wouldnāt be to an automated tool).
Review item
Objective
Review criteria
Review comment
Intended functionality
Despite all the items for publication readiness, the charm must work.
Charm does what it is meant to do - ideally done in a demo.
A complete and consistent appearance of the charm is required for a quality impression of the charm collection.
The overall appearance looks good, which means: * The name complies with the naming guidelines. * The publisher is identified. * The links are provided. * The documentation looks reasonable.
Source repository
Generally, the source code for charms must be accessible by the community for transparency and collaboration.
It is not entirely mandatory to have the charm published as OSS for review, but the repository must be accessible from the persons working on the review request.
Coding conventions
The source code of the charm is accessible in the sense of approachability. Consistent source code style and formatting are also considered a sign of being committed to quality.
The implemented checks for coding conventions are reasonable and implemented in the regular CI/CD implementation.
Release automation implementation
An implementation for automated releasing to charmhub.io improves the ability to provide updates covering vulnerabilities quickly.
Release automation for unstable channels to enable testing when new versions of the charm code or the workload become available.
Installation test implemented (could be part of the integration test)
In particular, for the charm review, assuring a reasonable test suite is essential to allow for automated releases in future. With this test, for every build, it is ensured that the installation is successful.
An implementation for checking the installation is present. The implementation should also check for successful installation as part of the automation, and the workload behaves as expected. At the time of review, the test runs successfully.
Installation test results
Availability of test results is mandatory for a working collaborative project.
URL to test results from CI/CD automation.
Integration tests implemented
In particular for the review of charms, assuring a reasonable test suite is important to allow for automated releases in future. With this test, for every build, it is ensured that the integration with other charms is successful.
An implementation for testing the required integrations (if applicable) is present. The implementation should also check for successful integration as part of he automation and the workload behaves as expected. At the time of review, the test runs successfully.
Integration test results
Availability of test results is mandatory for a working collaborative project.
URL to test results from CI/CD automation.
Documentation for usage
The documentation for using the charm should be separate from the documentation for developing or contributing to the charm.
URL to this documentation is present.
Documentation for contributing
The documentation for contributing to the charm should be separate from the documentation for developing or using the charm.
URL to this documentation is present.
Licensing statement
For the charm shared, OSS or not, the licensing terms of the charm are clarified (which also implies an identified authorship of the charm).
Just another question: given that all the āSparkā related charms have been now listed (spark-history server, spark integration hub and kyuubi), we would also like to request to get the Charmed Spark bundle listed.
@tony-meyer: I believe you have already reviewed part of its repository since a bunch of integration tests (when it comes to the integration between the various charms) are sitting here and part of its CI.
I can also raise a different thread for following up on this, but Iād like your guidance on how to approach this.
Excellent question :). We donāt really have a process for approving bundles - we were going to work on developing one, but then decided that since bundles are deprecated and new ones canāt be created, it wouldnāt be worth the time.
As you said, Iāve already seen some of it, as well as several of the charms. Iāll take an ad-hoc look at the repo & Charmhub page roughly based on the charm review process. Itāll probably be Friday before I can do that, though, sorry. I think we can keep using this port rather than making a new one.
@tony-meyer We have synced internally also with product, and indeed, given that yaml are deprecated, listing this may not be appropriate, providing false expectation that this is going to be maintained.
The reasoning for listing the bundle is rather that (since we have linked the charmhub page of the bundle to the Discourse documentation) this would provide a bit more visibility to the Spark docs, rather than the need of the bundle itself. But the docs will soon be migrated to RTD, so also this use-case is going to not be a relevant one.
Given this, it is probably sane to leave things as is, with the bundle unlisted, and the single charms listed
I believe we can close this thread! Thank you all!
