Gitlab-k8s hook failed: "install"

charm
,
#1

Hello Community,

I tried to follow this tutorial and I have a problem deploying gitlab-k8s . I have this error `hook failed: “install”. I tied both ways LOCAL and using AWS and I got the same result.

Model            Controller  Cloud/Region        Version  SLA          Timestamp
test-glab-local  micro       microk8s/localhost  2.8.7    unsupported  23:17:25+01:00

App          Version                         Status  Scale  Charm        Store       Rev  OS          Address        Notes
gitlab-k8s                                   error       1  gitlab-k8s   jujucharms    1  kubernetes                 hook failed: "install"
mariadb-k8s  registry.jujucharms.com/juj...  active      1  mariadb-k8s  jujucharms    3  kubernetes  10.152.183.98  

Unit            Workload  Agent  Address     Ports     Message
gitlab-k8s/0*   error     idle                         hook failed: "install"
mariadb-k8s/0*  active    idle   10.1.100.9  3306/TCP

I have this debug-log.

pedro@ubuntu-master-worker4g-vm:~$ juju debug-log
application-gitlab-k8s: 01:25:52 WARNING unit.gitlab-k8s/0.install     from charms.reactive.helpers import data_changed
application-gitlab-k8s: 01:25:52 WARNING unit.gitlab-k8s/0.install   File "lib/charms/reactive/helpers.py", line 21, in <module>
application-gitlab-k8s: 01:25:52 WARNING unit.gitlab-k8s/0.install     from charmhelpers.core import host
application-gitlab-k8s: 01:25:52 WARNING unit.gitlab-k8s/0.install   File "lib/charmhelpers/core/host.py", line 41, in <module>
application-gitlab-k8s: 01:25:52 WARNING unit.gitlab-k8s/0.install     __platform__ = get_platform()
application-gitlab-k8s: 01:25:52 WARNING unit.gitlab-k8s/0.install   File "lib/charmhelpers/osplatform.py", line 13, in get_platform
application-gitlab-k8s: 01:25:52 WARNING unit.gitlab-k8s/0.install     tuple_platform = platform.linux_distribution()
application-gitlab-k8s: 01:25:52 WARNING unit.gitlab-k8s/0.install AttributeError: module 'platform' has no attribute 'linux_distribution'
application-gitlab-k8s: 01:25:52 ERROR juju.worker.caasoperator.uniter.operation hook "install" (via explicit, bespoke hook script) failed: exit status 1
application-gitlab-k8s: 01:25:52 INFO juju.worker.caasoperator.uniter awaiting error resolution for "install" hook
application-mariadb-k8s: 01:25:06 INFO unit.mariadb-k8s/0.juju-log Reactive main running for hook update-status
application-mariadb-k8s: 01:25:06 INFO unit.mariadb-k8s/0.juju-log Invoking reactive handler: reactive/mysql.py:21:mariadb_active
application-mariadb-k8s: 01:25:06 INFO unit.mariadb-k8s/0.juju-log Invoking reactive handler: hooks/relations/mysql/provides.py:25:_handle_broken:server
application-mariadb-k8s: 01:25:06 INFO unit.mariadb-k8s/0.juju-log status-set: active: 
application-mariadb-k8s: 01:25:07 INFO juju.worker.caasoperator.uniter.operation ran "update-status" hook (via explicit, bespoke hook script)
application-gitlab-k8s: 01:25:13 INFO juju.worker.caasoperator.uniter awaiting error resolution for "install" hook
application-gitlab-k8s: 01:25:13 WARNING unit.gitlab-k8s/0.install Traceback (most recent call last):
application-gitlab-k8s: 01:25:13 WARNING unit.gitlab-k8s/0.install   File "/var/lib/juju/agents/unit-gitlab-k8s-0/charm/hooks/install", line 8, in <module>
application-gitlab-k8s: 01:25:13 WARNING unit.gitlab-k8s/0.install     caas_base.init_config_states()
application-gitlab-k8s: 01:25:13 WARNING unit.gitlab-k8s/0.install   File "lib/charms/layer/caas_base.py", line 35, in init_config_states
application-gitlab-k8s: 01:25:13 WARNING unit.gitlab-k8s/0.install     from charms.reactive import set_state
application-gitlab-k8s: 01:25:13 WARNING unit.gitlab-k8s/0.install   File "lib/charms/reactive/__init__.py", line 22, in <module>
application-gitlab-k8s: 01:25:13 WARNING unit.gitlab-k8s/0.install     from .endpoints import *  # noqa
application-gitlab-k8s: 01:25:13 WARNING unit.gitlab-k8s/0.install   File "lib/charms/reactive/endpoints.py", line 23, in <module>
application-gitlab-k8s: 01:25:13 WARNING unit.gitlab-k8s/0.install     from charms.reactive.helpers import data_changed
application-gitlab-k8s: 01:25:13 WARNING unit.gitlab-k8s/0.install   File "lib/charms/reactive/helpers.py", line 21, in <module>
application-gitlab-k8s: 01:25:13 WARNING unit.gitlab-k8s/0.install     from charmhelpers.core import host
application-gitlab-k8s: 01:25:13 WARNING unit.gitlab-k8s/0.install   File "lib/charmhelpers/core/host.py", line 41, in <module>
application-gitlab-k8s: 01:25:13 WARNING unit.gitlab-k8s/0.install     __platform__ = get_platform()
application-gitlab-k8s: 01:25:13 WARNING unit.gitlab-k8s/0.install   File "lib/charmhelpers/osplatform.py", line 13, in get_platform
application-gitlab-k8s: 01:25:13 WARNING unit.gitlab-k8s/0.install     tuple_platform = platform.linux_distribution()
application-gitlab-k8s: 01:25:13 WARNING unit.gitlab-k8s/0.install AttributeError: module 'platform' has no attribute 'linux_distribution'
application-gitlab-k8s: 01:25:13 ERROR juju.worker.caasoperator.uniter.operation hook "install" (via explicit, bespoke hook script) failed: exit status 1
application-gitlab-k8s: 01:25:13 INFO juju.worker.caasoperator.uniter awaiting error resolution for "install" hook
application-gitlab-k8s: 01:25:46 INFO juju.worker.caasoperator.uniter awaiting error resolution for "install" hook

It seems for some reason the charm doesn’t find the linux distro.
Any ideas on how to solve this issue?

#2

Hi

So this occurs due to Py3.8 removing linux_distribution (see https://bugs.python.org/issue28167). I’m going to assume that the unit was a focal machine?

A ‘simple’ way of fixing this for you is to install the charm onto a bionic machine (use the ‘series’ feature); that will have Python3.6 and the charm should install fine on that.

The problem has been addressed in charm-helpers in PR #423, so the charm just needs an update and it will be good to go. I couldn’t find where to file a bug for the charm unfortunately.

#3

Thanks @ajkavanagh
I thought that the distro package solved this for the version after 3.5.

I will take a look at the links you shared and try bionic.
You are right I used a focal machine and also I tried with Groovy.

merry Christmas

#4

HI @ajkavanagh

I followed your suggestion, I can’t do series in a kubernetes model.

:~$ juju deploy cs:~juju/gitlab-k8s --series xenial --force
ERROR series "xenial" in a kubernetes model not valid
pbravodez1@DESKTOP-3N8CH6C:~$ juju deploy cs:~juju/gitlab-k8s --series bionic --force
ERROR series "bionic" in a kubernetes model not valid
#5

I’ve not seen series kubernetes before. The metadata of the charm indicates that the series it supports is kubernetes. I’m guessing, then, to install it, you must have a k8s cloud first, then bootstrap the controller into that k8s cloud, add a model, and then deploy the charm. This seems to be what https://juju.is/docs/kubernetes is indicating; it seems you’ve done this, as your controller is running in microk8s (from microk8s/localhost).

Thus, I’m concluding that Microk8s is installed on the focal machine? I suspect to get this to work, you’d need to install Microk8s on a bionic machine, then bootstrap, then add a model, and finally deploy the bundle.

An alternative would be to rebuild the charm and deploy it from the locally built version. The source for the layer is at https://github.com/wallyworld/caas/tree/master/charms/gitlab (AFAICT). If you do a charm build (you’ll need the charm command, which is from the charm snap (sudo snap install charm)) in the directory of the gitlab directory (the layer), then it “should” build the charm and pull in the newer charm-helpers library, which would be compatible with focal. However, if you’re not confident with the command line, etc, then deploying Microk8s on bionic would be much easier!

Getting Started
#6

Hi Alex, we have each attempted a MicroK8s install on a bionic base and continue to encounter the same error.

@wallyworld I filed a bug at https://github.com/juju/charm-base-images/issues/3 because I didn’t know the true provenance of the charm and it looks like your repo isn’t taking issues.

#7

I have rebuilt the gitlab charm and published the new revision (2).
The rebuild pulls in the latest charm helpers with the fix and a quick deploy test shows the install error is no longer an issue.

1 Like
#8

Thanks Ian,
After the upgrade, now the charm is installed but still some errors. juju debug-log
application-gitlab-k8s: 12:01:11 ERROR unit.gitlab-k8s/0.juju-log Unable to find implementation for relation: provides of http
as a result, we are unable to connect for example curl [ip-address]. If we run microk8s kubectl get all --all-namespaces you will see that the status CrashLoopBackOff

#9

Unable to find implementation for relation looks like the charm is expecting to be related to something that provides an “http” interface. I’m not aware of the specifics of the gitlab charm, and my understanding is that charms that expect an interface should be setting a Blocked status, rather than crashing, but that is what I read from that error message.

#10

The charm was missing a layer declaration for http and the newer charm helpers library seems to be more strict on ensuring this.

After fixing that issue, the pod container initialisation failed due to not running as a privileged container. The additional pod spec metadata was added for this.

Now what happens is the gitlab charm deploys fine, and can be related to a database. When I tested with mariadb, the gitlab workload ran into an initialisation issue trying to set up a mariadb connection (see below). It seems using mariadb with gitlab is deprecated (the most recent images only support postgresql). This was tested with the most recent version 11 gitlab image (12.1 onwards are postgres only).

So the issues with the gitlab charm itself are sorted but there’s more work needed to figure out how to tweak the config/setup to properly use mariadb, or to test with postgresql given that’s the preferred upstream option moving forward. The gitlab-k8s charm that’s there now was done as a proof of concept so is nowhere near production quality. It seems there’s no other version that anyone has worked on.

    Mixlib::ShellOut::ShellCommandFailed
    ------------------------------------
    Expected process to exit with [0], but received '1'
    ---- Begin output of /opt/gitlab/bin/gitlab-rake cache:clear ----
    STDOUT: 
    STDERR: rake aborted!
    Gem::LoadError: Specified 'mysql2' for database adapter, but the gem is not loaded. Add `gem 'mysql2'` to your Gemfile (and ensure its version is at the minimum required by ActiveRecord).
    /opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:6:in `<top (required)>'
    /opt/gitlab/embedded/bin/bundle:23:in `load'
    /opt/gitlab/embedded/bin/bundle:23:in `<main>'
    
    Caused by:
    Gem::LoadError: mysql2 is not part of the bundle. Add it to your Gemfile.
    /opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:6:in `<top (required)>'
    /opt/gitlab/embedded/bin/bundle:23:in `load'
    /opt/gitlab/embedded/bin/bundle:23:in `<main>'
    Tasks: TOP => cache:clear => cache:clear:redis => environment
    (See full trace by running task with --trace)
    ---- End output of /opt/gitlab/bin/gitlab-rake cache:clear ----
    Ran /opt/gitlab/bin/gitlab-rake cache:clear returned 1