JUJU_DEV_FEATURE_FLAGS=raw-k8s-spec Did Not Work as Expected

juju
1

I was trying to play around with the raw-k8s-spec feature flag but did not work as expected. This is what I did. What am I missing?

# Set up system. There's a problem with Juju 2.8.1 and microk8s 1.18
# So we trick Juju 2.8.1 into thinking it's just connecting to a
# regular kubernetes cluster
sudo snap remove microk8s
sudo snap refresh --channel=2.8/stable juju --classic
sudo snap install --channel=1.18/stable microk8s --classic
sudo snap install --channel=1.18/stable kubectl --classic
sudo microk8s.config > ~/.kube/config
juju add-k8s k8s-1.18 --storage=hostpath

# And we're off to the races!
export JUJU_DEV_FEATURE_FLAGS=raw-k8s-spec
export JUJU_FEATURES=raw-k8s-spec
juju bootstrap k8s-1.18 juju-2-8-1
juju add-model demo
juju deploy ./doperator.charm --resource doperator-image=grafana/grafana:latest  # Built from https://github.com/relaxdiego/doperator
juju exec --application doperator -- status-set active
kubectl exec -it doperator-operator-0 -- /bin/bash
cat >rc.yaml<<EOF
kind: ReplicationController
apiVersion: v1beta3
metadata:
  name: frontend-v2
  labels:
    name: frontend
    group: kubersample
spec:
  replicas: 4
  selector:
    name: frontend
    version: v2
    group: kubersample
  template:
    metadata:
      labels:
        name: frontend
        version: v2
        group: kubersample
    spec:
      containers:
      - name: kubersample
        image: relaxdiego/kubersample:v2
        command:
        - /usr/local/bin/ruby
        - /usr/share/kubersample/app.rb
        ports:
        - containerPort: 4567
          name: app
          protocol: TCP
EOF
exit

juju exec --application doperator -- k8s-raw-set --file /var/lib/juju/rc.yaml
juju debug-log

# Clean up
juju destroy-model --destroy-storage --force demo
juju destroy-controller --destroy-storage juju-2-8-1
2

I suspect that the error log message saying the feature flag wasn’t enabled might be misleading. I really should review my yaml files and not just blindly pull them out of old repos without reviewing if their apiVersion field is still valid. I’m going to try again later today but with a valid manifest.

3

Meta: I’ve put this in the #devel category because it relates to internal Juju capabilities

4

What were you expecting?

5

@wallyworld @kelvin.liu might know whats going on here.

6

Sorry about that. Should’ve expounded on that in the first place.

I expected this to create the raw k8s resources that I specified:

juju exec --application doperator -- k8s-raw-set --file /var/lib/juju/rc.yaml
7

That part of the k8s manifest was causing errors when I try to create the resources directly via kubectl apply. Changing that to v1 fixes the problem on that level. I’m going to try and rerun the hook tool I used above using the fixed k8s manifest and see if that works.

8

Hi Mark,
It’s a controller config, you have to do it like this

$ juju controller-config features='[raw-k8s-spec]'

# or specify it in bootstrap time;

$ juju bootstrap microk8s k1  --config='features=[raw-k8s-spec]'
1 Like
9

Thanks for this correction, @kelvin.liu! I’ve tried it and that seems to have turned on the feature. However I am now faced with the following error when I run the command juju exec --application doperator -- k8s-raw-set --file /var/lib/juju/rc.yaml:

application-doperator: 13:22:48 ERROR juju.worker.uniter.context cannot apply changes: state changing too quickly; try again soon
application-doperator: 13:22:48 ERROR juju.worker.uniter resolver loop error: executing operation "run action 4": action "juju-run" (via unknown/invalid hook handler) failed: cannot apply changes: state changing too quickly; try again soon
application-doperator: 13:22:48 ERROR juju.worker.caasoperator exited "doperator/0": executing operation "run action 4": action "juju-run" (via unknown/invalid hook handler) failed: cannot apply changes: state changing too quickly; try again soon
application-doperator: 13:22:51 ERROR juju.worker.uniter resolver loop error: unknown operation kind run-action
application-doperator: 13:22:51 ERROR juju.worker.caasoperator exited "doperator/0": unknown operation kind run-action
application-doperator: 13:22:55 ERROR juju.worker.uniter resolver loop error: unknown operation kind run-action
application-doperator: 13:22:55 ERROR juju.worker.caasoperator exited "doperator/0": unknown operation kind run-action
application-doperator: 13:22:58 ERROR juju.worker.uniter resolver loop error: unknown operation kind run-action
application-doperator: 13:22:58 ERROR juju.worker.caasoperator exited "doperator/0": unknown operation kind run-action

Exact steps taken follows:

# Set up system. There's a problem with Juju 2.8.1 and microk8s 1.18
# So we trick Juju 2.8.1 into thinking it's just connecting to a
# regular kubernetes cluster
sudo snap remove microk8s
sudo snap remove juju
sudo snap install --channel=2.8/stable juju --classic
sudo snap install --channel=1.18/stable microk8s --classic
sudo microk8s.enable dns dashboard registry storage metrics-server ingress
sudo snap install --channel=1.18/stable kubectl --classic
mkdir -p ~/.kube
sudo microk8s.config > ~/.kube/config
juju add-k8s k8s-1.18 --storage=hostpath

# And we're off to the races!
juju bootstrap k8s-1.18 juju-2-8-1 --config='features=[raw-k8s-spec]'
juju add-model demo
juju deploy ./doperator.charm --resource doperator-image=grafana/grafana:latest  # Built from https://github.com/relaxdiego/doperator
juju exec --application doperator -- status-set active
kubectl exec -it doperator-operator-0 -- /bin/bash
cat >rc.yaml<<EOF
kind: ReplicationController
apiVersion: v1
metadata:
  name: frontend-v2
  labels:
    name: frontend
    group: kubersample
spec:
  replicas: 4
  selector:
    name: frontend
    version: v2
    group: kubersample
  template:
    metadata:
      labels:
        name: frontend
        version: v2
        group: kubersample
    spec:
      containers:
      - name: kubersample
        image: relaxdiego/kubersample:v2
        command:
        - /usr/local/bin/ruby
        - /usr/share/kubersample/app.rb
        ports:
        - containerPort: 4567
          name: app
          protocol: TCP
EOF
exit

juju exec --application doperator -- k8s-raw-set --file /var/lib/juju/rc.yaml
10

Hi, @mmaglana I think that state changing too quickly error may be caused by k8s-raw-set was called after k8s-spec-set.
So one application can either use k8s spec or k8s raw spec and it is changeable after the first run.
Also, it would be better to call charms.layer.caas_base.k8s_[raw|spec]_set to test specs via upgrade-charm.

1 Like