Issue with Dashboard proxy

juju
1

Hello, I’m pretty new here, and just working through the getting started guides. I’ve been working for over a day to deploy the dashboard, including tearing down and recreating my k8s cluster and building all from scratch. How I got here: Create AKS cluster.

  1. az aks get-credentials --resource-group rg-xxx --name cluster-name
  2. juju add-k8s sbx-k8s-cloud --cluster-name cluster-name
  3. juju bootstrap sbx-k8x-cloud
  4. juju switch controller
  5. juju deploy juju-dashboard-k8s dashboard
  6. juju integrate dashbord controller
  7. juju config dashboard juju-external-hostname=20.125.xx.yyy.sslip.io #juju expose demands a hostname. I don’t know why or what hostname it wants, this is the IP address for the load balancer of the controller
  8. juju expose dashboard
  9. juju dashboard Dashboard gives:
➜  sbx-steve git:(feature/charmed-poc) ✗ juju dashboard --debug                                                     
13:23:17 INFO  juju.cmd supercommand.go:56 running juju [3.2.0  gc go1.20.4]
13:23:17 DEBUG juju.cmd supercommand.go:57   args: []string{"juju", "dashboard", "--debug"}
13:23:17 INFO  juju.juju api.go:86 connecting to API addresses: [20.125.xx.yyy:17070]
13:23:18 DEBUG juju.api apiclient.go:1167 successfully dialed "wss://20.125.xx.yyy:17070/api"
13:23:18 INFO  juju.api apiclient.go:702 connection established to "wss://20.125.xx.yyy:17070/api"
13:23:19 DEBUG juju.api monitor.go:35 RPC connection died
ERROR running connection runner: starting tunnel proxy: connecting k8s proxy: Get "https://10.0.0.1:443/api/v1/namespaces/controller-sbx-k8s-cloud-westus3/services/dashboard": dial tcp 10.0.0.1:443: connect: connection refused
13:23:19 DEBUG cmd supercommand.go:548 error stack: 
Get "https://10.0.0.1:443/api/v1/namespaces/controller-sbx-k8s-cloud-westus3/services/dashboard": dial tcp 10.0.0.1:443: connect: connection refused
github.com/juju/juju/caas/kubernetes/provider/proxy.(*Proxier).Start.func1:117: connecting k8s proxy
github.com/juju/juju/cmd/juju/dashboard.tunnelProxyRunner.func1:255: starting tunnel proxy
github.com/juju/juju/cmd/juju/dashboard.(*dashboardCommand).Run.func3:175: running connection runner
github.com/juju/juju/cmd/juju/dashboard.(*dashboardCommand).Run:192: 

➜  sbx-steve git:(feature/charmed-poc) ✗ kubectl get svc -A 
NAMESPACE                          NAME                  TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)           AGE
controller-sbx-k8s-cloud-westus3   controller-service    LoadBalancer   10.0.255.80   20.125.xxx.yyy   17070:30706/TCP   156m
controller-sbx-k8s-cloud-westus3   dashboard             ClusterIP      10.0.112.69   <none>          65535/TCP         130m
controller-sbx-k8s-cloud-westus3   dashboard-endpoints   ClusterIP      None          <none>          <none>            130m
controller-sbx-k8s-cloud-westus3   modeloperator         ClusterIP      10.0.5.196    <none>          17071/TCP         154m
default                            kubernetes            ClusterIP      10.0.0.1      <none>          443/TCP           4h48m
kube-system                        kube-dns              ClusterIP      10.0.0.10     <none>          53/UDP,53/TCP     4h48m
kube-system                        metrics-server        ClusterIP      10.0.138.17   <none>          443/TCP           4h48m
➜  sbx-steve git:(feature/charmed-poc) ✗ kubectl describe svc -n controller-sbx-k8s-cloud-westus3 controller-service 
Name:                     controller-service
Namespace:                controller-sbx-k8s-cloud-westus3
Labels:                   app.kubernetes.io/managed-by=juju
                          app.kubernetes.io/name=controller
Annotations:              controller.juju.is/id: df4ad5d7-c8be-4cef-8dec-c48b542053e2
Selector:                 app.kubernetes.io/name=controller
Type:                     LoadBalancer
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.0.255.80
IPs:                      10.0.255.80
LoadBalancer Ingress:     20.125.xx.yyy
Port:                     api-server  17070/TCP
TargetPort:               17070/TCP
NodePort:                 api-server  30706/TCP
Endpoints:                10.244.0.15:17070
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

admin@sbx-k8s-cloud-westus3:controller$ show-application controller
controller:
  charm: juju-controller
  base: ubuntu@22.04
  channel: 3.2/stable
  constraints:
    arch: amd64
    mem: 1536
  principal: true
  exposed: true
  exposed-endpoints:
    "":
      expose-to-cidrs:
      - 0.0.0.0/0
      - ::/0
  remote: false
  life: alive
  endpoint-bindings:
    "": alpha
    dashboard: alpha
    website: alpha
admin@sbx-k8s-cloud-westus3:controller$ show-application dashboard
dashboard:
  charm: juju-dashboard-k8s
  base: ubuntu@22.04
  channel: stable
  constraints:
    arch: amd64
  principal: true
  exposed: true
  exposed-endpoints:
    "":
      expose-to-cidrs:
      - 0.0.0.0/0
      - ::/0
  remote: false
  life: alive
  endpoint-bindings:
    "": alpha
    controller: alpha
    dashboard: alpha
2

Hi @stvdilln, could you try without setting the external hostname and also skip the juju expose step? I think juju dashboard provides external access to the dashboard so it probably doesn’t need to be exposed as well.

3

I gave that try, and same result. The expose command came off of the blog about getting the dashboard running. I ran unexpose on controller and dashboard and ‘config reset’ on the hostnames setting, but same error. I looked through the source code and a proxy factory is creating this which I assume is running on my pc and not the controller. In which case contacting the Kubernetes API on a 10.x.x.x is wrong and needs to be public IP. I created a test pod and that url is reponsive inside the cluster.

4

Ah OK, in that case I think this might need some input from someone familiar with AKS and how the juju dashboard command sets up the proxy.

1 Like