Keeping the juju connection open

juju
#1

Hello folks,

I’m trying to do some local development against a remote k8s server.

My ADSL connection is pretty crap, but not that crap.

juju deploy ./my.charm

repeatedly gives me:

bugg@DESKTOP-JIMKO29:~/Projects/solr-k8s-charm$ juju deploy ./solr.charm --debug
11:05:18 INFO  juju.cmd supercommand.go:54 running juju [2.8.7 0 ee2cfeb2c8c716af763a011e184ddea879c0985d gc go1.14.13]
11:05:18 DEBUG juju.cmd supercommand.go:55   args: []string{"/snap/juju/14932/bin/juju", "deploy", "./solr.charm", "--debug"}
11:05:18 INFO  juju.juju api.go:67 connecting to API addresses: [0.0.0.0:17070]
11:05:18 DEBUG juju.api apiclient.go:1107 successfully dialed "wss://0.0.0.0:17070/api"
11:05:18 INFO  juju.api apiclient.go:639 connection established to "wss://0.0.0.0:17070/api"
11:05:18 INFO  juju.juju api.go:67 connecting to API addresses: [0.0.0.0:17070]
11:05:18 DEBUG juju.api apiclient.go:1107 successfully dialed "wss://0.0.0.0:17070/model/a61c590d-0c9d-4088-8a54-002ab80cd2f1/api"
11:05:18 INFO  juju.api apiclient.go:639 connection established to "wss://0.0.0.0:17070/model/a61c590d-0c9d-4088-8a54-002ab80cd2f1/api"
11:05:18 INFO  juju.juju api.go:67 connecting to API addresses: [0.0.0.0:17070]
11:05:18 DEBUG juju.api apiclient.go:1107 successfully dialed "wss://0.0.0.0:17070/api"
11:05:18 INFO  juju.api apiclient.go:639 connection established to "wss://0.0.0.0:17070/api"
11:05:48 DEBUG juju.api monitor.go:35 RPC connection died
11:05:48 DEBUG juju.api monitor.go:35 RPC connection died
11:05:48 DEBUG juju.api monitor.go:35 RPC connection died

As soon as I do the same but on a remote server, the deploy works. Is there a way to change the timeout?

#2

The default timeout values are quite liberal and I’m not aware of the ability to change them. Additionally if no timeout is found then there is no timeout applied at all.

Maybe a ping failed to get an ACK in time?

#3

Its pretty odd, but entirely reproducible. I get no deployment from either of my local machines, but works every time with a remote box.

Odd. Maybe its something to do with WSL instead.

#4

I’m assuming 0.0.0.0 is you editing out the actual IP addresses, as those wouldn’t actually work in practice. (that would be connecting to your local machine sort of connection).

It is intriguing that it connects successfully and then it is 30s later that it finally closes the connection. That does line up with the Ping() heartbeat check. (To keep the connection alive, and detect if it has gone away, we send a message every 30s.)
That looks like we are successfully connecting, but unsuccessfully getting actual requests between the two. It also shouldn’t take 30s for deploy of a simple charm to work, though since it is a local charm, it may be that it takes some time to do the upload. Do you have a rough estimate of how long you think it should take to upload the .charm file?

#5

Thanks John, yeah I did just change the ip addresses.

If I scp the solr.charm bundle to a remote server in the same DC it takes around 45-50 seconds

Capture

I appreciate this picture is mega crude from my VPN, but the bump on the left is the scp transfer, the bump on the right is the juju deploy, so its moving data around, it just gives up before its done. I did also wonder if it was because of my VPN bonding setup, so I switched that off earlier and it made no difference.

#6

I wonder if the VPN and/or your ISP has decided that the traffic looks suspicious, and is throttling or killing the connection.

What happens if you just scp a tarball of the charm to the controller? Do you get disconnects, or does the file upload without a hitch?

#7

Yeah well I did switch off the VPN in case it was doing something funky with the connections, because I use it to bond an ADSL and 4G connection. But disconnected it just uses the ADSL connection and I’ve never known them switch off any connections before.

I don’t think I have SSH access to the nodes but I’ll check.