Network binding service with Google Cloud

I was trying my “microsample” charm (rev: 9) on google cloud. The charm starts a simple flask application and tries to bind it to a specific ip.

This is the code where is is set and the critical path is here below:

# Get address + port
ADDRESS=$(unit-get public-address)
PORT=$(config-get port)

# set the microsample snap configuration
snap set microsample port=${PORT}
snap set microsample address=${ADDRESS}

So, the problem is now that the service tries to start up on the public ip, which is the real external IP in google-cloud, but which is not available on the deployed host.

This makes the service not able to start.

I have to ask here, what is the correct way to make my service properly set the address to listen to for ?

What address should I use to be able to:

  1. Reach it from internet (after a expose)
  2. To be able to relate it to a reverseproxy deployed with juju?

Using the “public-ip” is clearly the wrong way, but how?

(Update: I discovered differences between GCP and AWS here in relation to “private addresses” described below)

I also see that on GCP, the hooks tool command returns the IP address for the FAN network when requesting the private-address:

Google Cloud Platform

juju run --unit microsample/1 'unit-get private-address'
252.0.64.1

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc fq_codel state UP group default qlen 1000
    inet 10.166.0.4/32 scope global dynamic ens4
       valid_lft 2855sec preferred_lft 2855sec
3: fan-252: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
    inet 252.0.64.1/8 scope global fan-252
       valid_lft forever preferred_lft forever
Connection to 35.228.109.158 closed.

Where on AWS - the address returned is an private address in the 172.x.x.x range (Class-B).

juju run --unit microsample/1 'unit-get private-address'
172.31.23.235

… which will also properly allow me to reach the service after a “juju expose”. (This is not possible with GCP currently)

There is clearly a difference here which makes my above mentioned charm not work on GCP. Is this a bug?

This seems like known bug too?

https://bugs.launchpad.net/juju/+bug/1761838

unit-get private-address is deprecated in favour of network-get

network-get is space and cross model relations aware and provides a more complete view of the model. I am not sure without trying it whether the use of network-get has different behaviour to what which you are seeing with unit-get private-address, but network-get is where any work would be done to fix the issue.

juju help network-get

network-get returns the network config for a given binding name. By default
it returns the list of interfaces and associated addresses in the space for
the binding, as well as the ingress address for the binding. If defined, any
egress subnets are also returned.
If one of the following flags are specified, just that value is returned.
If more than one flag is specified, a map of values is returned.
    --bind-address: the address the local unit should listen on to serve connections, as well
                    as the address that should be advertised to its peers.
    --ingress-address: the address the local unit should advertise as being used for incoming connections.
    --egress-subnets: subnets (in CIDR notation) from which traffic on this relation will originate.
1 Like

@wallyworld - I’m exploring the spaces and subnets then.

  • I’ll start with adding a new model:
juju add-model docker google/europe-north1

Added ‘docker’ model on google/europe-north1 with credential ‘juju-gce-1-sa’ for user ‘erik-lonroth’

  • Trying to look at spaces:

juju spaces
cannot list spaces: spaces not supported (not supported)
ERROR cannot list spaces: spaces not supported (not supported)

OK, so there is an ERROR? Is this not supported on GCP? Is this a bug? Am I doing something wrong?

juju --version
2.8.1-focal-amd64

  • Checking in subnets …
juju subnets 
subnets:
  10.166.0.0/20:
    type: ipv4
    provider-id: default
    provider-network-id: default
    status: in-use
    space: alpha
    zones:
    - europe-north1-a
    - europe-north1-b
    - europe-north1-c
  252.0.0.0/8:
    type: ipv4
    provider-id: default-INFAN-10-166-0-0-20
    provider-network-id: default
    status: in-use
    space: alpha
    zones:
    - europe-north1-a
    - europe-north1-b
    - europe-north1-c
  • I’ll now deploy my microsample, that I’ve changed now to try to use the “private-address” instead which I hope would NOT return the fan network. But I will also try to to use “network-get --bind-address” to see if this would also return the fan-network.

juju deploy cs:~erik-lonroth/charm-microsample-10
Located charm “cs:~erik-lonroth/charm-microsample-10”.
Deploying charm “cs:~erik-lonroth/charm-microsample-10”.

juju run --unit microsample/0 ‘network-get --bind-address website’
252.0.128.1

  • Another try.

     juju run --unit microsample/0 'network-get website'
    
    bind-addresses:
    - macaddress: c2:6f:4f:85:49:55
      interfacename: fan-252
      addresses:
      - hostname: ""
        address: 252.0.128.1
        cidr: 252.0.0.0/8
    egress-subnets:
    - 252.0.128.1/32
    ingress-addresses:
    - 252.0.128.1

My conclusion is that the FAN network is used as the binding address on GCP which is not at all what happens on AWS, which delivers the 172-class-B network which also makes my service accessible when performing “juju expose microsample”.

This must be something wrong? At least, I’m not sure how to retrieve the tennant/project private 10.x.x.x address. I need this to get the service to properly be accessible from the public-address nat:ed to the private 10 address. This is a default behavior on AWS…

The workaround described in the bug works to remedy the situation: https://bugs.launchpad.net/juju/+bug/1761838

Spaces are not yet supported on Google cloud unfortunately.
The fact that subnet discovery is supported means that it should be doable without massive effort to get space support implemented.

In providers which do support spaces, I think the fan network is explicitly filtered out from the bind addresses, but @manadart could weigh in here to correct any misconceptions I have.

So think the issues you are hitting should be resolved once Google provider supports spaces. It’s good there’s a workaround in the meantime.

Got it, thanx.

It’s a bit unfortunate that the current default behavior on gcp would cause charms to break unless modifying the model. Also the bug remains since 2018…

I like gcp really, so if I could contribute to get this fixed upstream I would be glad to assist.

That said, I still don’t know exactly how to get the 10.x.x.x private address, but I’ll figure it out.

Fan addresses are not filtered out in the results of network-get, but we do order the results consistently to ensure that they are listed after their underlay address.

This means that for VMs bound/contrained to a space (subnets) with overlays, they will correctly represent their connectivity to the underlay network. Containers bound/constrained to Fan subnets will not see the underlay, and correctly return the Fan address.

@wallyworld I misremembered my prior GCE recon regarding spaces. What I said in our conversation this morning wasn’t quite right.

As @erik-lonroth demonstrated, subnet discovery is implemented for GCE. So we can allow space definition simply by turning on the appropriate flag.

The work required is actually in the networking config passed at instance creation. Based on a browse through the compute API code, we should be able to support multi-NIC for bindings/constraints by passing a list of compute.NetworkInterface containing values for Subnetwork. It doesn’t look too* hard.

* it never does.