JUJU controller tries for non-existent meta data url to pull the simplestream json file. Failing to build juju machines and applications

Hi all,

I have hit a dead end. whenever i try to add machines or deploy applications using juju. The controller machines fails to get published images meta data. It keeps on pointing to link1. this url is not accessible anymore. The installation use to work well but after Jan 1 2024 things have changes it does not works they way it was working.

I am trying install juju applications of LXD cluster

link 1 http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson

link 2 https://cloud-images.ubuntu.com/releases/

LXD version Client version: 5.19 Server version: 5.19

JUJU version’ 3.3.0-genericlinux-amd64 (tried on the classic 3.1.7 it dod not work either)

OS Jammy 22.04

Solns tried:

Hosted my own meta data internal site with pache2 it skips and defaults to link1

Changed juju version—same error

Tried with Ubuntu 20.04—same error

Boostraped the controller with link2 —same error

error

4-01-05 23:44:14 WARNING juju.worker.proxyupdater proxyupdater.go:297 unable to update apt mirrors: exit status 1, output: “” 2024-01-05 23:44:48 INFO juju.worker.logforwarder logforwarder.go:105 config change - log forwarding not enabled 2024-01-05 23:44:48 INFO juju.worker.upgrader upgrader.go:166 desired agent binary version: 3.3.0 2024-01-05 23:49:29 INFO juju.state addmachine.go:506 new machine “1” has preferred addresses: private “”, public “” 2024-01-05 23:50:30 WARNING juju.environs.simplestreams datasource.go:212 Got error requesting “link1”: Get “link1”: dial tcp 185.125.190.40:80: i/o timeout 2024-01-05 23:51:01 WARNING juju.environs.simplestreams datasource.go:212 Got error requesting “link1”: Get “link1”: dial tcp 185.125.190.37:80: i/o timeout 2024-01-05 23:51:33 WARNING juju.environs.simplestreams datasource.go:212 Got error requesting “link1”: Get “link1”: dial tcp 185.125.190.40:80: i/o timeout

There is logic to fall back from index2 to index if not found. It isn’t clear why this is not happening.

Can you change you model’s logging configuration to include juju.environs.simplestreams=DEBUG and try another deployment?

Hi ,

Thank you for your response!

I added it still I cannot see any debug logs

juju model-config -m test logging-config=“juju.environs.simplestreams=DEBUG”

controller-0: 17:17:07 DEBUG juju.worker.logger reconfiguring logging from “=DEBUG” to “juju.environs.simplestreams=DEBUG” machine-23: 17:17:07 DEBUG juju.worker.logger reconfiguring logging from “=DEBUG” to “juju.environs.simplestreams=DEBUG” unit-nginx-0: 17:17:07 DEBUG juju.worker.logger reconfiguring logging from “=DEBUG” to “juju.environs.simplestreams=DEBUG” controller-0: 17:21:00 ERROR juju.worker.provisioner 26 cannot get available image metadata: failed getting published images metadata from default ubuntu cloud images: cannot read index data, attempt count exceeded: cannot access URL “http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson”: Get “http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson”: dial tcp 185.125.190.40:80: i/o timeout controller-0: 17:23:38 ERROR juju.worker.provisioner 27 cannot get available image metadata: failed getting published images metadata from default ubuntu cloud images: cannot read index data, attempt count exceeded: cannot access URL “http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson”: Get “http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson”: dial tcp 185.125.190.40:80: i/o timeout controller-0: 17:25:13 ERROR juju.worker.provisioner 28 cannot get available image metadata: failed getting published images metadata from default ubuntu cloud images: cannot read index data, attempt count exceeded: cannot access URL “http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson”: Get “http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson”: dial tcp 185.125.190.37:80: i/o timeout controller-0: 17:27:14 ERROR juju.worker.provisioner 29 cannot get available image metadata: failed getting published images metadata from default ubuntu cloud images: cannot read index data, attempt count exceeded: cannot access URL “http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson”: Get “http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson”: dial tcp 185.125.190.40:80: i/o timeout controller-0: 17:32:31 ERROR juju.worker.provisioner 30 cannot get available image metadata: failed getting published images metadata from default ubuntu cloud images: cannot read index data, attempt count exceeded: cannot access URL “http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson”: Get “http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson”: dial tcp 185.125.190.37:80: i/o timeout controller-0: 17:35:35 ERROR juju.worker.provisioner 31 cannot get available image metadata: failed getting published images metadata from default ubuntu cloud images: cannot read index data, attempt count exceeded: cannot access URL “http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson”: Get “http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson”: dial tcp 185.125.190.37:80: i/o timeout controller-0: 17:37:18 ERROR juju.worker.provisioner 32 cannot get available image metadata: failed getting published images metadata from default ubuntu cloud images: cannot read index data, attempt count exceeded: cannot access URL “http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson”: Get “http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson”: dial tcp 185.125.190.40:80: i/o timeout controller-0: 17:43:06 INFO juju.worker.provisioner provisioning in zones: [fgdk8sqa-node1 fgdk8sqa-node2 fgdk8sqa-node3 fgdk8sqa-node4 fgdk8sqa-node5 fgdk8sqa-node6] controller-0: 17:43:06 INFO juju.worker.provisioner found machine pending provisioning id:33, details:33

Continued on Launchpad here.