Juju controller bootstrap fails because it doesn't use proxy even if set

Hi, i’m trying to add a new controller upgrading from juju 2.9 to juju 3.x in a arigapped enviroment with internet proxy

the command i use is: juju_31 bootstrap iaas-lab-1
–config juju-http-proxy=$http_proxy
–config juju-https-proxy=$https_proxy
–config juju-no-proxy=$no_proxy
–config apt-ftp-proxy=$http_proxy
–config apt-http-proxy=$http_proxy
–config apt-https-proxy=$https_proxy
–config snap-http-proxy=$http_proxy
–config snap-https-proxy=$https_proxy
–model-default juju-http-proxy=$http_proxy
–model-default juju-https-proxy=$https_proxy
–model-default juju-no-proxy=$no_proxy
–model-default apt-http-proxy=$http_proxy
–model-default apt-https-proxy=$https_proxy
–model-default snap-http-proxy=$http_proxy
–model-default snap-https-proxy=$https_proxy
–config default-base=ubuntu@22.04
–config default-series=jammy
–config api-port=443
–bootstrap-image=$IMAGE_ID
–bootstrap-base=ubuntu@22.04
–bootstrap-constraints=“allocate-public-ip=false instance-type=juju arch=amd64”
–debug newcontroller3dot1

but after the VM is deployed it fails pulling the juju-controller charm as it seems it doesn’t use the proxy variable but tried a direct fetch. am i missing something?

2023-04-14 14:27:02 DEBUG juju.cmd.jujud bootstrap.go:546 saving custom image metadata
2023-04-14 14:27:02 DEBUG juju.state.pool.txnwatcher txnwatcher.go:488 txn watcher: toolsmetadata 963e85e2-649f-44db-85c4-da510f849453:3.1.0-ubuntu-amd64 #2
2023-04-14 14:27:02 DEBUG juju.state.cloudimagemetadata image.go:87 inserting cloud image metadata for released:regionOne:22.04:amd64:::custom
2023-04-14 14:27:02 DEBUG juju.state.pool.txnwatcher txnwatcher.go:488 txn watcher: cloudimagemetadata released:regionOne:22.04:amd64:::custom #2
2023-04-14 14:27:32 DEBUG juju.state open.go:207 closed state without error
ERROR cannot deploy controller application: deploying charmhub controller charm: resolving "ch:juju-controller": resolving with preferred channel: Post "https://api.charmhub.io/v2/charms/refresh": dial
 tcp 185.125.188.59:443: i/o timeout
2023-04-14 14:27:32 DEBUG cmd supercommand.go:548 error stack:
Post "https://api.charmhub.io/v2/charms/refresh": dial tcp 185.125.188.59:443: i/o timeout
github.com/juju/juju/charmhub.(*apiRequester).doOnce:178:
github.com/juju/retry.Call:188:
github.com/juju/juju/charmhub.(*apiRequestLogger).Do:243:
github.com/juju/juju/charmhub.(*httpRESTClient).Post:352:
github.com/juju/juju/charmhub.(*refreshClient).refresh:168:
github.com/juju/juju/core/charm/repository.(*CharmHubRepository).refreshOne:472:
github.com/juju/juju/core/charm/repository.(*CharmHubRepository).ResolveWithPreferredChannel:77: resolving with preferred channel
github.com/juju/juju/cmd/jujud/agent.populateStoreControllerCharm:191: resolving "ch:juju-controller"
github.com/juju/juju/cmd/jujud/agent.(*BootstrapCommand).deployControllerCharm:96: deploying charmhub controller charm
github.com/juju/juju/cmd/jujud/agent.(*BootstrapCommand).Run:376: cannot deploy controller application
2023-04-14 14:27:32 DEBUG juju.cmd.jujud main.go:288 jujud complete, code 0, err <nil>
16:27:32 ERROR juju.cmd.juju.commands bootstrap.go:968 failed to bootstrap model: subprocess encountered error code 1
16:27:32 DEBUG juju.cmd.juju.commands bootstrap.go:969 (error details: [{github.com/juju/juju/cmd/juju/commands.(*bootstrapCommand).Run:1067: failed to bootstrap model} {github.com/juju/juju/environs/b
ootstrap.Bootstrap:736: } {github.com/juju/juju/environs/bootstrap.bootstrapIAAS:667: } {subprocess encountered error code 1}])
16:27:32 DEBUG juju.cmd.juju.commands bootstrap.go:1778 cleaning up after failed bootstrap
16:27:32 INFO  juju.provider.common destroy.go:21 destroying model "controller"
16:27:32 INFO  juju.provider.common destroy.go:32 destroying instances

Thanks for reporting - it appears to be a bug which we’ll need to fix. Can you file a bug (so that you can track progress of a fix).

In the meantime, you could dowload the juju-controller charm and specify it at bootstrap using

--controller-charm-path (= "")
    Path to a locally built controller charm

yes, i could deploy with that switch, but then i hit another issue in the migration i posted here: How to upgrade your Juju deployment from 2.9 to 3.x)