New Juju interactive shell

[wallyworld (Ian Booth)]

Juju has a new interactive shell.
It is available for testing in the 2.8 edge snap and upcoming 2.8.2 release candidate.

To invoke the shell, simply run juju without any arguments. The prompt will tell you your current controller and model and who you are logged in as. You can run any Juju command, including those which require prompts, and coloured output also works as expected. There’s also support for command history search via Ctrl+R, Ctrl+S, line editing navigation via Ctrl+A and Ctrl+E etc, and other such shell key bindings.

Note: the shell currently requires at least one controller be available to work. You can bootstrap new controllers or destroy controllers from the shell, but the shell will exit if there’s no controller available.

$ juju
Welcome to the Juju interactive shell.
Type "help" to see a list of available commands.
Type "q" or ^D or ^C to quit.

admin@ctrl:mymodel$ 
admin@ctrl:mymodel$ controllers
Use --refresh option with this command to see the latest information.

Controller  Model    User   Access     Cloud/Region        Models  Nodes  HA  Version
ctrl*       mymodel  admin  superuser  microk8s/localhost       2      1   -  2.9-beta1  

admin@ctrl:mymodel$ 
admin@ctrl:mymodel$ bootstrap lxd test
Creating Juju controller "test" on lxd/localhost
Looking for packaged Juju agent version 2.8.2 for amd64
No packaged binary found, preparing local Juju agent binary
Launching controller instance(s) on localhost/localhost...
 - juju-6380b7-0 (arch=amd64)          
Installing Juju agent on bootstrap instance
Fetching Juju Dashboard 0.2.0
Waiting for address
Attempting to connect to 10.115.246.87:22
Connected to 10.115.246.87
Running machine configuration script...
Bootstrap agent now started
Contacting Juju controller at 10.115.246.87 to verify accessibility...
Bootstrap complete, controller "test" is now available
Controller machines are in the "controller" model
Initial model "default" added
admin@test:default$
admin@test:default$ deploy mariadb
Located charm "cs:trusty/mariadb-7".
Deploying charm "cs:trusty/mariadb-7".
admin@test:default$ status
Model    Controller  Cloud/Region         Version  SLA          Timestamp
default  test        localhost/localhost  2.8.2    unsupported  12:09:40+10:00

App      Version  Status   Scale  Charm    Store       Rev  OS      Notes
mariadb           waiting    0/1  mariadb  jujucharms    7  ubuntu  

Unit       Workload  Agent       Machine  Public address  Ports  Message
mariadb/0  waiting   allocating  0                               waiting for machine

Machine  State    DNS  Inst id  Series  AZ  Message
0        pending       pending  trusty      starting

admin@test:default$ switch 
test:admin/default
admin@test:default$ switch ctrl
test:admin/default -> ctrl:admin/mymodel
admin@ctrl:mymodel$ whoami
Controller:  ctrl
Model:       mymodel
User:        admin
admin@ctrl:mymodel$ show-model
mymodel:
  name: admin/mymodel
  short-name: mymodel
  model-uuid: 9dcc1ecf-6ed4-4101-8a9b-4496b2a98955
  model-type: caas
  controller-uuid: 612965a0-fea6-449e-8a98-f0acdc161c43
  controller-name: ctrl
  is-controller: false
...

[zicklag]

Interesting! Is this by any chance related to the ideas around the new interactive shell for the Juju Dashboard?

[Dashboard] Juju web CLI investigation

In the screenshots of the Juju Dashboard, you may have noticed the persistent terminal UI that has been temporarily removed. We’ve been working on a few different approaches of providing a terminal-like experience that is both secure and fully-featured-enough to be useful in environments of various levels of security requirements. [16%20PM] Outside of the security implications, there will always be a tension between the available feature set of your local shell vs a remote one. We wouldn’t be …

[wallyworld (Ian Booth)]

Funny you should ask… we are working on an idea for the shell for the GUI dashboard to be implemented by a component in the controller which does not need any messy co-located LXD container as such. (the basis for the original GUI shell). Turns out we could package something to start with which can provide a nice CLI shell so we decided to do that. Hopefully it’s useful for folks.

[zicklag]

Cool, that’s a great idea. With the shell integrated into the controller that would make it way easier to develop an integrated web shell.

[jameinel (John A Meinel)]

I like the idea of an interactive shell being able to hold a bit of state and giving you prompts towards information.
However, introducing an interactive behavior by just calling ‘juju’ feels very much like a CLI compatibility break that doesn’t feel safe to change in a minor release. Would it be reasonable for 2.8.2 to have it as ‘juju shell’ or somesuch and then do the ‘by default’ as more of a 2.9 change (if we want it faster than a 3.0).

[petevg (Pete Vander Giessen)]

@jameinel: I see your concerns, and I think that answer probably depends on how the container that runs the interactive shell for the web CLI works. The idea is to have a container that contains only juju – no bash support at all – and my *nix fu is not strong enough to understand whether or not you could have a container that contained only juju but also the ability to parse that single “shell” argument.

I do know that the only change in behavior here is that you’d need to run juju --help to see the help, rather than just running juju, but I’m not sure that anybody would have scripted around that particular behavior.

[zicklag]

@petevg It is completely possible to run a specific command in a container, i.e. juju shell without needing bash in the container. If it were a Dockerfile it would be as simple as:

ENTRYPOINT "/bin/juju" # Or wherever Juju is
CMD [ "shell" ]

Otherwise I’m sure it’s just as easy in LXD. Executables, even without the presence of a shell such as bash can still be run with arguments.

[wallyworld (Ian Booth)]

With compatibility, we really care about not breaking people’s scripts, and users should be able to type what they have become used to get the job done etc. So any juju foo changes should only be additive to what’s there already.

But in this case we are talking about just juju invoked without any args. No one would have this in a script, and before this change it just prints some help text (as does juju -h or juju help). If a user types just juju, instead of getting wall of help text they are presented with a few lines telling them to type help for more information and explaining they are in an interactive shell. We thought any slight adjustment to adapt to this change is worth the benefit of having the shell more discoverable. Also, the expectation was that most people using something for the first time would type foo help and the Juju help text has been changed to explain the interactive shell in the first few lines.

So on balance, the change seems worth it.

[zicklag]

I’d personally lean towards not breaking the API, even if it seems trivial, but I think a great middleground would be just to make sure that stdin a TTY before attempting to start the interactive shell ( which I would assume it should do anyway ). That way, if it is used in a script, it will still just display the help because in a script stdin is not a TTY.

[jameinel (John A Meinel)]

It is still a minor release where people’s interaction with the tool is surprising. This is definitely a feature change, not just a trivial bugfix. If there is a strong need for it, then we can put that sort of a thing into a minor release (we certainly have in the past, but we’ve almost always regretted breaking things in a minor release because it erodes trust).
This is less about “people are scripting ‘juju’” and more about the significant surprise factor. Something that used to be non-interactive now drops you into a shell (that isn’t otherwise a scripting case) is perfectly fine for a 2.9 vs 2.8 and I wouldn’t worry about it. But in a minor release that seems really sketchy.

[jameinel (John A Meinel)]

I can’t see how anything that can run a command (exec juju) couldn’t pass arguments (exec juju shell).

[petevg (Pete Vander Giessen)]

Good points, all. I’ll poke @wallyworld about refactoring to call w/ “juju shell” for now, with a plan to invoke “juju shell” by default instead of “juju --help” during a major release.

[zicklag]

Can you ssh into units from the Juju shell? Will it just ssh, from a network perspective, from the Juju controller?

Edit: I just tested it actually, and you can ssh. Does the ssh session happen completely inside of the virtual, shell, though, or does it still require ssh on the host like the typical juju ssh command does?

Edit: Just tested it again. It uses ssh on the host. This means that the juju interactive shell could not be used from, the Juju GUI to ssh into units, right? Is this a planned feature?

---

Bit of feedback:

Help Message is inacurate when inside shell

When a command doesn’t exist it says to run juju --help, but when running juju --help you get:

ERROR juju: "juju" is not a juju command. See "juju --help".

Did you mean:
        gui

No clear way to exit the shell

Also, exit is not a command, so I can imagine a lot of users getting stuck in the terminal because they don’t know how to type an EOF with ctrl+D.

[hatch (Jeff Pihach)]

@zicklag

Edit: Just tested it again. It uses ssh on the host. This means that the juju interactive shell could not be used from, the Juju GUI to ssh into units, right? Is this a planned feature?

There are a subset of commands, like ssh, that will be unavailable when accessing this mechanism over the api. We found that allowing certain commands created an undesirable user experience and exposed potential vulnerabilities and until we could mitigate those they needed to be disabled. We will of course continue to revisit this over time but for now ssh access will only be available in the users local shell.

[wallyworld (Ian Booth)]

When you first enter the shell, there’s a help message that tells you that Q (or q) or ^C or ^D can be used to exit the shell. “q” was used because it’s only one character and easier to type than “exit”.

The help message does need to be tweaked when inside the shell, thanks for the feedback.

With ssh, when you use the shell from the command line, all commands are available, as the shell is just a convenient wrapper around the Juju CLI. However, the Dashboard GUI is where the restrictions on what is allowed to run are.

[zicklag]

@wallyworld

When you first enter the shell, there’s a help message that tells you that Q (or q) or ^C or ^D can be used to exit the shell. “q” was used because it’s only one character and easier to type than “exit”.

Ah, I must have missed it because I only ever concerned myself with exiting after I had typed some commands, and I never bothered to read the initial prompt. I tend to ignore the initial prompt for shells, just like the big 'ol one that starts at the top of Ubuntu shells and gives a bunch of status information that I usually don’t need to read.

I do think that having an exit alias would still be helpful if anybody ends up in the boat I was in.

[sabdfl (Mark Shuttleworth)]

Agree on exit, quit and q as aliases for ‘get me out of here’

Security is a primary focus for the shell; we want GUIs for Juju to be able to offer a shell for rapid CLI access from the GUI, with confidence that they are not offering a vector into the controller itself. @zicklag let us know what you need if you want to integrate the shell capability into Lens, it should be straightforward but if there are gotchas we can make time to address them.

[wallyworld (Ian Booth)]

From a terminal, the shell is just a convenience wrapper around the normal Juju CLI.

However, the shell access from the Dashboard GUI is locked down and will be undergoing a security audit before release. The main differences when using the GUI shell compared with the shell from a terminal:

• the available commands are whitelisted (opt in); things like ssh, scp, destroy-model and many others are excluded
• access to the controller filesystem is not allowed (os calls for file open etc return errors)

There’s no direct access / shortcuts to the controller internals - the embedded CLI used by the GUI goes through the exact same API layers which perform the normal credential validation as used by the CLI from a terminal.

[zicklag]

Awesome, thanks for the explanation @wallyworld.