I’m slightly in awe of your persistence. This isn’t a subsystem that I have a lot of familiarity with, but it certainly looks like you’ve encountered a significant bug.
If I edit accounts.yaml and put in the password I set, I’m logged in again. Presumably that is not how this is supposed to work…
I replied to the bug and noted this looks like 2.5. I’ve repeated the steps in 2.6 and 2.7 with it working correctly. I’m curious if we can test using the later client/controller combo? I know we did a bunch of solid work in 2.6 for the login/logout/auth processing.
Yes, I used a MAAS with a credential and noted the step by step in the bug post here:
After the latest notes in that bug I’m wondering if it’s something about how MAAS is setup that’s causing us to get different behavior? Can you speak to the MAAS setup? Are you using RBAC or somethings else non-default in the auth side we should be double checking?
It is very vanilla in that regard. Installed from canonical ISO. Literally I set a password on the default admin user and go.
@rick_h I have been trying to understand where in the apiserver code the absent macaroon would be issued. I can only find two candidates;
./apiserver/registration.go:61: m, err := h.ctxt.srv.authenticator.CreateLocalLoginMacaroon(userTag)
./apiserver/stateauthenticator/locallogin.go:114: m, err := h.authCtxt.CreateLocalLoginMacaroon(userTag)
That is, assuming CreateLocalLoginMacaroon() is relevant? I figure the first of these is in relation to the ‘juju register’ command.
The code in locallogin.go looks promising, but this authentication path is apparently not exercised when I use ‘juju change-user-password’. Rather, the authentication lands me here;
(dlv) bt
0 0x0000000002ec7a2f in github.com/juju/juju/apiserver/stateauthenticator.(*modelUserEntity).PasswordValid
at /home/ubuntu/go/src/github.com/juju/juju/apiserver/stateauthenticator/modeluser.go:119
1 0x0000000002928427 in github.com/juju/juju/apiserver/authentication.(*AgentAuthenticator).Authenticate
at /home/ubuntu/go/src/github.com/juju/juju/apiserver/authentication/agent.go:39
2 0x0000000002929de5 in github.com/juju/juju/apiserver/authentication.(*UserAuthenticator).Authenticate
at /home/ubuntu/go/src/github.com/juju/juju/apiserver/authentication/user.go:77
3 0x0000000002ec299a in github.com/juju/juju/apiserver/stateauthenticator.authenticator.Authenticate
at /home/ubuntu/go/src/github.com/juju/juju/apiserver/stateauthenticator/context.go:147
4 0x0000000002ec9364 in github.com/juju/juju/apiserver/stateauthenticator.(*authenticator).Authenticate
at <autogenerated>:1
5 0x0000000002ec0c8f in github.com/juju/juju/apiserver/stateauthenticator.(*Authenticator).checkCreds
at /home/ubuntu/go/src/github.com/juju/juju/apiserver/stateauthenticator/auth.go:152
6 0x0000000002ec05be in github.com/juju/juju/apiserver/stateauthenticator.(*Authenticator).AuthenticateLoginRequest
at /home/ubuntu/go/src/github.com/juju/juju/apiserver/stateauthenticator/auth.go:114
7 0x0000000002e17878 in github.com/juju/juju/apiserver.(*admin).authenticate
at /home/ubuntu/go/src/github.com/juju/juju/apiserver/admin.go:272
8 0x0000000002e157e9 in github.com/juju/juju/apiserver.(*admin).login
at /home/ubuntu/go/src/github.com/juju/juju/apiserver/admin.go:90
9 0x0000000002e15483 in github.com/juju/juju/apiserver.(*admin).Login
at /home/ubuntu/go/src/github.com/juju/juju/apiserver/admin.go:66
10 0x0000000000465015 in runtime.call512
at /snap/go/4286/src/runtime/asm_amd64.s:526
11 0x00000000004f0914 in reflect.Value.call
at /snap/go/4286/src/reflect/value.go:447
12 0x00000000004efbea in reflect.Value.Call
at /snap/go/4286/src/reflect/value.go:308
13 0x0000000000c9c70e in github.com/juju/juju/vendor/github.com/juju/rpcreflect.newMethod.func8
at /home/ubuntu/go/src/github.com/juju/juju/vendor/github.com/juju/rpcreflect/type.go:344
14 0x0000000000c9b652 in github.com/juju/juju/vendor/github.com/juju/rpcreflect.methodCaller.Call
at /home/ubuntu/go/src/github.com/juju/juju/vendor/github.com/juju/rpcreflect/value.go:129
15 0x0000000000c9d1f2 in github.com/juju/juju/vendor/github.com/juju/rpcreflect.(*methodCaller).Call
at <autogenerated>:1
16 0x0000000000ca19a8 in github.com/juju/juju/rpc.(*Conn).runRequest
at /home/ubuntu/go/src/github.com/juju/juju/rpc/server.go:571
17 0x0000000000466ab1 in runtime.goexit
at /snap/go/4286/src/runtime/asm_amd64.s:1333
None of AuthenticateLoginRequest() or anything called appears to do anything with macaroons.
What am I missing here? At what point should a macaroon be issued when I do;
juju bootstrap mycloud mycontroller
juju change-user-password
?
@rick_h this is what appears to be the problem. Ignore mostly everything above!
testuser@juju:~$ python -m json.tool <.local/share/juju/cookies/mycontroller.json
[
{
"CanonicalHost": "192.168.200.33",
"Creation": "2019-11-29T14:01:05.610856762Z",
"Domain": "192.168.200.33",
"Expires": "2019-11-29T14:00:06Z",
"HostOnly": true,
"HttpOnly": false,
"LastAccess": "2019-11-29T14:01:05.610856762Z",
"Name": "macaroon-d5c5f4b6f406b6b0e20d6b01a13b1d1dabf449f145b0c5e23281fa2a4b802ca8",
"Path": "/auth",
"Persistent": true,
"Secure": false,
"Updated": "2019-11-29T14:01:05.610856762Z",
"Value": ""
}
]
The cookie apparently expired before it was minted. I expect this is probably due to the clock on the controller being about 5 minutes slow.
What keeps time on the controller allocated from MAAS?
Oh wow, ok. This explains a lot. I would expect most servers to run ntpd out of the box. There is an ntp charm you could try out maybe? For something baked into MAAS would have to hit them up. Might be worth checking their discourse.
Sorry I neglected to respond here, I wrote an explanation in the bug.
Time sync on MAAS is managed by crony. In my case there was no default route, so it wasn’t working.