I have some models that have had their disks wiped out and are no longer functional. How do I go about removing them from the controller? I have tried “juju remove-machine # --force [–no-wait]”. One of the machines has been fully deleted from the cloud substrate. How do I tell the controller to just “let it go”?
3 Likes
You’re on the right track. The juju remove-machine --force should work. There have been a number of issues fixed in --force removal over the last couple of versions of Juju so it might be worth checking what version you’re on if it’s not working for you.
The controller is on 2.7.0 but the rest of the models are all on 2.6.10. The remove machine happily says that it will do it, but doesn’t accomplish the task.
By using command "juju destroy-model " .The model will be deleted.
destroy-model gets stuck the same way. I managed to remove the machines, but destroy model is still
stuck waiting for some volumes to be deleted:
juju destroy-model --force --no-wait bionic-stein-base-2
WARNING! This command will destroy the "bionic-stein-base-2" model.
This includes all machines, applications, data and other resources.
Continue [y/N]? y
Destroying model
Waiting for model to be removed, 3 volume(s)....................................
........................................
Any idea on how to wipe the model?
3 Likes
This is common for me aswell to end up with stuck models. My only way has been to delete everything in the model manually. Machines etc. Not sure exactly how.
I wrote a post about this the other day actually and it seems you experience the same situation.
@timClicks I think this is worth noticing since people end up with stuck models and the juju messages will not help us to resolve it. I think the - - force flag needs to get more muscles or assist resolving stuck models.
I agree, but it’s a difficult problem. When Juju hangs like this, it is waiting for confirmation from the cloud that the resource(s) have been removed.
@sombrafam can you please try this command?
juju destroy-model --destroy-storage bionic-stein-base-2
It looks you have found a bug with some of the internal logic. By design, we don’t destroy storage unless it’s explicitly asked for.
We should be alerting you of this rather than waiting forever. You may ask why --force does not do this by default… and that’s because --force is designed to prevent transient errors that are raised by the cloud from preventing the operation.
Juju has a policy of not deleting things unless you are explicit. That policy has existed since before --force was put in place. Now that --force exists, perhaps we should assume that means --destroy-storage?
2 Likes
I had a password change in my vSphere “cloud” which caused all kinds of havoc with my controllers, models, and vms. I finally got one of the controllers to accept the new password and it then worked to delete the vms and the model as requested.
I have other controllers that will not complete the credential change because one or more of the vms now claim to be unavailable even though I can still 'juju ssh …" to them.
These I am still struggling with.
1 Like
This has been a long lasting issue for me aswell and led to many troublesome hours of debugging and chasing problems.
I would suggest the process of nuking models could at least be more descriptive of what ‘phases’ in the destroy-model is at.
Today, it’s just a single spell that tells very little about what is actually going on behind the scenes and it’s difficult to debug what’s working and what’s not.
It would also generate better bug reports and let the community also help more in the process.
May be we should be an option to cleanup any juju data related to a given model.
As its very common to have situations where you totally loose you cloud or parts of it?
juju destroy-model --cleanup
About ‘–force’ what does it force through?
1 Like
That sounds the proper approach to me. From the user POV, calling force
already means he is giving up everything in the model.
Disregard my last question about ‘–force’ , I didn’t have read all your comment.
The first model I was trying to delete was deleted. There are some others that
still cant be removed.
After 15 years of work on OpenStack they still cannot ship something that installs without causing extreme pain and systems stuck in an infinite loop. I wish I had stayed with Microsoft Windows 95
1 Like
Welcome to the community.
Please reach out if there are specific issues we can help you with.
Hi All,
I did try with both --force and without --Force option for destroying the models but it is still stuck in “destroying status” as shown below
$ juju models
Controller: lxd-mgmt
Model Cloud/Region Type Status Machines Units Access Last connection
controller management/default lxd available 1 1 admin just now
kubeflow* microk8s-train/localhost kubernetes destroying 0 - admin 23 seconds ago
Even the machine ended up in reboot and all the respective kubernetes pods are cleaned up but juju is still stuck in destroying phase. I dont see any new logs under “juju debug-log --replay”. How can I properly cleanup this dead model
Hi folks,
I encountered this issue again and discovered that the solution involves removing the model and its references in the database. To accomplish this, log into the Mongo database and execute the script below with your model ID (which can be found in .local/share/juju/models.yaml):
var db = db.getSiblingDB('juju');
var cleanup_model_uuids = [
"your-model-id-here"
];
db.getCollectionNames().forEach(function(coll) {
var count = db.getCollection(coll).count({"model-uuid": {$in: cleanup_model_uuids}});
if (count > 0) {
var deleted = db.getCollection(coll).remove({"model-uuid": {$in: cleanup_model_uuids}});
print("Deleted " + deleted + " objects from " + coll);
}
});
Once that’s done, you’ll need to manually remove all resources in the provider (VMs, containers, etc.) and also delete the associated entries in .local/share/juju/models.yaml.
3 Likes
I had a similar issue with orphans Models Here is what I adapted from the code above
var db = db.getSiblingDB('juju');
var cleanup_model_uuids = [
"2d6e042f-e18e-45a6-85c7-59e84970a6ae",
"7bf6e622-1a30-4b0f-8682-984e50142f64",
"1dbcd9f4-921d-4871-8165-5160ea73c430",
"3b2af8d5-c40e-49d6-8518-9288ef129030",
"a5838049-f66e-45c2-804b-c76347ea7935",
];
db.getCollectionNames().forEach(function(coll) {
var count = db.getCollection(coll).count({"model-uuid": {$in: cleanup_model_uuids}});
if (count > 0) {
var deleted = db.getCollection(coll).remove({"model-uuid": {$in: cleanup_model_uuids}});
print("Deleted " + deleted + " objects from " + coll);
}
});
db.models.remove({"_id": {$in: cleanup_model_uuids}});