As we’re working through charmcraft commands and features, one of the issues we’ve been talking about is what files should charmcraft build copy into the charm it creates.
One of the things charm developers have told us about the operator framework is that they enjoyed being able to do just juju deploy . and have everything from their development tree in the charm. This however does come at a cost of complexity on creating and laying out the charm (including needing to get all the runtime dependencies yourself), as well as limitations around what platforms it’s possible to deploy to (namely, only those that exactly match your dev box). charmcraft is tackling both these via its build command, but that takes us away from exactly the feature people have told us is nice, to repeat, that the charm deployed is exactly what’s locally.
Now, accomplishing exactly that is easy, we just need to make sure the built charm has everything in it. That’s certainly possible. We can also implement a ‘copy everything that juju deploy does’, which is not exactly everything but comes close (it skips .git, say).
However, doing that brings about the risk that we’d be copying secrets into the charm. Keep in mind the charm would then be put in the store, potentially automatically (e.g. by CI), so it could be rather more risky than the development-oriented juju deploy . command.
On the other hand, not including everything means there will be a situation where the charm does not work and the developer needs to iterate the charm to include things that it didn’t.
So both alternatives are surprising, in different ways. Currently we’re leaning towards copying a limited set of things, with support for specifying more (via a MANIFEST.in like any python project), because we feel that the surprise of copying secrets is a lot worse than the surprise of missing something.
Does that sound about right? Are there patterns you all are using that would like to be supported ‘by default’? Are we giving the secrets argument too much weight?
Alternative 1: If there’s a .git/ directory present, charmcraft build can potentially run a git export command which would export everything that’s needed by the charm to some location minus the things listed in .gitignore (which should include secrets). You can then copy the contents of the produced archive to the charm. No need to maintain yet another file which could be easily missed by the developer and go out of sync quickly.
Perhaps charmcraft would be able to produce a “included” filelist and honor an “excluded” filelist. The included filelist could be generated by charmcraft whereas the excluded would be editable to allow exclusion of any file.
That would also create a very clear way for a charmer to understand what charmcraft actually does instead of making it magic.
I’m concerned that it just duplicates information already somewhere else. Setting that aside, the name implies Juju will recognize that file when it’s really just charmcraft build that’s looking at it.
We are working in git branches, and we already need to solve the secrets issue with git. I’d suggest ‘charm build’ warns if ‘git status’ lists untracked files, requiring explicit confirmation or a --force option to proceed. ‘charm build’ also needs to ignore files listed in .gitignore. In this way, developers only have to solve the secrets problem once. Relying on .jujuignore or .gitignore won’t work, as they are blacklists and not particularly useful for stopping secret leaks, which really need a whitelist. Blacklists require all developers to follow standards on where they keep their junk and to not make any mistakes, which is why git doesn’t automatically commit untracked files without explicit permission, and why we shouldn’t include unknown files is the ‘charm build’ without explicit permission.
I see no problem with a git dependency. The same logic could be implemented for other VCS systems like Bazaar, but it honestly doesn’t seem worth the effort (the bulk of charms using Bazaar are only that way because nobody has made the effort to migrate them to git). It would be trivial to skip the ‘git status’ check if $CHARM_DIR/.git doesn’t exist, but I can argue that ‘charm build’ would also need explicit approval to proceed in that situation (but it won’t make any practical difference, so whatever).
‘charm build’ should probably additionally ignore files listed in .jujuignore, although it is mostly redundant with .gitignore. It does mean you can have files checked into your charm source that never end up in the charm store. Maybe developer documentation, makefiles, custom build steps etc. would go in here. In any case, as mentioned above, you can’t rely solely on .jujuignore for keeping secrets because it is a blacklist.
I think that sticking to .jujuignore is clean and very precise. It’s also immediately visible in a repo. Even better would be if charmcraft init plopped a .jujuignore with some opinionated defaults in it at the top level of the project directly (things like ignoring .git, .bzr, and common folders like tests) that would make the pattern very visible to developers writing their first charm. I personally feel that layer-basic had a few ways to handle this problem and no clear “default”, so it could be trickier than necessary to work out what was being included at which stage of the build and deploy lifecycle for a reactive charm.
I feel that the pattern of specifying which files to include would be more surprising generally, especially to folks new to writing charms or even python code, but it definitely also has benefits - namely being deeply explicit, and if we were to provide a default manifest on charm creation it could be a great way to set up a very opinionated project skeleton and inform the workflow of charm development and repository layout directly - something new developers and charm authors could struggle with.
Dependency management is also worth tackling - in charm build and layer-basic the dependency management suffered from a similar issue of not being explicit - there were many, many ways to land a dependency in the final-state deployed charm, either during build time or deploy time, leading to different sets of challenges based on approach and developer expectation. To answer the question of " what files should charmcraft build copy into the charm it creates" - I think it would be pretty neat if charmcraft supported popular Python dependency tools in charmcraft build, and the result would be that the dependencies were either added as submodules or vendored (e.g. .venv or a wheelhouse) so that dependencies could be present in a format understood by/acceptable for Juju deployment. So you can then do charmcraft build and have a CWD that can be juju deploy .'d straight on into Juju.
Having worked with deploy-time dependencies in charms, I think vendoring is the correct balance between usability and deployability with the repo-cruft trade-off being noted. The downside of course is you can end up with a Docker-esque situtation where you have charms that have old outdated libraries vendored into the built charm - but if the means for specifying the dependencies at build-time is dynamic enough (similar to using loosely constrained dependencies in a requirements.txt or Pipfile) then a charm build and push would bring things right up to date - making the process as frictionless as possible I think is how you avoid that issue. Juju installing dependencies at deploy time is another answer to this, but as we all know that’s a process fraught with peril, especially in locked down/isolated environments, as was/is possible with layer-basic, so it would be good to not rely on that if possible.
WRT dependency management, charmcraftalready pulls dependencies from a requirements.txt and bundles them into a venv-like library directory (not actually a venv because we use the system python). On the “be opinionated” track, we’re unlikely to support anything other than a requirements.txt for this
WRT charmcraft build && juju deploy ., we’ve found that too hard to do, but you can do charmcraft build && juju deploy *.charm, and maybe soon charmcraft deploy as a shorthand for doing those two (but there are still medium-hard questions to be answered wrt packaging of that in a strictly-confined snap that charmcraft is and aims to continue to be).
That actually works since it’s ubiquitous enough that most folks are familiar with it but without constraining others who might want to use a tool that locks down dependencies in the requirements.txt file. E.g. pip-tool’s pip-compile or poetry’s export feature.
