Speeding up the packing of charms

This post offers some hints to speed up the charmcraft pack operation. Work is being done to understand internal times and improve them, refer to this post about performance analysis for more information.

Avoid using Python dependencies from GitHub

The virtual environment creation and Python dependencies compilation and installation is an expensive process and takes a substantial proportion of the whole packing time.

This process is of course fully executed the first time a charm is packed (or after a charmcraft clean), but if the dependencies didn’t change in the subsequent runs it’s quite fast, as the virtual environment is already created and pip detects that everything is up to date.

That said, if a dependency is specified from Github (e.g. git+https://github.com/canonical/operator/#egg=ops or git+https://github.com/canonical/craft-cli.git@4af19f9c0da733321dc754be1180aea28f3feeb1), pip fails to detect if there is an update for the dependency and will always download it.

This means that a charmcraft pack after any simple file change (like improving charm.py) will involve some pip downloading and processing that could be avoided.

Beware of any local file change

When packing a charm all project files are taken in consideration for the building process.

Not only the files that end up in the final .charm file, but all of them, as any file can potentially be using by the building infrastructure. This means that if any file changes in the project, the building step is re-executed.

When working from local sources (which is the default), if you note that the building is triggered in a re-packing situation when you thought it shouldn’t, beware that you may have automatic processes run that end up in files modifications (e.g. a “magic prompt” that shows the project’s Git branch, which dirties the .git directory).

5 Likes