The results are pretty interesting, as a 70% improvement is certainly something
to note, especially when your binaries are sometimes over 100 megabytes
I just don’t think that packing executables is a good thing to do for open
source projects that have a wide reach, and especially one that is used for
managing and provisioning critical infrastructure.
I like reverse engineering, and by extension, looking into how computer malware
works. Most malware that isn’t a toy these days are packed, and usually packed
with a custom build of upx, such that running “upx -d” with a vanilla build of
upx fails, to slow down malware researchers.
When I see a packed binary in the wild, I immediately think that someone is
trying to hide something, and the binary is likely malicious.
AV companies are on the same wavelength these days, and will flag a packed
binary as suspicious. As we expect juju to grow and to enter more enterprise
environments, more and more will start using enterprise AV inside their
juju controllers or user’s clients, or perhaps even within their deployed
AV software really don’t like the fact that a binary loads a stub loader,
decompresses the binary, and then executes it from memory. The program running
in memory is different from the binary on disk, which makes it difficult or
impossible to determine if the program running in memory has been tampered with
or overwritten, and would set off some AV alarms about application integrity.
Packing can also change the programs entrypoint, which then causes problems
down the line when you want to debug your binaries, as suddenly machine addresses
might not match debug symbols, which could be bad if a juju user hits a bug
that is critical to their production environment, and it would delay debugging
Decompressing the binary to run it also takes time, and at 100mb a pop, this
would be noticeable to users. If a provisioned machine is getting low on memory
and is battling the OOM daemon, restarting a killed juju daemon gets harder as
the unpacker needs additional space to read in the program, decompress it and
then jump to its entry point, which can sometimes cost packed binary size +
uncompressed binary size memory.
Since juju is really at the core of deployments, it is very important to think
about the security triad of Confidentiality, Integrity and Availability. I think
packing juju binaries could compromise integrity and availability of juju, and
would introduce more risks than the advantages of disk space being saved.
So, while packing can give you good results in reducing binary size, there are
good reasons why packing isn’t more widespread, and in the context of juju, I
think it is better to leave binaries as is, and live with large binary sizes.