Remote debugging a charm with VS Code

This is a quick guide on how to connect a remote debugger to a charm, I will be using VS Code. A similar approach could be used to get something working on PyCharm (sorta, see section at the bottom). I had to adapt this a little since I run Juju in a VM, so this is a slightly simplified version of what I’ve been doing. Hopefully I didn’t miss anything when I “simplified” it, but if I did please let me know.

This shouldn’t take more than a few minutes to get running (minus the time to pack the charm). It’s pretty straight-forward.

Add the hooks and re-pack the charm

  1. Add debugpy to your requirements.txt

  2. Create a file in your charm’s src/ directory, for example src/remote_debug.py, and copy the following contents to it. This will keep most of the changes in a separate file and make it easier to include as you need it. I add this file to my local ignore (.git/info/exclude) and then forget it exists until I need it.

    import logging
    from typing import Optional
    
    import debugpy  # type: ignore
    import ops
    
    logger = logging.getLogger(__name__)
    
    
    class RemoteDebuggerCharmBase(ops.CharmBase):
        def __init__(self, *args, **kwargs):
            _patch_breakpoint_hook()
            super().__init__(*args, **kwargs)
    
    
    def _wait_for_debugger(name: Optional[str] = None):
        if not debugpy.is_client_connected():
            debugpy.listen(("0.0.0.0", 5678))
            logger.info("Waiting for debugger to attach")
    
            debugpy.wait_for_client()
    
    
    def _runcall_patch(*args, **kwds):
        _wait_for_debugger()
        # TODO: It would be better to call set_trace() here, but I couldn't get it
        # to work with the debugger
        return args[0](*args[1:], **kwds)
    
    
    def _patch_breakpoint_hook():
        logger.info("Installing breakpoint() hook")
        # Installs a hook, so that we block when we hit a breakpoint that we told
        # juju to watch using `juju debug-code --at=hook <unit> <event-hook>`
        ops.framework.pdb.runcall = _runcall_patch
    
  3. Import RemoteDebuggerCharmBase and inherit from it in the charm you would like to debug. For example, if I’m debugging the vault-k8s charm, I would replace this line in charm.py:

    class VaultCharm(CharmBase):
    

    With

    class VaultCharm(RemoteDebuggerCharmBase):
    
  4. Repack the charm

    charmcraft pack
    

Deploy the modified charm

juju deploy ./vault-k8s_ubuntu-22.04-amd64.charm vault-k8s --trust -n 5 --resource=vault-image=ghcr.io/canonical/vault:1.15.3

Add the debugging config to vscode

Get the IP address of the unit you would like to debug

unit_name="vault-k8s/1"

juju show-unit "${unit_name}" | yq ".*.address"

Add the the following to your launch.json file. Make sure to replace the host key with the IP address you got above.

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Remote Attach",
            "type": "python",
            "request": "attach",
            "connect": {
                "host": "<UNIT IP ADDRESS>",
                "port": 5678
            },
            "pathMappings": [
                {
                    "localRoot": "${workspaceFolder}",
                    "remoteRoot": "."
                }
            ],
            "justMyCode": true
        }
    ]
}

Tell juju to debug the hook, and wait for it to fire

juju debug-code --at=hook "${unit_name}" update-status

This will cause juju to launch a tmux session and wait for the hook to be run. You may be familiar with the debug-code command already. But with the RemoteDebuggerCharmBase, instead of launching pdb when the hook is run, you should instead see a message that says “Waiting for debugger to attach”.

Since we’re debugging update-status, we can make this happen a little faster by speeding up the update interval

juju model-config update-status-hook-interval=5s

Connect VS Code

At this point, you can click the “play” button in VS Code, and it should connect to the remote debugger. However, once it connects it will immediately resume running. So, first set a breakpoint in VS Code. Otherwise, it should stop if an exception is thrown.

If everything worked out, you can now step through the code. You may need to add some pathMappings to the launch.json file to jump into library code correctly but anything in src/ should be working.

Consider using something like jhack to sync your code when you make changes.

Re: PyCharm

As I understand it, PyCharm doesn’t support the “Debug Adapter Protocol” that debugpy implements, so you would likely need to use pydevd. pydevd in turn doesn’t support connections from IDE → debugger, so you’d need to reverse the direction of the connection and have the debugger contact the IDE.

6 Likes

Lovely @danielarndt! I’m thinking about how to streamline the UX of this and especially avoid having to repack a charm or touch its dependencies in order to use this functionality.

My initial thoughts are on https://github.com/canonical/jhack/pull/126 , if you’d like to take a look/contribute, more than welcome!

I didn’t yet get it to work, the process gets stuck and I’m not sure why

2 Likes

Awesome. I was hoping that by sharing my hack it would be made better by others!

On the weekend, I meant to take a look at what you have in that PR but I was fighting some environment issues. Maybe today I’ll have a little more luck.

1 Like

@danielarndt - this is super interesting, thanks! There might be some crossover with what @babakks is doing with the charmcraft IDE extension?

@tmihoc do you think we should include this in the SDK docs somewhere? :slight_smile:

1 Like

@jnsgruk Definitely! I’m away this week but will follow up with @danielarndt next week.

1 Like

@danielarndt This is awesome! Thanks for sharing.

@jnsgruk Thanks for the hint. It’d be great if we could add such functionality to the extension. I’m thinking about automating some steps of the procedure described here (if not all) by the extension.

@danielarndt If you don’t mind, I’ll figure out a feasible implementation path (from the extension’s perspective) and then reach out to pick your brain.

1 Like

@babakks Sorry for the slow reply on this, but yes, I’d love to help productionize this. I realize it’s quite the hack right now. Reach out whenever you want to chat.

1 Like