Setting Mellanox RX and TX ring sizes with Juju

juju
,
1

I posted a similar discussion in MAAS discourse at Setting Mellanox RX and TX ring sizes with MAAS - Features - MAAS | Discourse

I’m using MAAS and Juju to deploy OpenStack on top of servers that have Mellanox MT27710 ConnectX-4 Lx NICs. After some performance tests, I found that raising the RX and TX ring sizes of the Mellanox NICs offered better performance and less packet drops. I set the RX and TX sizes with:

sudo ethtool -G rx 8192 tx 8192

I was trying to find the best way to configure this so that it survives reboots and redeploys. I ended up adding a script to cloudinit-userdata in the Juju model defaults that adds a systemd service on each deployed node that runs those commands at each boot.

I’m thinking however that these settings should be configurable elsewhere in an easier manner.

I was wondering at first if MAAS would be the right place, just like we can configure MTU settings in MAAS. But I’d like to know what the Juju team thinks too about if Juju (or a Juju charm maybe?) could support this.

Do you think it makes sense to file a feature request for Juju to support setting Mellanox ring sizes?

Thank you.

1 Like
2

Agree, this sounds like specific config of the network which would be done in MAAS in the interface config.

You can work around it, as you have, with custom model level cloud-init config or even going so far as to provide a subordinate charm that exposed stuff like that out but it seems that what you’re really looking to do is tweak the underlying hardware characteristics/config vs application level modeling with Juju.

3

We are using Mellanox Infiniband and need to add ppa:s and yum repos for centos to get OFED drivers in place. We do that using “cloudinit-userdata” and what you can do, is to add preruncmd/postruncmd entries in there aswell.

This is an example we need to be able to deploy python charms on centos7, behind a corporate proxy.

Its nasty, I know, but it works.

cloudinit-userdata: |
  yum_repos:
    epel-release:
      baseurl: http://download.fedoraproject.org/pub/epel/7/$basearch
      enabled: true
      failovermethod: priority
      gpgcheck: true
      gpgkey: http://download.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7
      name: Extra Packages for Enterprise Linux 7 - Release
  preruncmd:
    - yum --setopt=proxy=http://myproxy.com:8080 -y install epel-release redhat-lsb-core sudo openssh-server python36-virtualenv python36-PyYAML python3 snapd
    - echo "proxy=http://myproxy.com:8080" >> /etc/yum.conf
  postruncmd:
   - usermod -p '*' ubuntu
   - 'systemctl start sshd'
   - 'systemctl enable sshd'
   - 'systemctl start snapd'
   - 'systemctl enable snapd'

(You also need to set snap proxy settings which is not in this example…)