MongoDB How To - Compile Deb package

taurus · 17 July 2023 11:37

How to compile PSMDB (Percona Server for MongoDB) Ubuntu Jammy package

Note: it is a “JFYI only manual” based on official build instruction.

Launch isolated/clean VM

multipass launch 22.04 -c 8 -m 32G -d 100G # Disk 60G min!

Launched: elevating-ewe

Login into newly created VM:

multipass shell elevating-ewe

Create a folder to build source into:

sudo mkdir -p /mnt/psmdb-50/test && sudo chmod -R 0777 /mnt/psmdb-50

Install Docker

All operations are performed in a clean Docker container:

sudo apt-get install ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install --yes docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker $USER
# re-login to multipass VM!

Fetch build script

Here and below, make sure you are using the proper tag/folder, etc!

cd /mnt/psmdb-50/
wget https://raw.githubusercontent.com/percona/percona-server-mongodb/psmdb-5.0.18-15/percona-packaging/scripts/psmdb_builder.sh -O psmdb_builder.sh

Fix the issues in a build script:

sed -i 's/percona-release_1.0-27.generic_all.deb/percona-release_1.0-28.generic_all.deb/g' psmdb_builder.sh # BUG1 see https://repo.percona.com/apt/pool/testing/p/percona-release/

BUG 2: skip dh-systemd for xjammy in psmdb_builder.sh:

ubuntu@elevating-ewe:/mnt/psmdb-50$ diff -u psmdb_builder.sh.ORIG psmdb_builder.sh.HACKED
--- psmdb_builder.sh.ORIG	2023-07-14 13:42:04.216438052 +0200
+++ psmdb_builder.sh.HACKED	2023-07-14 13:46:35.965056138 +0200
@@ -433,7 +433,7 @@
       DEBIAN_FRONTEND=noninteractive apt-get -y install curl lsb-release wget apt-transport-https software-properties-common
       export DEBIAN=$(lsb_release -sc)
       export ARCH=$(echo $(uname -m) | sed -e 's:i686:i386:g')
-      wget https://repo.percona.com/apt/pool/testing/p/percona-release/percona-release_1.0-27.generic_all.deb && dpkg -i percona-release_1.0-27.generic_all.deb
+      wget https://repo.percona.com/apt/pool/testing/p/percona-release/percona-release_1.0-28.generic_all.deb && dpkg -i percona-release_1.0-28.generic_all.deb
       if [ x"${DEBIAN}" = "xbionic" -o x"${DEBIAN}" = "xfocal" ]; then
         add-apt-repository -y ppa:deadsnakes/ppa
       elif [ x"${DEBIAN}" = "xstretch" -o x"${DEBIAN}" = "xbuster" ]; then
@@ -718,7 +718,7 @@
     sed -i 's:@@LOGDIR@@:mongodb:g' ${BUILDDIR}/debian/mongod.default
     sed -i 's:@@LOGDIR@@:mongodb:g' ${BUILDDIR}/debian/percona-server-mongodb-helper.sh
     #
-    if [ x"${DEBIAN}" = "xbullseye" -o x"${DEBIAN}" = "xxenial" ]; then
+    if [ x"${DEBIAN}" = "xbullseye" -o x"${DEBIAN}" = "xxenial" -o x"${DEBIAN}" = "xjammy" ]; then
         sed -i 's:dh-systemd,::' ${BUILDDIR}/debian/control
     fi
     #
@@ -786,7 +786,7 @@
     cp -av percona-packaging/debian/rules debian/
     set_compiler
     fix_rules
-    if [ x"${DEBIAN}" = "xbullseye" -o x"${DEBIAN}" = "xxenial" ]; then
+    if [ x"${DEBIAN}" = "xbullseye" -o x"${DEBIAN}" = "xxenial" -o x"${DEBIAN}" = "xjammy" ]; then
         sed -i 's:dh-systemd,::' debian/control
         sed -i 's:etc/:/etc/:g' debian/percona-server-mongodb-server.conffiles
     fi

Choose “proper” tags

To build MongoDB from sources you have to provide the following parameters:

branch (git tag),
psm_ver (MongoDB version, the first part of git tag above)
psm_release (Percona release version, the last part from git tag above)
mongo_tools_tag (the content of file MONGO_TOOLS_TAG_VERSION, CHOOSE PROPER GIT TAG!!!). Sources.
jemalloc_tag (is already hard-coded inside psmdb_builder.sh, CHOOSE PROPER GIT TAG!!!). Sources.

The current examples for MongoDB 5 and 6:

# For MongoDB 5
# --branch=release-5.0.18-15 --psm_ver=5.0.18 --psm_release=15 --mongo_tools_tag=100.7.0 --jemalloc_tag=psmdb-3.2.11-3.1
# For MongoDB 6
# --branch=release-6.0.6-5 --psm_ver=6.0.6 --psm_release=5 --mongo_tools_tag=100.7.0 --jemalloc_tag=psmdb-3.2.11-3.1

Double check the versions you are going to use, to avoid 8+ hours rebuilding procedure!

Create MongoDB Source Tarball

docker run -ti -u root -v /mnt/psmdb-50:/mnt/psmdb-50 centos:7 sh -c '
set -o xtrace
cd /mnt/psmdb-50
bash -x ./psmdb_builder.sh --builddir=/mnt/psmdb-50/test --install_deps=1
bash -x ./psmdb_builder.sh --builddir=/mnt/psmdb-50/test --repo=https://github.com/percona/percona-server-mongodb.git \
--branch=release-5.0.18-15 --psm_ver=5.0.18 --psm_release=15 --mongo_tools_tag=100.7.0 --jemalloc_tag=psmdb-3.2.11-3.1 --get_sources=1
'

Check source tarball has been created:

ls -la /mnt/psmdb-50/source_tarball/
> -rw-r--r--. 1 root root 90398894 Oct  1 10:58 percona-server-mongodb-5.0.18-15.tar.gz

Set proper PPA version and distribution:

Fix the package version and distribution to set proper PPA version:

ubuntu@sharing-gnatcatcher:/mnt/psmdb-60$ diff -u psmdb_builder.sh psmdb_builder.sh.HACKED 
--- psmdb_builder.sh	2023-07-17 12:22:59.420837322 +0200
+++ psmdb_builder.sh.HACKED	2023-07-17 11:30:08.319081502 +0200
@@ -708,7 +708,7 @@
     set_compiler
     fix_rules
 
-    dch -D unstable --force-distribution -v "${VERSION}-${RELEASE}" "Update to new Percona Server for MongoDB version ${VERSION}"
+    dch -D jammy --force-distribution -v "${VERSION}-${RELEASE}-0ubuntu0.22.04.1~ppa1" "Update to new Percona Server for MongoDB version ${VERSION}"
     dpkg-buildpackage -S
     cd ../
     mkdir -p $WORKDIR/source_deb
@@ -768,7 +768,7 @@
     sed -i 's|go build|go build -a -x|' mongo-tools/build.sh
     sed -i 's|exit $ec||' mongo-tools/build.sh
     . ./mongo-tools/set_tools_revision.sh
-    dch -m -D "${DEBIAN}" --force-distribution -v "${VERSION}-${RELEASE}.${DEBIAN}" 'Update distribution'
+    dch -m -D "${DEBIAN}" --force-distribution -v "${VERSION}-${RELEASE}-0ubuntu0.22.04.1~ppa1.${DEBIAN}" 'Update distribution'
     export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
     if [ x"${DEBIAN}" = "xstretch" ]; then
       CURL_LINKFLAGS=$(pkg-config libcurl --static --libs)
ubuntu@sharing-gnatcatcher:/mnt/psmdb-60$

Build Ubuntu Jammy source package

docker run -ti -u root -v /mnt/psmdb-50:/mnt/psmdb-50 ubuntu:jammy sh -c '
set -o xtrace
export EMAIL=data-platform@lists.launchpad.net
export DEBEMAIL=data-platform@lists.launchpad.net
export DEBFULLNAME="Canonical Data Platform"
apt update && apt install --yes lsb-release
cd /mnt/psmdb-50
bash -x ./psmdb_builder.sh --builddir=/mnt/psmdb-50/test --install_deps=1
bash -x ./psmdb_builder.sh --builddir=/mnt/psmdb-50/test --repo=https://github.com/percona/percona-server-mongodb.git \
--branch=release-5.0.18-15 --psm_ver=5.0.18 --psm_release=15 --mongo_tools_tag=100.7.0 --jemalloc_tag=psmdb-3.2.11-3.1 --build_src_deb=1
'

Check source package has been created:

ls -la /mnt/psmdb-50/source_deb/
> -rw-r--r--. 1 root root    10724 Oct  1 11:45 percona-server-mongodb_5.0.18-15.debian.tar.xz
> -rw-r--r--. 1 root root     1528 Oct  1 11:45 percona-server-mongodb_5.0.18-15.dsc
> -rw-r--r--. 1 root root     2075 Oct  1 11:45 percona-server-mongodb_5.0.18-15_source.changes
> -rw-r--r--. 1 root root 90398894 Oct  1 11:45 percona-server-mongodb_5.0.18.orig.tar.gz

Build Ubuntu Jammy DEB package

docker run -ti -u root -v /mnt/psmdb-50:/mnt/psmdb-50 ubuntu:jammy sh -c '
set -o xtrace
cd /mnt/psmdb-50
export EMAIL=data-platform@lists.launchpad.net
export DEBEMAIL=data-platform@lists.launchpad.net
export DEBFULLNAME="Canonical Data Platform"
apt update && apt install --yes lsb-release
bash -x ./psmdb_builder.sh --builddir=/mnt/psmdb-50/test --install_deps=1
bash -x ./psmdb_builder.sh --builddir=/mnt/psmdb-50/test --repo=https://github.com/percona/percona-server-mongodb.git \
--branch=release-5.0.18-15 --psm_ver=5.0.18 --psm_release=15 --mongo_tools_tag=100.7.0 --jemalloc_tag=psmdb-3.2.11-3.1 --build_deb=1
'

Check Ubuntu Jammy DEB has been created:

ls -la /mnt/psmdb-50/deb/
> -rw-r--r--. 1 root root 2301998432 Oct  1 13:16 percona-server-mongodb-dbg_5.0.18-15.bullseye_amd64.deb
> -rw-r--r--. 1 root root   14872728 Oct  1 13:16 percona-server-mongodb-mongos_5.0.18-15.bullseye_amd64.deb
> -rw-r--r--. 1 root root   35356944 Oct  1 13:16 percona-server-mongodb-server_5.0.18-15.bullseye_amd64.deb
> -rw-r--r--. 1 root root   12274928 Oct  1 13:16 percona-server-mongodb-shell_5.0.18-15.bullseye_amd64.deb
> -rw-r--r--. 1 root root   26784020 Oct  1 13:16 percona-server-mongodb-tools_5.0.18-15.bullseye_amd64.deb
> -rw-r--r--. 1 root root      18548 Oct  4 13:16 percona-server-mongodb_5.0.18-15.bullseye_amd64.deb

Sign sources

JFYI, historically Percona uses test folder for build artifacts:

sudo apt install devscripts dput
debsign -k <your_truted_gpg_key> test/percona-server-mongodb_5.0.18-15_source.changes

Configure dput

cat ~/.dput.cf
[mongodb-5]
fqdn = ppa.launchpad.net
method = ftp
incoming = ~<your_launchpad_nick>/+archive/ubuntu/<your_ppa>
login = <your_launchpad_nick>
allow_unsigned_uploads = 0

Push to PPA

dput mongodb-5 test/percona-server-mongodb_5.0.18-15_source.changes

Validate results

Check the content and install your Debian packages as PPA:

https://launchpad.net/~<your_nick>/+archive/ubuntu/<your_ppa>

Possible issues:

Issue 1: Error “gpg: … Error checking signature from …”

> dput test-psmdb ppa/percona-server-mongodb_5.0.18-15_source.changes 
D: Setting host argument.
Checking signature on .changes
gpg: ppa/percona-server-mongodb_5.0.18-15_source.changes: Error checking signature from AAABBBCCCDDDEEE: SignatureVerifyError: 0
Checking signature on .dsc
gpg: ppa/percona-server-mongodb_5.0.18-15.dsc: Error checking signature from AAABBBCCCDDDEEE: SignatureVerifyError: 0
...
Successfully uploaded packages.

To fix an GPG error “Error checking signature from” add default-key and trusted-key key to gpg.conf (see bugreport):

> cat ~/.gnupg/gpg.conf
use-agent
default-key <your_gpg_fingerprint>
trusted-key <your_gpg_fingerprint>