Deploying FINOS Legend on Kubernetes

pedroleaoc · 26 September 2021 18:02

Charmed Legend

In this tutorial, you will learn how to use Juju and Charmed Operators to deploy an instance of the Legend stack on Kubernetes.

The applications stack will be deployed as a bundle in the same cloud. Juju allows, however, for you to deploy each application on a different cloud and then integrate the stack across your estate.

The preparation steps might take 20-30 minutes depending on your setup, but the actual Legend deployment takes about 5 minutes. We will start from a fresh system, provision a Kubernetes instance, install Juju, and deploy Legend.

If you have any questions, you can post a question on the Charmed Legend channel on Mattermost or use the finos-legend tag on Discourse.

This is a 100% open source initiative, you are welcome to suggest changes to the text of this tutorial or the code in the bundle. Everyone and anyone is welcome!

What you will need

Laptop
Internet connection
Kubernetes cluster - If you don’t have one, we will show you how to get one running in your laptop
GitLab instance - If you don’t have one, we will show you how to use gitlab.com or deploy a local instance of GitLab
Juju

What you will achieve

At the end of this tutorial you will have an instance of the Legend stack running on Kubernetes.

Kubernetes cluster

Juju will bootstrap to most Kubernetes clusters (regardless of the vendor) and you can use your own for the rest of this tutorial. If you don’t have access to a cluster, we will show you how to install MicroK8s—a lightweight distribution of Kubernetes that can run in your laptop.

You can learn how to use your own cluster here.

Kubernetes cluster - MicroK8s

I. Install MicroK8s

$ snap install microk8s --classic

That’s all. Now you have access to a full-fledged Kubernetes distribution in your computer.

II. Configure MicroK8s

The Legend stack requires resources from MicroK8s that can be enabled as addons. We will also create an alias and create a microk8s group.

A. Add an alias for `kubectl`

sudo snap alias microk8s.kubectl kubectl

B. Add your user to the microk8s group

$ sudo usermod -a -G microk8s $USER $ sudo chown -f -R $USER ~/.kube $ newgrp microk8s

C. Enable MicroK8s resources

$ sudo microk8s.enable dns storage ingress

III. Check MicroK8s status

The operations above might require a few seconds to take effect. You can check the MicroK8s’ status by running:

‘$ microk8s status --wait-ready’

Your MicroK8s is now ready.

GitLab

The Legend Stack uses GitLab for storing artifacts created on Studio. This tutorial provides you with three options to get your stack up and running:

Existing GitLab installation
gitlab.com
Deploy GitLab with docker-compose

We are currently working on a charm for GitLab that would be included in the finos-legend-bundle; with that, you would get a local GitLab instance already integrated with your Legend stack without the need to run extra configurations to pass tokens.

GitLab—Existing GitLab instance

You can skip this step completely. We will create a token and pass them to the Legend stack in the next steps.

GitLab— gitlab.com

Most enterprise use-cases will require a local private GitLab instance. For the purpose of this tutorial, the easiest might be to use gitlab.com, though the deployment and integration steps are exactly the same for any other kind of GitLab you might decide to use.

Prerequisites:

Access to the portal of gitlab.com
Application creation rights on said account

Creating the GitLab Application:

Log in to Gitlab
Go top-left to User Settings > Applications
Create a new application with the following:
- Name: “Legend Demo”
- Confidential: yes
- Scopes: openid, profile, api
- Redirect URI: set it to http://localhost:8080/callback
Save the Client ID and Secret for later

GitLab—Deploy GitLab with docker-compose

The docker compose file used for this is as follows:

version: '3.6'
services:
  gitlab:
    image: 'gitlab/gitlab-ee:latest'
    restart: always
    hostname: 'gitlab'
    domainname: 'gitlab.local'
    container_name: 'gitlab'
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'https://gitlab.local:443'
        gitlab_rails['initial_root_password'] = '${FINOS_GITLAB_PASSWORD:?no gitlab password set}'
        gitlab_rails['gitlab_shell_ssh_port'] = 2224
        gitlab_rails['lfs_enabled'] = true
        nginx['listen_port'] = 443
        nginx['ssl_certificate'] = '/etc/gitlab/ssl/gitlab.local.crt'
        nginx['ssl_certificate_key'] = '/etc/gitlab/ssl/gitlab.local.key'
        letsencrypt['enable'] = false
    ports:
      - '443:443'
      - '2224:2224'
    volumes:
      - '${GITLAB_HOME:?no gitlab home set}/config:/etc/gitlab'
      - '${GITLAB_HOME:?no gitlab home set}/logs:/var/log/gitlab'
      - '${GITLAB_HOME:?no gitlab home set}/data:/var/opt/gitlab'
      - './certs:/etc/gitlab/ssl'
    networks:
      legend:
        ipv4_address: 172.18.0.2
networks:
  legend:
    ipam:
      driver: default
      config:
        - subnet: 172.18.0.0/16

Prior to deploying Gitlab using this docker compose file two environment variables need to be set:

GITLAB_HOME : This is the folder location under which Gitlab will store all its data.
FINOS_GITLAB_PASSWORD : This is the root password for Gitlab.

Also note that this docker compose file sets a static IP address for Gitlab because the IP address is used in generating TLS certificates for Gitlab. This IP subnet and address may be changed to suit your own deployment. Generation of the TLS certificate can now be done using a OpenSSL configuration file (cert.cnf) as follows

[ req ]
default_bits                  = 2048
distinguished_name            = req_distinguished_name
req_extensions                = req_ext
[ req_distinguished_name ]
countryName                   = US
stateOrProvinceName           = NY
localityName                  = NY
organizationName              = XX
commonName                    = gitlab.local
[ req_ext ]
subjectAltName                = @alt_names
[alt_names]
DNS.1=gitlab.local
IP.1=172.18.0.2

Note that the OpenSSL config file uses the same IP as in the docker compose file to specify Subject Alternative Names. Using this config file TLS certificates may be generated using the following command line:

$ openssl req -newkey rsa:2048 -nodes -keyout "${HOST_KEY_FILE}" -x509 -days 365 -out "${HOST_CERT_FILE}" \
-config "cert.cnf" -extensions req_ext -subj "/C=US/ST=NY/L=NY/O=XX/CN=${HOST_DNS_NAME}"
$ openssl x509 -in "${HOST_CERT_FILE}" -outform der -out "${HOST_DER_FILE}"

This command line generates self signed certificates. Assuming the docker compose file (above) is in your current directory and the certificates are in a subdirectory certs in the same directory, a Gitlab instance may be launched using the standard command line such as docker-compose up -d.

Install Juju

In this tutorial, Juju will help with the deployment and integration of Legend. In a production environment, however, Juju models offer a powerful framework that allow human operators to better understand their infrastructure and applications.

Install Juju

$ snap install juju --classic

You should see the MicroK8s installation by running: $ juju clouds

If you are using your own cluster, you can add the credentials for your cloud as described here.

At this point, Juju knows that MicroK8s exists, but it is not yet present in the cluster. If you run $ juju status you will see an error. That is because, although Juju is aware of your cluster, it has not installed a controller to it yet.

Deploying Legend

Up until now, we’ve been preparing our system to host Legend. In the next steps, we will

Bootstrap Juju to your cluster
Add a model to host Legend
Deploy the bundle
Pass the GitLab token to the stack

1. Bootstrap to your cluster

In Juju terms, “bootstrap” means “install a controller”. A Juju controller is the part of Juju that runs in your cluster and controls the models.

To install the Juju controller in your cloud, you can runjuju bootstrap <cloud_name> <controller_name>. You can bootstrap to your own cluster (cloud_name can be found with the juju status command).

$ juju bootstrap microk8s finos-legend-controller

This command will take a few seconds to execute.

Running juju status will now show the Juju Controller in your cloud.

2. Create a model

Juju’s Day 2 operations are implemented via models. In the Kubernetes world, models are namespaces. You can learn more about Model Driven Operations here.

Models are add with the juju add-model <model_name> command.
To add a model where we will deploy the Legend stack, we can run: $ juju add-model finos-legend-model

In a production environment, you might want to implement different access permissions for different applications or deploy Legend in a hybrid cloud topology. You would create different models to accommodate these operations and Juju would support the cross-cloud integration.

You are now ready to start deploying applications!

3. Deploy Legend

When you deploy an application with Juju, the installation code in the charmed operator will run and set up all the resources and environmental variables needed for the application to run properly. In the case of this tutorial, we are deploying a bundle—which not only describes what applications we want deployed, but also the relations between them.

To deploy your bundle in the fresh model, run:

$ juju deploy finos-legend-bundle --channel=beta

As this bundle wasn’t released to stable—which generally means it’s ready for production—the extra flag beta is necessary.

In another terminal window, you can run the command below to see the applications being deployed and the integration code running.

$ watch --color juju status --color

Watching the status of this command is especially important since some code might run even after the deployment is finished (for example, integration routines).

After a couple of minutes, you’ll notice that your deployment looks like this:

[image]

This is perfectly normal, as the Legend stack is now waiting for you to pass the GitLab token as a configuration parameter.

The way you will do that depends on your GitLab instance. The next step will go through the commands you need to run for each of the options.

4. Passing the GitLab token

At this point, your Legend stack is deployed and integrated, but you still need to configure it with the GitLab token.

The process to do that will depend on your GitLab instance. We will show options for a local deployment or gitlab.com.

4. Passing the GitLab token—local deployment

If you are using your already existing GitLab instance or one deployed with docker compose, you can follow the steps below to inform the Legend stack about your access token.

Convert the certificate (’*.der’) into base64:

$ CERT=`base64 -w 0 /path/to/certfile.der

Configure the finos-legend-gitlab-integrator charm with your GitLab credentials. The default gitlab-host value for GitLab deployed via docker-compose is 172.18.0.2.

juju config finos-legend-gitlab-integrator-k8s \
     gitlab-host= <gitlab-host> gitlab-host-der-b64="$CERT" gitlab-port=443 \
     access-token="<access-token>"

You will be able to see the applications reacting to this new configuration by runnig the juju status command.

Once all the applications have become active, you can go to Authorizing the user and application.

4. Passing the GitLab token — gitlab.com

Fetching the redirect URIs

You can run the commands below to get the redirect URIs that will be later used by gitlab.com.

Get the name of the GitLab integrator charm:

$ juju status | grep gitlab
finos-legend-gitlab-integrator-k8s/0*  active    idle   10.1.184.238

Run an action against the charm to get the URIs

$ juju run-action finos-legend-gitlab-integrator-k8s/0 get-redirect-uris
Action queued with id: "2"

Show the action output:

$ juju show-action-output 2
UnitId: finos-legend-gitlab-integrator-k8s/0
id: "2"
results:
  result: |-
    http://10.1.184.224:6060/api/callback
    http://10.1.184.236:7070/api/auth/callback
    http://10.1.184.236:7070/api/pac4j/login/callback
    http://10.1.184.241:8080/studio/log.in/callback
status: completed
timing:
  completed: 2021-09-27 18:50:39 +0000 UTC
  enqueued: 2021-09-27 18:50:38 +0000 UTC
  started: 2021-09-27 18:50:38 +0000 UTC

You can now enter these URIs.

Setting the above redirect URIs into GitLab:

Log back into your GitLab portal.
Go to the application created previously.
Edit the Redirect URI setting of the application.
Paste the output of the result field from the juju show-action-output command run previously.

You will be able to see the applications reacting to this new configuration by running the juju status command.

Once all the applications have become active, you can go to Authorizing the user and application.

Authorizing the user and application

Run juju status to get the ip of the pod where studio is running. Enter that in your browser with the port <ip>:8080 and authorise the GitLab prompt.

Do the same for sdlc at <ip>:7070/api/auth/aautorize..

Accessing the studio dashboard

You should now be able to access the Legend Studio dashboard at:

<studio_ip>:8080/studio

Next steps

You can now add models and interact with your Legend Stack deployed on Kubernetes.

You can experiment with cross-model relations if you would like to deploy one of the applications in a different cloud.
Juju is not only about deployment, you can find a rich list of lifecycle management operations you can perform once your application is deployed.
Try on a different Kubernetes flavour.

pedroleaoc · 7 April 2022 08:35

pedroleaoc · 14 October 2022 11:31

Deploying FINOS Legend on Kubernetes

Charmed Legend

What you will need

What you will achieve

Kubernetes cluster

Kubernetes cluster - MicroK8s

I. Install MicroK8s

II. Configure MicroK8s

A. Add an alias for kubectl

B. Add your user to the microk8s group

C. Enable MicroK8s resources

III. Check MicroK8s status

GitLab

GitLab—Existing GitLab instance

GitLab— gitlab.com

Prerequisites:

Creating the GitLab Application:

GitLab—Deploy GitLab with docker-compose

Install Juju

Install Juju

Deploying Legend

1. Bootstrap to your cluster

2. Create a model

3. Deploy Legend

4. Passing the GitLab token

4. Passing the GitLab token—local deployment

4. Passing the GitLab token — gitlab.com

Fetching the redirect URIs

Setting the above redirect URIs into GitLab:

Authorizing the user and application

Accessing the studio dashboard

Next steps

A. Add an alias for `kubectl`