Portable GitLab — GitLab In An LXD Container

Photo by Markus Spiske on Unsplash

GitLab is an entire DevOps lifecycle tool with a web-based interface. Within the GitLab platform we are provided with a git-repository manager, wiki, issue-tracking, integration, and deployment pipelines and best of all it is all open source software! It also makes for a great, self-hosted replacement for GitHub.

There are plenty of organizations that run GitLab on bare metal, in a dedicated Virtual Machine (VM) in VMWare, Hyper-V, or Linux KVM, as well as on cloud host instances from providers such as AWS, Azure, and GCP. The cloud platforms have some built-in ability to be scalable and some form of high availability for migrating to a new host if there is a failure at a lower level. For those that host locally on bare metal or in a VM we have to deal with high availability and scalability ourselves.

There are official GitLab Docker images that can be used if Docker is your container of choice. However, I prefer to use Linux containers (LXD) as it provides more of standalone container that can more closely mimic a dedicated machine. Spinning up new LXD containers is fast, easy, and allows me to use them as lightweight VMs.

In order to get started you will need to have LXD installed on your host machine. On Ubuntu this is as simple as:

snap install lxd

Followed by creating an initial configuration with:

lxd init

The defaults should be fine, however, I prefer to use “dir” as the storage backend as it allows me to easily move the container around the filesystem with symlinks.

If you want to make sure LXD is installed and configured you run this quick command:

lxc info

You will be presented with a bunch of output about the host system and the LXD and the available features. The rest of this guide assumes you have a working LXD install.

Now that we have LXD installed and configured, let’s go ahead and create a new container for dedicated GitLab use on our host system:

lxc launch ubuntu: gitlab

The above command will create a new container, using the latest stable Ubuntu and name the container gitlab.

Now, before we proceed with installing GitLab prerequisites, we need to tweak a few things on the host and with the container’s configuration.

lxc stop gitlab

With the container stopped, we need to temporarily give the container privileged access to the host. This is needed in order for the GitLab installer to set some kernel configuration options.

lxc config set gitlab security.privileged=true

If the host system is running Ubuntu with apparmor, we will need to tell apparmor not to restrict the container (you can change it back later after the install if you wish.)

lxc config set gitlab raw.lxc "lxc.apparmor.profile=unconfined"

Next, let’s make a backup of our current kernel settings:

sudo sysctl -a > sysctl.bak

Now that we have a backup, we can go ahead and start our new container up:

lxc start gitlab

Give it a few seconds and check the status with:

lxc list

If it shows the gitlab container as running, we can go ahead and move our work to inside the container:

lxc exec gitlab /bin/bash

We should now be presented with a root shell that is running side the container! Since we are running inside the container as root from here we do not need to run “sudo” before any commands while inside the container.

Our first task from inside the container, is to remount the kernel’s proc filesystem from read-only to read-write mode. This is required otherwise the gitlab installer will not be able to make any changes to the kernel’s configuration. Once we finish the install you will not need to execute the below command for normal operation.

mount -o remount rw /proc/sys

Now we can move on to checking for updates:

apt update
apt upgrade

If there are any available updates now would be a good time to install. This will ensure we are working with the latest security and bug fixes.

Next up, we need to make sure we have some basic packages installed:

apt install -y curl openssh-server ca-certificates tzdata perl

Now we will want to install postfix in order for GitLab to have the ability to send notifications.

apt install -y postfix

You will be prompted to select a configuration type for the postfix mail server, you will want to select “Internet Site” from the list and if there happens to be any additional questions just hit enter to select the defaults.

We are now ready to add the official GitLab apt repository for the community edition (note: if using the enterprise edition, change “ce” to “ee”.) with this curl command:

curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | bash

Now we can finally get to installing GitLab itself, again we are going to install the community edition (ce) if you are instead installing the enterprise edition (ee) you will need to change the package name. Also, if you are not using a real domain name or you have your own SSL certificates, you can add “LETSENCRYPT=”false”” before “apt” to disable the automatic SSL certificate creation. We also need to tell the installer what our intended URL is, even if it is not a real DNS name. For example, in testing I often use “https://gitlab.lxd” as the EXTERNAL_URL.

EXTERNAL_URL="https://gitlab.example.com" LETSENCRYPT="false" apt install gitlab-ce

The above step will take a while, so go grab some coffee and a bagel.

When the install finishes you will be be given some info about the state of things and returned to the container’s bash prompt. If you encounter any errors you can always run the above apt command again and it will perform an upgrade and no data (if any) will be lost. I sometimes encounter errors with network connections during the install, such is the life of working from home, and re-running the install allows me to work through them.

In order to access the GitLab web interface, we will need the IP address from the lxc output:

canutethegreat@diagonalley:~$ lxc info lxc
Name: gitlab
Location: none
Remote: unix://
Architecture: x86_64
Created: 2021/02/04 19:00 UTC
Status: Running
Type: container
Profiles: default
Pid: 72313
Ips:
  lo: inet 127.0.0.1
  lo: inet6 ::1
  eth0: inet 10.12.206.95 veth4544f711
  eth0: inet6 fd42:197c:de0f:bd1e:216:3eff:fe30:9fe5 veth4544f711
  eth0: inet6 fe80::216:3eff:fe30:9fe5 veth4544f711
Resources:
  Processes: 384
  CPU usage:
    CPU usage (in seconds): 1382
  Memory usage:
    Memory (current): 2.83GB
    Memory (peak): 2.84GB
    Swap (current): 79.26MB
    Swap (peak): 79.20MB
  Network usage:
    eth0:
      Bytes received: 890.94kB
      Bytes sent: 476.43kB
      Packets received: 808
      Packets sent: 391
    lo:
      Bytes received: 415.07MB
      Bytes sent: 415.07MB
      Packets received: 87005
      Packets sent: 87005

or from the ip command from inside the container:

root@gitlab:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
13: eth0@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:30:9f:e5 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.12.206.95/24 brd 10.12.206.255 scope global dynamic eth0
       valid_lft 2961sec preferred_lft 2961sec
    inet6 fd42:197c:de0f:bd1e:216:3eff:fe30:9fe5/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 3492sec preferred_lft 3492sec
    inet6 fe80::216:3eff:fe30:9fe5/64 scope link 
       valid_lft forever preferred_lft forever

As you can see from both of the output’s above, the container has an IPv4 IP address of “10.12.206.95.”

We will enter “https://<ip>” (“https://10.12.206.95” in my case) in our web browser running on the host and we will be presented with the initial login screen:

Screenshot after initial install of gitlab

Go ahead and type in a secure password for the root user and type it in a second time to confirm. We will then be asked to login, type in “root” as the username and the password we just set:

That’s it, we now having a running gitlab install inside an LXD container! Where you go from here depends on what you need. I typically create a user account for myself so that I am not using the root account for everything, even if I’m just testing.

Now that we have completed the initial setup, we can optionally undo one of the steps related to allowing the container to access the kernel as it is only needed for installation and upgrades. From the host run:

lxc config set gitlab security.privileged=false

to remove the containers privileged access to the host.

It is also worth mentioning that if you ever remove the gitlab container and wish to revert the kernel changes, you can do so on the host if you have your backup file handy:

cat sysctl.bak | sudo sysctl -e -p -

I find it interesting to check on the resource consumption of the gitlab container. You can get info from LXD on the host about the container by running the same command we used to get the IP address previously:

lxc info gitlab

The output on my test machine after logging into gitlab web interface look like this:

Name: gitlab
Location: none
Remote: unix://
Architecture: x86_64
Created: 2021/02/04 19:00 UTC
Status: Running
Type: container
Profiles: default
Pid: 72313
Ips:
  lo: inet 127.0.0.1
  lo: inet6 ::1
  eth0: inet 10.12.206.95 veth4544f711
  eth0: inet6 fd42:197c:de0f:bd1e:216:3eff:fe30:9fe5 veth4544f711
  eth0: inet6 fe80::216:3eff:fe30:9fe5 veth4544f711
Resources:
  Processes: 370
  CPU usage:
    CPU usage (in seconds): 94
  Memory usage:
    Memory (current): 2.59GB
    Memory (peak): 2.80GB
    Swap (current): 41.24MB
  Network usage:
    eth0:
      Bytes received: 797.37kB
      Bytes sent: 469.80kB
      Packets received: 342
      Packets sent: 327
    lo:
      Bytes received: 5.60MB
      Bytes sent: 5.60MB
      Packets received: 1501
      Packets sent: 1501

I find that my GitLab containers seem to typically use around ~4GB of memory with only a couple of users.

With GitLab up and running in an LXD container we can use the power of LXD to do cool things such as snapshots, live migrations, and cloning as well as take advantage of security by design, advanced resource control, and more! If you have multiple machines set up with LXD clustering you can selfhost and also not have to worry about downtime due to node failures.

The multitude of features provided by GitLab and the flexibility of LXD makes for an awesome combination to use in both my development environment and production. I can spin-up test containers in dev, deploy new versions in prod, and migrate to new hardware with ease. I hope this article finds you well and has provided some useful information on what it looks like to run GitLab inside LXD and how to get started with it. I enjoy receiving feedback; be it suggestions, corrections, or questions. Feel free to drop some love, be safe, and hack away!

I Love Coffee! https://ko-fi.com/canutethegreat