Pull GitLab Remote Backup Bash Script

GitLab's backup files are generated locally on the server by default, though there are a lot of remote backup provider, I still want to pull backup files to local machine and not rely on any third party service.

If you are using GitLab on a remote server and want to backup your data, you can use the following bash script to download the latest backup file from the server to your local machine.

Pull GitLab Backup Files from Server

#!/usr/bin/env bash

set -v

# Declare remote server variables
user="<User name>"
server="<Server IP>"
remote_dir="<Remote directory>"
backups_dir="$remote_dir/data/backups/"
port=<Port>

# Local directory to download file to
local_dir="<Local directory to download file to>"

# Get the latest file from server
latest_file=$(ssh -i "</path/to/custom/key>" -p $port $user@$server  "ls -t $backups_dir | head -1")
echo $latest_file

mkdir -p $local_dir

# Download latest file to local machine
scp -P $port $user@$server:"$backups_dir/$latest_file" $local_dir

echo "Latest file $latest_file downloaded to $local_dir"

# Backup config files, which contains secrets that are needed for restoring the backup
# zip $config_dir on remote server and download it to local using ssh and scp
ssh -i "</path/to/custom/key>" -p $port $user@$server "cd $remote_dir && zip -r config.zip ./config"
scp -i "</path/to/custom/key>" -P $port $user@$server:"$remote_dir/config.zip" $local_dir

Automate This Script in Docker Container

I decide to deploy this script as a service in a Docker container, so it can be used as automated backup solution.

Let's just write a simple nodejs app that runs cron job and can be dockerized as a service.

const cron = require('node-cron');
const { exec } = require('child_process');

// Run the script every day at 1:00 AM
cron.schedule('0 1 * * *', () => {
  exec('bash ./pull-gitlab-backup.sh', (err, stdout, stderr) => {
    if (err) {
      console.error(err);
      return;
    }
    console.log(stdout);
  });
});

Dockerize it and run it as a service.

FROM node:14.15.4-alpine3.12

WORKDIR /usr/src/app

COPY . /usr/src/app

RUN npm install

CMD [ "node", "index.js" ]

Credits

This article was written with the help of ChatGPT and Bing Chat.