One post tagged with "Shell Scripting"

Introduction​

This document outlines the steps taken to deploy and configure GitLab Runners, including the installation of Terraform, ensuring that the application team can focus solely on writing pipelines.

Architecture​

The following diagram displays the solution architecture.

AWS CloudFormation is used to create the infrastructure hosting the GitLab Runner. The main steps are as follows:

1. The user runs a deploy script to deploy the CloudFormation template. The template is parameterized, and the parameters are defined in a properties file. The properties file specifies the infrastructure configuration and the environment in which to deploy the template.
2. The deploy script calls CloudFormation CreateStack API to create a GitLab Runner stack in the specified environment.
3. During stack creation, an EC2 autoscaling group is created with the desired number of EC2 instances. Each instance is launched via a launch template created with values from the properties file. An IAM role is created and attached to the EC2 instance, containing permissions required for the GitLab Runner to execute pipeline jobs. A lifecycle hook is attached to the autoscaling group on instance termination events, ensuring graceful instance termination.
4. During instance launch, GitLab Runner will be configured and installed. Terraform, Git, and other software will also be installed as needed.
5. The user may repeat the same steps to deploy GitLab Runner into another environment.

Infrastructure Setup with CloudFormation​

Customizing the CloudFormation Template​

The initial step in deploying GitLab Runners involved setting up the infrastructure using AWS CloudFormation. The standard CloudFormation template was customized to fit the unique requirements of the environment.

CloudFormation Template Location: GitLab Runner Template

CloudFormation Template Location: GitLab Runner Scaling Group / Cluster Template

For any automation requirement or issues, please reach out to us Contact Us

Parameters used:

Deploying the CloudFormation Stack​

To deploy the CloudFormation stack, use the following command. This command assumes you have AWS CLI configured with the appropriate credentials:

aws cloudformation create-stack --stack-name amazon-ec2-gitlab-runner-demo1 --template-body file://gitlab-runner.yaml --capabilities CAPABILITY_NAMED_IAM

To update the stack, use the following command:

aws cloudformation update-stack --stack-name amazon-ec2-gitlab-runner-demo1 --template-body file://gitlab-runner.yaml --capabilities CAPABILITY_NAMED_IAM

This command will provision a CloudFormation stack similar to table shown below:

Shell-Based Installation Approach​

Rather than using Docker, in your environment, you can use Shell (kernel) for installing GitLab Runner and Terraform directly on the EC2 instances. Using shell rather than container provides the following benefits:

• Simpler Debugging: Direct installation via shell scripts simplifies the debugging process. If something goes wrong, engineers can SSH into the instance and troubleshoot directly rather than dealing with Docker container issues.
• Performance Considerations: Running the runner directly on the EC2 instance reduces the overhead introduced by containerization, potentially improving performance.

Installation Commands​

Below are the key commands used in the shell script for installing GitLab Runner and Terraform:

#!/bin/bash
# Update and install necessary packages
yum update -y
yum install -y amazon-ssm-agent git unzip wget jq

# Install Terraform
wget https://releases.hashicorp.com/terraform/1.0.11/terraform_1.0.11_linux_amd64.zip
unzip terraform_1.0.11_linux_amd64.zip
mv terraform /usr/local/bin/

# Install GitLab Runner
sudo curl -L --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64
sudo chmod +x /usr/local/bin/gitlab-runner
sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash
sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
sudo gitlab-runner start

# Source GitBash
echo 'export PATH=$PATH:/home/gitlab-runner' >> ~/.bashrc
source ~/.bashrc

Configuration and Usage​

Registering the GitLab Runner​

Once the GitLab Runner is installed, it needs to be registered with your GitLab instance. This process can be automated or done manually. Below is an example of how you can register the runner using the gitlab-runner register command:

gitlab-runner register \
  --non-interactive \
  --url "https://gitlab.com/" \
  --registration-token "YOUR_REGISTRATION_TOKEN" \
  --executor "shell" \
  --description "GitLab Runner" \
  --tag-list "shell,sgkci/cd" \
  --run-untagged="true" \
  --locked="false"

A simple command:

sudo gitlab-runner register --url https://gitlab.com/ --registration-token <Your registration token>

Example:
sudo gitlab-runner register --url https://gitlab.com/ --registration-token GR1348941Du4BazUzERU5M1m_LeLU

This command registers the GitLab Runner to your GitLab project, allowing it to execute CI/CD pipelines directly on the EC2 instance using the shell executor.

Attaching Runner to GitLab Repo​

Navigate to Repo → Settings → CI/CD. Your runner should show up. Click "Enable for this project," after which the runner should be visible.

Note: To ensure that the runner picks up your job, ensure that the right tag is in place, and you may need to disable the Instance Runners.

🔚 Call to Action​

Choosing the right platform depends on your organizations needs. For more insights, subscribe to our newsletter for insights on cloud computing, tips, and the latest trends in technology. or follow our video series on cloud comparisons.

Interested in having your organization setup on cloud? If yes, please contact us and we'll be more than glad to help you embark on cloud journey.

💬 Comment below:
Which tool is your favorite? What do you want us to review next?