Continuous Integration and Continuous Development(CI/CD)

Continuous Integration and Continuous Development(CI/CD) is a new methodology, which is gaining huge space in the software industry and got popularity in less time due to its features. However, there are many companies that already have started the software development process using this methodology and others are on this way.  There are many advantages of CI/CD, which attracted developers and the software industry towards it and some are below:

• Speed: Continuous integration is meant to be rapid with instant feedback
• Accuracy: Applying automation to the deployment process is a great starting point
• Reliability: Having a reliable CI/CD pipeline improves the speed of building and deploying new commits significantly
• Smaller Code Changes: Using continuous testing, these small pieces can be tested as soon as they are integrated into the code repository
• Faster Release Rate: Failures are detected faster and as such, can be repaired faster, leading to increased release rates

Stages of CI/CD

There are no defined stages of CI/CD. However, CI/CD is a very flexible methodology and developers can set these stages according to their choice. There are below some standard choice of CI/CD stages and these are all in the pipeline if previous stages fail it will not go to the next stage.

👁 Image

1. Version Control: It is used to manage the changes in code, documentation, etc. For example, GitLab, GitHub, TFS, etc.
2. Build: This stage build the solution of the software product and store the artifacts
3. Unit Test: After successfully building the solution in stage 2, it will go for unit testing if it has any.
4. Deploy: After successfully building and Unit testing stages, the solution will be deployed in a temporary environment.
5. Automated Test: This stage will run all the automated tests to verify that product is able to fulfill all the requirements before it deploys to production.
6. Deploy To Production: After successfully running all the test cases and verification, now this stage will deploy the product in a production environment.

There are many software tools available which are providing an implementation of CI/CD, and some are below and these can be used by developers or companies' choice and each one has its own characteristics:

• Jenkins
• Circleci
• TeamCity
• Bamboo
• GitLab

GitLab CI/CD Requirements

There can be any software tool to implement CI/CD as mentioned above. However, GitLab providing CI/CD implementation in a flexible way, and without much knowledge or other additional software, anyone who has some basic knowledge about software development, can easily setup CI/CD configuration with GitLab. Below are some requirements:

1. Version Control: GitLab is itself a version control to implement CI/CD
2. GitLab Runner: It is an important tool to implement CI/CD and need to install it before any configuration of CI/CD. It picks up and execute jobs for GitLab when anyone commits the changes. There are two types of Runner - Shared Runners and Specific Runner.
   • Shared Runner: These Runners execute code of different projects on the same Runner and use shared runners when you have multiple jobs with similar requirements.
   • Specific Runner: It is an individual runner and each project will be executed by a separate runner and managed by the only developer who is working on this specific project.
3. GitLab Runner Executors: GitLab Runner implements a number of executors that can be used to run builds in different scenarios. There are several executors available in GitLab, which can be implemented according to the requirements of the project or the availability of the software tools. Some executors are below:
   • SSH: The SSH executor supports only scripts generated in Bash and the caching feature is currently not supported. This is a simple executor that allows you to execute builds on a remote machine by executing commands over SSH.
   • Shell: Shell is the simplest executor to configure. All required dependencies for your builds need to be installed manually on the same machine that the Runner is installed on.
   • Parallels: This executor configuration is used with Virtual Box.
   • VirtualBox: This type of executor allows to use an already created virtual machine, which is cloned and used to run your build.
   • Docker: A great option is to use Docker as it allows a clean build environment, with easy dependency management (all dependencies for building the project can be put in the Docker image).
   • Docker Machine (auto-scaling): The Docker Machine is a special version of the Docker executor with support for auto-scaling. It works like the normal Docker executor but with the build, hosts created on demand by Docker Machine.
   • Kubernetes executor: The Kubernetes executor allows to use an existing Kubernetes cluster for your builds.
   • Custom executor: The Custom executor allows to specify your own execution environments.
4. Yml File: Yml file is needed to run the CI/CD and it will need to store in a source file of the project, and it includes configuration of each technology and proceeds method.                         

GitLab Runner Windows

Follow the below steps, download GitLab Runner on Windows, and go through commands to verify and register it.  

1. Download the GitLab runner from  https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-windows-386.exe 2. Create a folder in your local directory (for ex. C:\GitLab-Runner) and rename the downloaded file to gitlab-runner.exe, store it in this folder. 

3. Type below command, to verify and register it.

GitLab Runner Commands

There are many commands of GitLab Runner if already installed in the system, For example, start GitLab Runner, Stop GitLab Runner, Register, etc. 

Command	Description
gitlab-runner.exe register	Register the project with CI/CD
gitlab-runner.exe start	Start the runner
gitlab-runner.exe stop	Stop the runner
gitlab-runner.exe status	To know the status of gitlab-runner
gitlab-runner unregister --name  test-runner	Unregister Runner of a project and replace test-runner with your runner name and this name can be found inside the config.toml file (where your gitlab.exe) available.
gitlab-runner unregister --url http://gitlab.example.com/ --token t0k3n	Remove Runner by URL and token
gitlab-runner unregister --all-runners	Unregister All Runners
gitlab-runner restart	This command stops and then starts the GitLab Runner service
gitlab-runner uninstall	This command stops and uninstalls the GitLab Runner from being run as a service
gitlab-runner exec	To see a list of available executors, run
gitlab-runner --help	Check a recent list of commands by executing
gitlab-runner run --help	can see the name of the environment variable
gitlab-runner --debug	To run a command in debug mode
gitlab-runner exec shell	To see a list of all available options for the shell executor, run

GitLab Runner Linux 

Follow the below steps to download GitLab Runner on Linux Environment and go through the below commands to verify and register a project.

1. sudo curl -L --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64 [Simply download one of the binaries for your system]
2. sudo chmod +x /usr/local/bin/gitlab-runner [Give it permissions to execute]
3. sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash [Create a GitLab CI user]
4. sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner [Install and run as service]
5. sudo gitlab-runner start
6. After doing the successful installation now GitLab Runner started and can perform to do the registration of the repository.

Command	Description
sudo gitlab-runner register	Register the project with CI/CD
sudo gitlab-runner start	Start the runner
sudo gitlab-runner stop	Stop the runner
sudo gitlab-runner status	To know the status of gitlab-runner
sudo gitlab-runner unregister --name  test-runner	Unregister Runner of a project and replace test-runner with your runner name and this name can be found inside the config.toml file (where your gitlab.exe) available
sudo gitlab-runner unregister --url http://gitlab.example.com/ --token	Remove Runner by Url and token
sudo gitlab-runner unregister --all-runners	Unregister All Runners
sudo gitlab-runner restart	This command stops and then starts the GitLab Runner service
sudo gitlab-runner uninstall	This command stops and uninstalls the GitLab Runner from being run as a service
sudo gitlab-runner exec	To see a list of available executors, run
sudo gitlab-runner --help	Check a recent list of commands by executing
sudo gitlab-runner run --help	Can see the name of the environment variable
sudo gitlab-runner --debug	To run a command in debug mode
sudo gitlab-runner exec shell	To see a list of all available options for the shell executor, run

CI/CD Windows Project Configuration

Follow the below steps to configure CI/CD with Windows Environment projects:

1. Go to the downloaded gitlab-runner .exe path, where you have stored it, and open the command prompt. (follow above windows command)
2. Check the gitlab-runner status, stop it if it is already running before registering any new project with CI/CD
3. Now type, gitlab-runner.exe register and fill the below details
4. Enter your GitLab instance URL: Go to GitLab Account -> Select Project which you want to implement CI/CD -> Settings -> CI/CD
5. Enter the gitlab-ci token for this runner: Go to GitLab Account -> Select Project which you want to implement CI/CD -> Settings -> CI/CD.
6. Enter the gitlab-ci description for this runner: put any name (for ex: MyRunner1) and this just to remember which runner' is running.
7. Enter the gitlab-ci tags for this runner: This is tag to start the CI / CD can put any name or no name it's not required (for ex: my-tag, another-tag)
8. Enter the executor: need to select one executor from the above-given list of executors and every executor has their own characteristics.
9. Now the project is registered successfully and start the GitLab runner.

👁 Image

CI/CD Linux Project Configuration

Follow the below steps to configure CI/CD with Linux Project.

1. Go to your Linux machine, where you have installed the GitLab Runner and Git
2. Check the GitLab runner status, if it is already running stop it before registering any new project
3. Type sudo gitlab-runner register -> to register the new project with CI/CD
4. Now follow all the below steps, same as Windows, and fill in the required information

Add Yml File in Source Code

This is an important step to include yml file to your source code directory, if you do not include it CI/CD will not work.  This file includes a configuration that how this project will run.

For Example, .Net project Yml File format 

variables:
NUGET_PATH: 'C:\Tools\Nuget\nuget.exe'
MSBUILD_PATH: 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Current\Bin\MSBuild.exe'
XUNIT_PATH: packages\xunit.runner.console.2.3.1\tools\net452
UNITTEST_FOLDER: '.\tests\CiCdExample.Tests\bin\Release\'

stages:
- build
- test
- deploy

build_job:
stage: build
only:
- branches
script:
- '& "$env:NUGET_PATH" restore'
- '& "$env:MSBUILD_PATH" /p:Configuration=Release /clp:ErrorsOnly'
- '& "$env:MSBUILD_PATH" src\CiCdExample\CiCdExample.csproj /p:DeployOnBuild=true /p:Configuration=Release /P:PublishProfile=FolderProfile.pubxml'
artifacts:
expire_in: 20 days
paths:
- '.\src\CiCdExample\bin\Release\Publish\'
- '$env:UNITTEST_FOLDER'
- '.\$env:XUNIT_PATH\*.*'

test_job:
stage: test
only:
- branches
script:
- '& "$env:XUNIT_PATH\xunit.console.exe" "$env:UNITTEST_FOLDER\CiCdExample.Tests.dll"'
dependencies:
- build_job

deploy_job:
stage: deploy
only:
- branches
script:
- 'xcopy /y /s ".\src\CiCdExample\bin\Release\Publish\*.*" "C:\Tools\inetpub\wwwroot\ci-cd-example"'
dependencies:
- build_job
- test_job

C/C++ Yml Project format

image: gcc

stages:
- build
- test

build_job:
stage: build
only:
- master
script: 
- cd sourcecode
- export CXX=/usr/bin/g++
- export CC=/usr/bin/gcc
- chmod -R 777 *
- ./compileproject.sh

artifacts:
expire_in: 365 days
paths:
- sourcecode/testdata_1.0

test_job:
stage: test
only:
- master
script:
- pwd 
- ls -lrt sourcecode/testdata_1.0
- cd testDir
- chmod -R 777 *
- ./runtests.sh

dependencies:
- build_job