Jenkins Integration with Rafay Systems

Jenkins and Rafay can be used together to build highly automated self-service pipelines for build and release processes. Users can quickly deploy their applications to 100s of clusters across heterogeneous providers.

CI/CD using Jenkins and Rafay

CI Platforms like Jenkins can integrate directly with SCM such as GitHub to “checkout” source code, “build” a container image and “push” the resulting image to your preferred container registry.

Jenkins based pipelines can also be easily configured to directly interact with the Rafay Controller using the Rafay CLI utility.

For GitOps type pipelines, you can store the workload’s configuration YAML in your Git repository.

Illustrative 6-Stage Pipeline

The Jenkins pipeline itself is a groovy based script that can be extensively customized to suit your exact needs.

Once the pipeline reaches stage 5, Jenkins instructs the Rafay Controller to deploy the workload using the “Rafay CLI” and the workload’s “YAML based Configuration”. A full audit trail is available for all actions performed on the Rafay Controller using the Rafay CLI.

Jenkins Credentials 

This example assumes that you are familiar with the basics of Jenkins and know how to create and manage a Jenkins pipeline. 

Ensure that you configure required secrets etc into your Jenkin server’s “Credentials” repository. 

Note that the pipeline will automatically download and initialize the Rafay CLI. 

Download Rafay Workload Config YAML

Once a workload has been configured/tested successfully using the Rafay App Console, the workload’s YAML can be downloaded and modified programmatically as required. There are two approaches for this. 

Download via App Console 

Click on “Action” for your Workload and select “Download Workload Config”

Download via Rafay CLI

The Rafay CLI can be used to programmatically retrieve the workload’s YAML. 

Example Pipeline Groovy Script

#!/usr/bin/env groovy
pipeline {
  agent any

  /* Information that Jenkins will require to access the following
   * 1. Git repository
   * 2. Container Registry 
   * 3. Rafay Controller 
   * Use Jenkins Credentials to securely store access credentials
  */
  environment{
      /* Location of our Jenkins Server */
      AWS_REGION="us-east-1"

      /* Rafay Controller Details */
      RAFAY_REST_ENDPOINT="https://app.rafay.dev/"
      RAFAY_USERNAME = credentials('RAFAY_USERNAME_SB')
      RAFAY_API_KEY=credentials('RAFAY_API_KEY_SB')
      RAFAY_SECRET_KEY=credentials('RAFAY_SECRET_KEY_SB')

      /* Container Registry Details */
      RAFAY_REGISTRY_ENDPOINT="rcr.rafay-edge.net:6000"
      RAFAY_ORGANIZATION_LABEL="test_prod_org/kutumba"
      RAFAY_REGISTRY_SECRET=credentials('RAFAY_REGISTRY_SECRET_SB')
      registryCredential = 'rafay_registry_km_sb'

      /* Container Image Details */ 
      dockerImage = ''
      ALPINE_IMAGE="alpine:3.7"
      BUILD_NUMBER="${env.BUILD_NUMBER}"

      /* Application Workload Details */ 
      WORKLOAD_NAME="demo-jenkins-pipeline"
      CONTAINER_NAME='httpbin'
      WORKLOAD_HOST='demo-jenkins-pipeline.run.rafay-edge.net'
  }

  stages {

      /* FIRST STAGE in our Pipeline 
       * Jenkins will checkout source from the specified GitHub repository 
      */ 
      stage("STEP 1: Checkout Source Code") {
        steps {
            checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'git-jenkins', url: 'https://github.com/RafaySystems/rcloud']]])

        }
      }

      /* SECOND STAGE in our Pipeline 
       * Jenkins will compile the source code 
      */
      stage("STEP 2: Compile Source Code") {
        steps{
            sh label: '', script: '''cd components/httpbin
                            make'''
        }
      }

      /* THIRD STAGE in our Pipeline 
       * Jenkins will build a Docker Container Image
      */
      stage("STEP 3: Build Container Image") {
        steps{
            script {
                dockerImage = docker.build("${RAFAY_REGISTRY_ENDPOINT}/${RAFAY_ORGANIZATION_LABEL}/httpbin:${env.BUILD_NUMBER}", "--build-arg ALPINE=$ALPINE_IMAGE ./components/httpbin/")
            }
        }
      }

      /* FOURTH STAGE in our Pipeline
       * Jenkins will push the container image to the specified Container Registry 
      */ 
      stage('STEP 4: Push Image to Registry') {
        steps{
          script {

              /* Push the Container Image to the Configured Container Registry */ 
              docker.withRegistry("https://${RAFAY_REGISTRY_ENDPOINT}", registryCredential ) {
              dockerImage.push()
              }
          }
        }
      }

      /* FIFTH STAGE in our Pipeline 
       * Jenkins will ask the Rafay Controller to Create and Publish the Application Workload 
      */ 
      stage("STEP 5: Create and Publish Workload in Rafay") {
        steps {

            /* Download, Install and Initialize the Rafay CLI */ 
            /* This ensures that you always have the latest version */ 
            sh label: '', script: '''rm -f rafay-cli-linux-amd64.tar.bz2
            wget https://s3-us-west-2.amazonaws.com/rafay-prod-cli/publish/rafay-cli-linux-amd64.tar.bz2
            tar -xf rafay-cli-linux-amd64.tar.bz2
            chmod 0755 rafay-cli
            echo "RAFAY_CLI_LOG_FILE_LOCATION=${HOME}/.rafay/cli/cli.log" > rafay-cli-config-sb
            echo "RAFAY_REGISTRY_ENDPOINT=$RAFAY_REGISTRY_ENDPOINT" >> rafay-cli-config-sb
            echo "RAFAY_REST_ENDPOINT=$RAFAY_REST_ENDPOINT" >> rafay-cli-config-sb
            echo "RAFAY_API_KEY=$RAFAY_API_KEY" >> rafay-cli-config-sb
            echo "RAFAY_API_SECRET=$RAFAY_SECRET_KEY" >> rafay-cli-config-sb
            echo "RAFAY_USERNAME=$RAFAY_USERNAME" >> rafay-cli-config-sb
            echo "RAFAY_REGISTRY_SECRET=$RAFAY_REGISTRY_SECRET" >> rafay-cli-config-sb
            echo "RAFAY_ORGANIZATION_LABEL=$RAFAY_ORGANIZATION_LABEL" >> rafay-cli-config-sb
            ./rafay-cli init -f rafay-cli-config-sb -v

            /* Download the Workload's Rafay YAML File. You can also store the YAML file in your Github Repo */ 
            rm -f demo-jenkins-pipeline.yaml
            wget http://km-webserver.dev.rafay-edge.net/demo-jenkins-pipeline.yaml

            /* Use the Rafay CLI to Create a Workload based on the YAML File */
            ./rafay-cli workload create demo-jenkins-pipeline.yaml

            /* Set the New Container Image in our Workload */ 
            ./rafay-cli workload set-image $WORKLOAD_NAME $CONTAINER_NAME $RAFAY_ORGANIZATION_LABEL/httpbin:${BUILD_NUMBER}

            /* Rafay Provides a unique version for every Workload Publish Operation */ 
            workload_version=`./rafay-cli workload publish $WORKLOAD_NAME|sed -n -e 2p|cut -d\' \' -f8|sed \'s/.$//\'`

            /* Publish can take a few minutes if your container images are large
             * Check to make sure Workload was Successfully Published by Rafay 
             */ 
            workload_status="WorkloadNotPublished"
            workload_status_iterations=1
            while [ "$workload_status" != "WorkloadPublished" ];
            do 
                workload_status=`./rafay-cli workload status $WORKLOAD_NAME -p $workload_version |jq .publish_status|cut -d\'"\' -f2`
                echo $workload_status
                sleep 10
                if [ $workload_status_iterations -ge 30 ];
                then
                    break
                fi
                workload_status_iterations=$((workload_status_iterations+1))
            done
            '''
        }
      }

      /* SIXTH STAGE in our Pipeline
       * Jenkins will initiate Functional and Non-Functional Tests on the Published Application Workload 
      */
      stage("STEP 6: Test Application Workload") {
        steps {
            sh label: '', script: '''
            sleep 60

            /* A simple test to check if we can access the Workload via https
             * Replace this with your Functional and Non Functional Tests 
            */ 
            wget https://$WORKLOAD_HOST -qS -O /dev/null
            wget https://$WORKLOAD_HOST/ip -qS -O /dev/null
            wget https://$WORKLOAD_HOST/edgeinfo -qS -O /dev/null
            ''' 
        }
      }
    }
 }