Creating and Reusing Bash Variables Across Multiple Steps in a Harness Pipeline

Let's explore how to create and reuse variables across multiple steps in a Harness pipeline.

In Step_1_Capture, we are capturing some simple delegate information in variables from a Bash execution. In Step_2_Echo, we are using the captured variables from Step 1 to print them on the screen and perform a ping operation.

We will use three variables to demonstrate three scenarios:

1. A variable whose name never changes across Bash and steps (kernel variable)

2. A variable whose name changes multiple times across Bash and steps (processor variable)

3. A variable whose value we want to keep as a secret but still use in Bash operations (IP address variable)

We will create the two steps of type 'Shell Script'.

Note: If you have a habit of leaving spaces around equal signs in scripts, it will not work in Bash and will cause unexpected behavior. For example, in Python, writing a = "sample" works, but in Bash, we have to write a="sample" without any spaces to make it work flawlessly.

Returning to Harness, here's how to create a variable:

1. In the step configuration panel, go to the "Optional Configuration" dropdown.

2. Click on the button named "Add Output Variable".

3. A new row will be created with three fields: Name, Type, and Bash Output Variable.

   • Name: The name you want your variable to have in the Harness setup.

   • Type: Choose either String, Number, or Secret.

   • Bash Output Variable: The name of the variable in the Bash script that we want to store when the script ends.

Here is the Bash script for this step:

kernel=$(uname -s)
processor=$(uname -p)
ip=$(hostname -I)

When we run this step, the following output will appear:

To use these variables in another step, we need to use them as input variables and reference them in the new step. Here's how to get their reference strings:

1. Go to the Output tab in the pipeline execution results.

2. You'll see the variables under "Output Variables" with their captured values.

   • For secret variables, note that the value is hidden using *.

3. To get the variable's reference string:

   • Hover over the variable name.

   • A copy icon will appear beside the variable name with the text "Click to copy".

   • Click on that icon to copy the reference string for that variable.

     

Now, let's create another step and set up the input variables:

1. In the new step, go to the "Optional Configuration" dropdown.

2. Click on "Add Input Variable" to create a new row.

3. In this new row:

   • Name: Provide the name you want to use for the variable in the Bash script.

   • Type: Even if the variable was captured as a secret in the previous step, declare the type here as String or Number (it will still be treated as a secret during execution).

   • Value: Paste the reference string you copied from the "Output Variables" section for each variable.

Now we can use these variables in the Bash script as ${variable_name}.

Note that we have used braces {}.

Here is the Bash script for this step:

echo "The kernel is: ${kernel}"
echo "The processor is: ${processor_renamed}"
echo "The ip address is: ${ip_address_reuse}"
ping -c 3 ${ip_address_secret}

In the second step, we use ping with the -c 3 option to limit the ping attempt count to 3. In the execution logs, you can see that whenever the IP address is referred to in the Bash script, it is represented using *.

If you want to view the values used, go to the Input tab and look at the variables in the "Environment Variables" section.

Note: We have limited the execution of this pipeline to run through a delegate named "one" because the ping command do not work on a generic delegate by default. The ping tool has been installed on the "one" delegate using “microdnf install iputils”.

Here is the YAML of the complete pipeline, which can be used to recreate the pipeline:

pipeline:
  name: Variable Demo
  identifier: Variable_Demo
  projectIdentifier: learning_winrm
  orgIdentifier: default
  tags: {}
  stages:
    - stage:
        name: Stage_1
        identifier: Stage_1
        description: ""
        type: Custom
        spec:
          execution:
            steps:
              - step:
                  type: ShellScript
                  name: Step_1_Capture
                  identifier: Step_1_Capture
                  spec:
                    shell: Bash
                    executionTarget: {}
                    source:
                      type: Inline
                      spec:
                        script: |-
                          kernel=$(uname -s)
                          processor=$(uname -p)
                          ip=$(hostname -I)
                    environmentVariables: []
                    outputVariables:
                      - name: kernel
                        type: String
                        value: kernel
                      - name: pro
                        type: String
                        value: processor
                      - name: ip_address
                        type: Secret
                        value: ip
                  timeout: 10m
              - step:
                  type: ShellScript
                  name: Step_2_Echo
                  identifier: Step_2_Echo
                  spec:
                    shell: Bash
                    executionTarget: {}
                    source:
                      type: Inline
                      spec:
                        script: |-
                          echo "The kernel is: ${kernel}"
                          echo "The processor is: ${processor_renamed}"
                          echo "The ip address is: ${ip_address_reuse}"
                          ping -c 3 ${ip_address_secret}
                    environmentVariables:
                      - name: kernel
                        type: String
                        value: <+pipeline.stages.Stage_1.spec.execution.steps.Step_1_Capture.output.outputVariables.kernel>
                      - name: processor_renamed
                        type: String
                        value: <+pipeline.stages.Stage_1.spec.execution.steps.Step_1_Capture.output.outputVariables.pro>
                      - name: ip_address_secret
                        type: String
                        value: <+pipeline.stages.Stage_1.spec.execution.steps.Step_1_Capture.output.outputVariables.ip_address>
                    outputVariables: []
                  timeout: 10m
        tags: {}
        delegateSelectors:
          - one