Comprehensive Guide to Linux Shell Scripting with Examples

Linux shell scripting is an important aspect of learning Linux as it enables you to automate tasks. Through automation you can schedule housekeeping jobs like backups, monitor system performance, and perform data processing tasks.

By the end of this article, you will understand:

• What is a shell script?

• How to write and execute a simple shell script.

• How to use variables, control structures, and functions in shell scripts.

• How to automate system tasks using shell scripts.

• Examples of shell scripts for automating system tasks, and more.

Introduction to Linux Shell Scripting

What is a shell script?

A shell script is a collection of commands that are executed in sequence and stored in a file. For example, you can write a script that echoes a message to a specific file and then copies it to another location. That script would use commands like echo, cp, and mv to accomplish the task.

Basic components of a shell script

A shell script has a few basic components:

• Shell scripts start with a shebang line that specifies the shell interpreter to use. For example, #!/bin/bash specifies that the script should be executed using the Bash shell.

• The filenames of shell scripts typically end with .sh to indicate that they are shell scripts.

• Shell scripts must have the execute permission set to run.

Creating and running a simple script

Now that you know what a shell script looks like, we can start writing a very simple shell script.

The goal of the script is to print a message to the console. Here's how you can do it:

• Create a new file called hello.sh using a text editor like nano or vim.

• Add the following content to the file:

#!/bin/bash
echo "Hello, World!"

• Save the file and exit the text editor.

• Provide execute permission to the script using the chmod command:

chmod +x hello.sh

• Run the script using the following command:

./hello.sh

If everything is set up correctly, you should see the message Hello, World! printed to the console.

Using comments

Comments are lines in a script that are ignored by the shell interpreter. To add comments to a script, you can use the # character.

For example:

#!/bin/bash

# This is a comment and would be ignored
echo "Hello, World!"

Variables and Data Types

Declaring and using variables

In Bash, variables are defined using the syntax variable_name=value. The variable's value can be accessed using the $ symbol.

#!/bin/bash

message="Happy coding!"
echo $message

The above script will print Happy coding! to the console.

As far as data types are concerned, Bash treats all variables as strings. However, based on the usage, you can treat them as integers or other data types.

Types of variables: local and environment

There are two main types of variables in Bash: local and environment variables.

Local variables are defined within a script and are only accessible within that script. They are not available to other scripts or programs.

Environment variables are defined in the shell environment and are accessible to all scripts and programs. You can set environment variables using the export command. They are mostly used to store system-wide settings and configurations.

String and numeric operations

As we discussed previously, Bash treats all variables as strings. However, you can perform arithmetic operations on variables by using the let command or double parentheses (( )).

#!/bin/bash

num1=10
num2=5

let result=num1+num2
# same output as: 
# result=$((result=num1+num2))

echo $result

Control Structures

Conditional statements: if, else, elif

Conditionals are expressions that evaluate to either true or false. Depending on the result of the evaluation, the script will execute different blocks of code.

The structure of conditional statements in Bash is as follows:

if [ condition ]; then
    # code block
elif [ condition ]; then
    # code block
else
    # code block
fi

Looping in Bash Scripting- for and while loops

Loops are used to execute a block of code multiple times. Loops can also iterate over numbers, strings, or until a specific condition is met.

1. For Loop

A for loop allows you to execute statements a specific number of times.

Looping with Numbers

The following example demonstrates a loop that iterates ten times, printing numbers from 1 to 10.

Example:

#!/bin/bash

for i in {1..10}
do
    echo $i
done

# Output
1
2
3
4
5
6
7
8
9
10

Looping with Strings

A for loop can also iterate over a set of predefined strings.

Example:

#!/bin/bash

for color in orange green yellow  
do
    echo $color
done

# Output
orange
green
yellow

2. While Loop

A while loop repeatedly executes a block of code as long as a specified condition remains true. To prevent an infinite loop, a counter must be incremented within the loop.

Looping Until a Condition is Met

The example below prints numbers from 1 to 100. The condition [[ $i -le 100 ]] ensures the loop runs until i reaches 100, and the counter statement (( i += 1 )) increases the value of i after each iteration.

Example:

#!/bin/bash

i=1
while [[ $i -le 100 ]] ; do
   echo "$i"
   (( i += 1 ))
done

Comparison and Logical Operators in Bash

Comparison Operators

Comparison operators are used to compare numbers and strings.

Numeric comparisons: are used with [ ], [[ ]], or (( )).

a=100
b=200

if [ "$a" -lt "$b" ]; then
  echo "$a is less than $b"
fi

String comparisons: are used with [ ] or [[ ]].

Example:

str1="a"
str2="b"

if [[ "$str1" < "$str2" ]]; then
  echo "$str1 comes before $str2"
fi

Logical Operators

Logical operators combine conditions.

Logical AND (&&):

• Returns true if both conditions are true.

• If the first condition is false, the second condition is not evaluated.

Example:

if [ condition1 ] && [ condition2 ]; then
  # code block
fi

The same can be written using [[ ]]:

if [[ condition1 && condition2 ]]; then
  # code block
fi

Logical OR (||):

• Returns true if at least one of the conditions is true.

• If the first condition is true, the second condition is not evaluated.

Example:

if [ condition1 ] || [ condition2 ]; then
  # code block
fi

The same can be written using [[ ]]:

if [[ condition1 || condition2 ]]; then
  # code block
fi

Logical NOT (!):

• Reverses the result of the condition. If the condition is true, ! makes it false, and vice versa.

Example:

if ! [ condition ]; then
  # code block
fi

Arithmetic Comparisons

For numeric comparisons, (( )) can be used without -eq, -lt, etc.

Example:

a=100
b=15

if (( a > b )); then
  echo "$a is greater than $b"
fi

Combining logical and comparison operators

You can combine logical and comparison operators to create complex conditions.

Example:

a=10
b=20

if [[ "$a" -gt 5 && "$b" -lt 20 ]]; then
  echo "Both conditions are true"
fi

Functions in Shell Scripts

Functions are used to organize code to avoid repetition. For example you might create a function that is called whenever a user provides input, parses it for errors, and returns the result for further processing.

Defining a function

A function is defined using this format:

function function_name {
  # code block
}

Calling a function

You can call a function by using its name followed by parentheses.

function_name

Passing arguments to functions

You can pass arguments to functions by using $1, $2, etc., to access the first, second, and so on arguments.

function greet {
  echo "Hello, $1!"
}

greet "Alice"
# Output: Hello, Alice!

Returning values from functions

Returning values from functions in Bash is a little different from other languages. You can:

• Use echo to print values and capture them in a variable.

• Use return to return a status code(0 to 255). 0 is considered a success, and any other value is considered a failure. 1 is general failure and remaining error codes are specific to the error type.

# example using echo

function add {
  echo $(( $1 + $2 ))
}

result=$(add 10 20)

echo $result
# Output: 30

# example using return

function check_odd() {
  if (( $1 % 2 != 0 )); then
    return 0  # Success for odd numbers
  else
    return 1  # Failure for even numbers
  fi
}

check_odd 5 && echo "Odd number" || echo "Even number"
check_odd 8 && echo "Odd number" || echo "Even number"

# Output:
# Odd number
# Even number

Input and Output

Reading user input

Reading input from the user or getting it from a file is a common task in shell scripting. In this section, we will discuss three ways to gather input for further use.

Reading the user input using read

#!/bin/bash

echo "Enter your name:"
read name
echo "Hello, $name!"

Reading input from the command line

You can pass arguments to a script when you run it from the command line. These arguments can be accessed using $1, $2, etc.

#!/bin/bash

echo "Hello, $1!"

When you run the script with an argument like this:

./hello.sh Alice

The output will be:

Hello, Alice!

Reading input from a file

You can read input from a file line by line and provide it as a source of input for your script.

Here is an example of reading input from a file named file_input.txt.

#!/bin/bash

while read line; do
  echo "Line: $line"
done < file_input.txt

Displaying output

Scripts can display outputs in several ways. Let's look at some common methods.

Printing to the terminal using echo

The echo command is used to print messages to the terminal.

#!/bin/bash

echo "Hello, World!"

As a result, you will see Hello, World! printed to the terminal.

Redirecting output to a file

You can redirect the output of a script to a file using the > operator.

./sample.sh > output.txt

You can use >> to append the output to a file.

./sample.sh >> output.txt

Data streams and redirection

In Linux, there are three standard data streams used for handling input and output operations in processes.

1. Standard Input (stdin)- File descriptor 0:

This stream is used to receive input data into a program. By default, it takes input from a keyboard but it can be redirected.

For example, the cat command without arguments, waits for input from the keyboard. Once you have entered the text, press Ctrl+D to send an EOF (End of File) signal.

Alternately, you can redirect the input from a file using the < operator.

cat < file.txt

This will display the contents of file.txt on the terminal.

2. Standard Output (stdout) - File descriptor 1:

This stream is used to send the normal output of a program. By default, it displays output on the terminal.

For example, cat with a filename argument will display the contents of the file on the terminal.

cat file.txt

You can also redirect the output(stdout) to a file using the > operator.

cat file.txt > output.txt

With the above command, the contents of file.txt will be written to output.txt.

3. Standard Error (stderr) - File descriptor 2:

This stream is used for displaying error messages. By default, it displays error messages on the terminal, separate from stdout.

For example, if you try to access a non-existent file with cat, you will see an error message on the terminal.

cat non_existent_file.txt

You can redirect that error(stderr) to a file using the 2> operator.

cat non_existent_file.txt 2> error.txt

With the above command, the error message will be written to error.txt.

For efficient troubleshooting, you can redirect both stdout and stderr to separate files.

cat non_existent_file.txt > output.log 2>&1

• output.log redirects stdout to the output.log.

• 2>&1 redirects stderr(2) to the same location as stdout(1).

Scheduling tasks with cron jobs

Now that you know how to write and execute scripts, you can automate certain tasks using cron jobs. Cron is a time-based job scheduler Linux. It allows you to schedule tasks to run at specific times or intervals.

For example, you might schedule a script to run every Wednesday at 3:00 AM to back up your data using cron. Below is the syntax to schedule a cron job:

* * * * * /path/to/script.sh

Order wise, the * symbol represents:

• Minute (0-59)

• Hour (0-23)

• Day of the month (1-31)

• Month (1-12)

• Day of the week (0-7, where 0 and 7 represent Sunday)

A script that runs every Wednesday at 3:00 AM would look like this:

0 3 * * 3 /path/to/script.sh

Using crontab to manage cron jobs

The crontab utility is used to add, remove or list cron jobs.

To list existing cron jobs, run:

crontab -l

Using the text editor of your choice, you can add, remove, or modify cron jobs using the command:

crontab -e

Practical Example- create a housekeeping script

Now it is time to put your knowledge to the test by creating a housekeeping script that performs the following tasks:

• Finds log files(extension .log) older than 7 days.

• Compresses them into a .tar.gz archive.

• Moves the archive to a backup directory.

This is how the script would look:

#!/bin/bash

# Define backup directory and archive name
BACKUP_DIR="/backup"
ARCHIVE_NAME="$BACKUP_DIR/logs_$(date +%Y-%m-%d).tar.gz"

# Ensure the backup directory exists
mkdir -p "$BACKUP_DIR"

# Find and compress log files older than 7 days, then delete them if successful
find /var/log -name "*.log" -mtime +7 -print0 | tar --null -czvf "$ARCHIVE_NAME" --files-from=- && find /var/log -name "*.log" -mtime +7 -delete

echo "Backup completed: $ARCHIVE_NAME"

Conclusion

In this article, you explored how you can use shell scripting to automate tasks and manage system operations efficiently. You can apply the skills learned in this article to real-world scenarios, such as creating housekeeping scripts for system maintenance.

We hope you found this article useful. Consider sharing with others.

Stay tuned for more content related to Linux, DevOps and Networking here on our Linked page.

Also, subscribe to our newsletter below.