The sed Command in Linux

The sed command in Linux stands for Stream Editor. It is a powerful utility that parses and transforms text, using a simple, compact programming language. sed is particularly useful for editing large files or streams without opening them in a traditional text editor. It operates line by line, applies operations specified by the user, and outputs the result to the standard output.

How sed Works

The workflow of sed (Stream Editor) involves reading, executing commands, and displaying the output, operating in a cycle over the input stream.

Here's a step-by-step explanation of how sed works:

1. Reading: sed reads one line at a time from the input stream. This can be a file or input piped from another command.

2. Placing: Each line read into sed is placed into a space called the "pattern space," where the operations are performed. The end-of-line newline character is temporarily removed during this step.

3. Executing Commands: Once a line is in the pattern space, sed executes all the commands you've specified in the order they were given. These commands can modify the contents of the pattern space (like replacing text, deleting the line, etc.), and they can be conditioned to only run if the line matches a specified pattern. sed commands are powerful and can include:

   • Substitution (e.g., s/pattern/replacement/): Search for a pattern and replace it with the specified text.

   • Deletion (e.g., d): Delete lines that match a specified pattern or line number.

   • Insertion and Appending (e.g., i\text, a\text): Insert or append text before or after a matching pattern or line number.

   • Reading and Writing (e.g., r filename, w filename): Read content from a file into the pattern space, or write content from the pattern space to a file.

   • Line Addressing: Apply commands to specific lines or ranges of lines, either specified by line numbers or patterns.

4. Printing: After executing all commands on the line in the pattern space, sed by default prints the contents of the pattern space to the standard output. It can be redirected to a file or piped to another command for further processing.This behavior can be modified with options like -n, which suppresses automatic printing unless explicitly told to print (e.g., with the p command).

5. Cycle Continues: The pattern space is then cleared, and sed reads the next line of input, continuing the cycle until all lines from the input have been processed.

Linux sed Syntax

The main syntax for using Linux sed command is:

sed [OPTIONS]... [SCRIPT] [INPUTFILE...]

• [OPTIONS]: These are command-line options that modify how sed operates. Some of the most commonly used options include:

  • -e script: Allows you to specify a script containing one or more sed commands. This option is useful when chaining multiple commands.

  • -i: Enables in-place editing of files. When used, sed modifies the input file directly.

  • -n: Suppresses automatic printing of the pattern space. Typically used with the p command to print specific lines.

• [script]: This is where you specify the sed commands to be executed. The script usually contains one or more commands that tell sed how to process the input text.

• [input-file]: This is the source of input data for sed to process. It can be one or more filenames. If no input file is specified, sed reads from the standard input, which allows it to be used in pipelines with other commands.

Hands-on Exercise Overview

This hands-on exercise shows ten commonly used sed commands and a foxinbox.txt file with the following content was taken as a sample file:

Hands-on Exercise

1. To replace text, use the substitute command s and delimiters (in most cases, slashes - /) for separating text fields.

    sed 's/old_string/new_string/' filename.txt
   

   For example, to replace instances of box with the word bin, run:

   

   💡

   It does not matter what is used as the delimiter. Use any other delimiter instead of the forward slash ( / ) if the forward slash must be searched for.

2. To replace multiple instances of the same word within a single line, add the g flag to the command to change all of them:

    sed 's/old_string/new_string/g' filename.txt
   

   For example, to replace the word box with the word bin in the file foxinbox.txt every time, type:

   

   💡

   By default, sed only replaces the first occurrence of the specified string in each line. It searches for the first instance of the specified word in a line, replaces it, and moves on to the next line.

3. To replace a specific occurrence of the string in a line, add a number flag such as 1, 2 etc:

    sed 's/old_string/new_string/#' filename.txt
   

   For example, to substitute the second occurrence of the word box in each line of the file with the word bin, use this command:

   

4. To print out just the lines that have substitution under the given conditions, use the syntax:

    sed -n 's/old_string/new_string/p' filename.txt
   

   • -n option disables automatic printing.

   • -p instructs sed to print lines where substitution occurs.

For example, to replace the second instance of the word box in a line and print the line where the change took place:

5. To ignore case while substituting text, add the i subcommand at the end of the command:

    sed 's/old_string/new_string/i' filename.txt
   

   For example, the command for changing upper and lowercase instances of the word fox in the foxinbox.txt file is:

   

6. To substitute a string in a specific line, add the line number as a prefix to the s subcommand:

    sed '# s/old_string/new_string/' filename.txt
   

   For example, to replace the word socks with sandals only in the fourth line (4) of the text use the command:

   

7. To replace multiple instances of a string within a line range, but not the entire text, specify the range where you want sed to substitute. The syntax is:

    sed '#,# s/old_string/new_string/' filename.txt
   

   Replace the first # with the initial line number and the second # with the last line number you want to include.

   For instance, to replace the last two instances of the word socks in the file foxinbox.txt (located in the fourth and sixth line) with the word sandals, run:

   

8. To delete a line from a file with the sed command, use the d subcommand and the syntax:

    sed '#d' filename.txt
   

   Specify the line number you want to remove instead of the hash (#) symbol and run the command.

   For instance, to remove the second line from the foxinbox.txt file, type:

   

9. To use sed to delete lines within a line range, follow the syntax:

    sed '#,#d' filename.txt
   

   Replace the hash symbols with the beginning and end of the line range.

   For example, to delete lines 2, 3, and 4 from the foxinbox.txt file, run the command:

   

10. To delete empty lines (that contain no characters), run:

    sed '/^$/d' filename.txt
    

    The command uses a regular expression pattern ^$ to match empty lines. The ^ represents the start of a line, and $ represents the end of a line.

    For instance, to remove empty lines from the foxinbox.txt, run this command:

    

References:

1. Linux Crash Course - The sed Command

2. sed - wikipedia

3. sed, a stream editor - gnu

4. sed Command by IBM

5. Linux sed Command: How To Use the Stream Editor by phoenixNAP

6. How to Use sed Command to Delete a Line by phoenixNAP

7. How to Use Sed to Find and Replace a String in a File by phoenixNAP