📂 Compression • 🔍 Searching • 🔄 Redirection • ✍️ vi Editor

After exploring the basics of the Linux shell in Part I, today we dive deeper into real-world operations that DevOps professionals use every day.

---

📦 File Compression and Archiving

🔢 Viewing File Sizes

du -sk test.img      # Size in KB
du -sh test.img      # Human-readable format
ls -lh test.img      # Long list with size

📂 Archiving Files Using tar

• tar creates archive files (tarballs).

• Useful for grouping and backing up files.

tar -cf test.tar file1 file2
tar -tf test.tar         # View contents
tar -xf test.tar         # Extract contents
tar -zcf test.tar.gz .   # Compress with gzip

---

📉 File Compression Tools

• gzip, bzip2, xz — Compress files with .gz, .bz2, .xz extensions

gzip file.txt
bzip2 file.txt
xz file.txt

gunzip file.txt.gz
bunzip2 file.txt.bz2
unxz file.txt.xz

🧾 Viewing Compressed Files Without Extracting

zcat file.txt.gz
bzcat file.txt.bz2
xzcat file.txt.xz

---

🔍 Searching for Files and Patterns

📁 Locate Files

locate City.txt
sudo updatedb   # Refresh locate DB

🕵️ Find Files by Name

find /home/user -name City.txt

🧬 Pattern Searching with grep

grep "second" sample.txt          # Basic search
grep -i "Capital" sample.txt      # Case-insensitive
grep -v "printed" sample.txt      # Invert match
grep -w "exam" examples.txt       # Whole word match
grep -r "text" /home/user         # Recursive search

🔁 Contextual Search

grep -A1 "Arsenal" file.txt    # 1 line After match
grep -B1 "Chelsea" file.txt    # 1 line Before match
grep -A1 -B1 "Goal" file.txt   # 1 line Around match

---

🔄 Input/Output Redirection

🎯 Redirect STDOUT

echo $SHELL > shell.txt      # Overwrite
echo $SHELL >> shell.txt     # Append

⚠️ Redirect STDERR

cat missing_file 2> error.txt
cat missing_file 2>> error.txt
cat missing_file 2> /dev/null   # Suppress errors

📬 Piping Between Commands

grep Hello sample.txt | less

🧪 Tee Command (Split STDOUT)

echo $SHELL | tee shell.txt
echo "text" | tee -a shell.txt   # Append mode

---

✍️ vi Editor Essentials

🧭 Modes

• Command Mode: Navigation & commands (default)

• Insert Mode: i, I, a, o, etc. to write text

• Last Line Mode: : to save, quit, etc.

💡 vi Commands Cheat Sheet

:w       # Save
:q       # Quit
:wq      # Save and quit
:q!      # Force quit without saving
:set nu  # Show line numbers
dd       # Delete line
yy, p    # Copy & paste
u        # Undo

Tip: Most modern systems use vim even if you run vi.

---

🧪 Lab – Working With Shell Part II

1. Create a compressed tarball:

    tar -cf /home/bob/python.tar /home/bob/reptile/snake/python
    gzip /home/bob/python.tar
   

2. Unzip a .gz file:

    gunzip /home/bob/birds/eagle/eaglet.dat.gz
   

3. Find a lost file:

    sudo find / -name caleston-code
   

4. Store service file path:

    echo /etc/systemd/system/dummy.service > /home/bob/dummy-service
   

5. Search IP inside /etc:

    sudo grep -ir 172.16.238.197 /etc/ > /home/bob/ip
   

6. Create file with text:

    echo "a file in my home directory" > /home/bob/file_wth_data.txt
   

7. Redirect Python script error:

    python3 /home/bob/my_python_test.py 2>/home/bob/py_error.txt
   

8. Copy first line from .gz without extracting:

    zcat /usr/share/man/man1/tail.1.gz | head -1 > /home/bob/pipes
   

---

🧠 Advanced Tips

✅ Combine Commands Smartly

find / -name "*.log" | xargs grep -i error

✅ View Disk Usage Summary

du -ah --max-depth=1 /var | sort -hr | head -10

✅ Record and Replay Terminal Sessions

script session.log      # Start recording
exit                    # Stop recording

✅ Use less on Compressed Logs

zless /var/log/syslog.1.gz

🧠 Advanced Concepts in Shell (Part II)

---

📦 1. Creating Compressed Archives in One Step

Combine tar and compression in one command:

Combine tar and compression in one command:

# Create a .tar.gz file directly
tar -czvf archive.tar.gz folder/

# Create a .tar.bz2 file
tar -cjvf archive.tar.bz2 folder/

# Create a .tar.xz file
tar -cJvf archive.tar.xz folder/

✅ z → gzip, j → bzip2, J → xz

🔄 2. Multi-command Pipelines with Tee for Debugging

# Log command output and process simultaneously
grep "ERROR" app.log | tee errors.txt | wc -l

This finds all lines with "ERROR", writes them to errors.txt, and counts them.

---

🔐 3. Redirect Both STDOUT and STDERR to a File

command > output.log 2>&1

• This merges STDERR (2) with STDOUT (1) and logs all output.

Or in short:

command &> output.log

---

🔍 4. Search in Compressed Logs (without decompressing)

zgrep "pattern" file.log.gz
bzgrep "pattern" file.log.bz2
xzgrep "pattern" file.log.xz

These commands combine grep with zcat, bzcat, or xzcat behind the scenes.

---

🧪 5. Dry Run Before Deleting Files with find

# Check files older than 10 days
find /var/log -type f -mtime +10 -exec ls -lh {} \;

# Delete after confirming
find /var/log -type f -mtime +10 -delete

---

🔂 6. Run and Time Multiple Commands

time (gzip largefile && echo "Done")

This gives the total time taken for all commands inside the parentheses.

---

💾 7. Efficiently Monitor Live File Changes

tail -F /var/log/syslog | grep --line-buffered "ERROR"

• tail -F auto-follows file even after rotation.

• --line-buffered ensures grep outputs in real-time.

---

✍️ 8. Set vi as Default Editor for All CLI Operations

export VISUAL=vi
export EDITOR=vi

Add it to ~/.bashrc or ~/.profile to make it permanent.

---

🔧 9. Switch vi into Read-Only Mode

vi -R file.txt

This is useful for reviewing logs or config files without accidental edits.

---

🎯 10. Use xargs with find/grep for Better Performance

find /var/log -name "*.log" | xargs grep -i "fail"

This runs fewer processes compared to calling grep per file individually.