Shell Scripting Basic to Advanced

📍Introduction

Bash stands for "Bourne Again SHell." It's a command-line interpreter and scripting language that is widely used in Unix-like operating systems. Bash is the default shell on most Linux distributions and macOS systems. It provides a way to interact with your computer's operating system through a command-line interface, executing commands and scripts to perform various tasks.

Shell scripting is the process of writing scripts (sequences of commands) that are executed by a shell, such as Bash. These scripts allow you to automate tasks, perform system administration tasks, and carry out a wide range of operations through the command line. Shell scripting is an essential skill for system administrators, developers, and anyone who works with Unix-like systems.

Here are some key aspects of Bash and shell scripting:

Command Execution: You can use Bash to execute individual commands by typing them directly into the command line. For example, you can run commands to navigate the file system, manipulate files, install software, and more.
Scripting: Bash scripts are files containing a series of commands that can be executed in sequence. These scripts allow you to automate complex tasks and workflows.
Variables: Bash allows you to define and use variables to store data that can be reused throughout your scripts. Variables can store strings, numbers, and other types of data.
Control Structures: Bash supports control structures like if statements, loops (for and while), and case statements. These structures enable you to create logic and conditionally execute commands.
Input and Output: You can read input from users using the read command and display output using the echo command. Redirecting output and input streams allows you to manipulate data and communicate with files and other programs.
Functions: Bash allows you to define and use functions, which are reusable blocks of code. Functions are useful for encapsulating logic and making scripts more modular.
Command-Line Arguments: Bash scripts can accept command-line arguments, which are values passed to the script when it's executed. These arguments can influence the behavior of the script.
Environment Variables: Bash provides access to environment variables, which are system-wide settings and data that can be used in your scripts.
Script Execution: You can execute a Bash script by making it executable (chmod +x) and then running it using ./scriptname.sh.

Shell scripting is a powerful skill for automating routine tasks, managing system configurations, and creating custom tools. It's widely used in various domains, including system administration, software development, DevOps, and more. Learning Bash and shell scripting can significantly improve your efficiency and productivity when working with Unix-like systems.

⭐ Basic

➡️ Bash Script

Create a Script File: Open your terminal and navigate to the directory where you want to create the script file. Use the touch command to create the file:

touch devops_first.sh

Edit the Script: You can use any text editor to edit the script file. For example, if you're using the nano text editor:

nano devops_first.sh

Inside the editor, you can add your script code.

Add Script Code: Here's an example of a simple script that prints "Hello, DevOps!" when executed:

#!/bin/bash

echo "Hello, DevOps!"

Save and Exit the Editor: For nano, press Ctrl + O to save the file and then Enter. To exit, press Ctrl + X.

Run the Script: To run the script, use the bash command followed by the script's filename:

bash devops_first.sh

This will execute the script and display "Hello, DevOps!" in the terminal.

Make the Script Executable: If you want to make the script executable, you need to change its file permissions using the chmod command:

chmod +x devops_first.sh

This command adds the execute permission to the file.

Run the Executable Script: Now you can run the script directly without using the bash command:

./devops_first.sh

This will also execute the script and display "Hello, DevOps!"

➡️ Variable and User Input

Script with the variable definition, user input section, and the corresponding outputs for both the variable and user input:

#!/bin/bash

# VARIABLE
name="chekit"

echo "hello ${name}, please enter your age"  # {} - Placeholder

# User Input
read age

echo "my age is ${age}"

# Output for Variable
# Output: hello chekit, please enter your age

# Output for User Input
# Sample input: 25
# Output: my age is 25

Explanation:

The script begins with the #!/bin/bash shebang line.
name="chekit" assigns the value "chekit" to the variable name.
echo "hello ${name}, please enter your age" outputs "hello chekit, please enter your age".
read age prompts the user to input their age and stores it in the variable age.
echo "my age is ${age}" displays the entered age as "my age is 25" (assuming the entered age is 25).

When you run the script, you'll see both the output for the variable and the user input sections.

➡️ User Arguments

script with the user argument section and the corresponding output:

#!/bin/bash

# User Argument
echo "chekit: Hello, I am a $1"

# Output
# Command: ./devops_first.sh DevOps_Engineer
# Output: chekit: Hello, I am DevOps_Engineer

Explanation of the added section:

echo "chekit: Hello, I am $1": This line uses the value of the first command-line argument provided to the script (in this case, "DevOps_Engineer") to generate the output message "chekit: Hello, I am DevOps Engineer".
The comments provide context for the command and its corresponding output.

When you run the script with the command ./devops_first.sh DevOps_Engineer, it will display the output message "chekit: Hello, I am chekit".

➡️ If-Else

Here's the script section for the if-else statement:

#!/bin/bash

# IF-ELSE
if [ "$1" = "like" ]; then
    echo "Hey Please $1 this Blog"
else
    echo "Okay, then please follow"
fi

# Output for Command: ./devops_first.sh like
# Output: Hey Please like this Blog

# Output for Command: ./devops_first.sh 
# Output: Okay, then please follow

Explanation of the if-else statement:

if [ "$1" = "like" ]: This line checks if the first command-line argument ($1) is equal to the string "like". The [ ... ] is used for conditional testing, and the = operator is used for comparison.
then: This keyword marks the beginning of the code block to be executed if the condition is true.
echo "Hey Please $1 this Blog": If the condition is true (i.e., the first argument is "like"), this line will be executed, and it will output "Hey Please like this Blog".
else: This keyword marks the beginning of the code block to be executed if the condition is false.
echo "Okay, then please follow": If the condition is false (i.e., the first argument is not "like"), this line will be executed, and it will output "Okay, then please follow".
fi: This keyword marks the end of the if-else block.

When you run the script with different command-line arguments (e.g., ./devops_first.sh like or ./devops_first.sh follow), it will execute the appropriate branch of the if-else statement and provide the corresponding output.

➡️ Elif

Script that utilizes elif for conditional checks and the corresponding outputs:

#!/bin/bash

a=1100
b=200
c=300

if [[ $a -gt $b && $a -gt $c ]]; then
    echo "A is Biggest"
elif [[ $b -gt $a && $b -gt $c ]]; then
    echo "B is greatest"
else
    echo "C is biggest"
fi

Explanation:

The script compares the values of variables a, b, and c to find the greatest one.
[[ $a -gt $b && $a -gt $c ]] checks if a is greater than both b and c. If true, it outputs "A is Biggest".
[[ $b -gt $a && $b -gt $c ]] checks if b is greater than both a and c. If true, it outputs "B is greatest".
If none of the conditions above are met, it means that neither a nor b are the greatest, so it outputs "C is biggest".

Here's the output when you run the script:

# Output: A is Biggest

This is because a (1100) is greater than both b (200) and c (300).

⭐⭐ Advanced

➡️ For Loop

Script that utilizes a for loop to print numbers from 1 to 10, along with the corresponding output:

#!/bin/bash

# Using a for loop to print numbers from 1 to 10
for ((i=1; i<=10; i++))
do
    echo "$i"
done

Explanation:

The script uses a for loop to iterate through the numbers from 1 to 10.
((i=1; i<=10; i++)) sets up the loop with an initialization (i=1), a condition (i<=10), and an increment (i++).
Inside the loop, echo "$i" prints the value of i.

Here's the output when you run the script:

# Output:
# 1
# 2
# 3
# 4
# 5
# 6
# 7
# 8
# 9
# 10

Example for loop to iterate through files and display their names, along with the corresponding output:

#!/bin/bash

# Creating files named file1.txt to file10.txt
touch file{1..10}.txt

# Using a for loop to iterate through files and display their names
for FILE in *.txt
do
    echo $FILE
done

Explanation:

touch file{1..10}.txt creates 10 files named file1.txt to file10.txt.
The for loop iterates through the files that match the pattern *.txt.
for FILE in *.txt assigns each file's name to the variable FILE in each iteration of the loop.
echo $FILE prints the value of the FILE variable, which is the name of each file.

Here's the output when you run the script:

# Output:
# file1.txt
# file10.txt
# file2.txt
# file3.txt
# file4.txt
# file5.txt
# file6.txt
# file7.txt
# file8.txt
# file9.txt

➡️While Loop

Here's an example of a while loop in a shell script:

#!/bin/bash

count=1

while [ $count -le 5 ]; do
    echo "Iteration $count"
    ((count++))
done

Explanation:

The script uses a while loop to iterate from 1 to 5.
The condition [ $count -le 5 ] checks if the value of the variable count is less than or equal to 5.
Inside the loop, echo "Iteration $count" prints the current iteration number.
((count++)) increments the value of the count variable by 1 in each iteration.

Here's the output when you run the script:

Iteration 1
Iteration 2
Iteration 3
Iteration 4
Iteration 5

The script uses the while loop to iterate through the numbers and displays the iteration number in the output for each iteration. The loop continues until the condition becomes false (when count exceeds 5).

➡️ Functions

Script that defines a function to add a user to your system, along with the corresponding usage example:

#!/bin/bash

add_user()
{
    USER=$1
    PASS=$2

    useradd -m -p $PASS $USER && echo "Successfully added user"
}

# MAIN

add_user ChekitS test123

Explanation:

The script defines a function named add_user that takes two parameters: $1 (for the username) and $2 (for the password).
Inside the function, the useradd command is used to create a new user with the provided username and password. The -m flag creates the user's home directory, and -p sets the password. If the user creation is successful, it echoes "Successfully added user".
In the MAIN section, the add_user function is called with the arguments "ChekitS" as the username and "test123" as the password.

Here's the output when you run the script:

# Output:
# Successfully added user

The script calls the add_user function with the provided arguments and outputs "Successfully added user" if the user creation is successful.

⭐⭐⭐ Real World Scripts

➡️ System Backup🔄

Script for taking system backups along with the corresponding output:

#!/bin/bash

src_dir=/home/ec2-user/shell_scripts
tgt_dir=/home/ec2-user/backups

curr_timestamp=$(date "+%Y-%m-%d-%H-%M-%S")
backup_file=$tgt_dir/$curr_timestamp.tgz

echo "Taking backup on $curr_timestamp"

tar czf $backup_file --absolute-names $src_dir

echo "Backup complete"

# Saving and exiting the editor

# Running the backup script
bash backup.s

Explanation:

The script takes a backup of the directory located at src_dir and compresses it into a .tgz file in the tgt_dir.
The curr_timestamp variable holds the current date and time in the format YYYY-MM-DD-HH-MM-SS.
The backup_file variable stores the path to the backup file, including the timestamp in the filename.
The tar command creates a compressed archive of the source directory. The czf flags indicate creating a gzip-compressed archive (c), specifying the archive filename (f), and adding files to the archive in compressed form (z). The --absolute-names flag ensures that absolute paths are preserved during archiving.

Here's the expected output when you run the script:

Taking backup on 2023-08-21-12-52-09
Backup complete

This output indicates that the script is taking a backup and completes the backup process successfully.

Keep in mind that proper permissions and configurations are important for the directories and files involved in the backup process.

➡️ Disk Usage💽

Script named check_disk.sh that incorporates the df -H command along with the awk command, along with the expected output:

#!/bin/bash

# Display filesystem usage using df and awk
df -H | awk '{print $1 " " $5}'

Expected Output (sample output):

Filesystem Usage
/dev/sda1 18%
tmpfs 0%
/dev/sdb1 12%
/dev/sdc1 9%

Explanation:

The script starts with a shebang line #!/bin/bash to specify the interpreter.
df -H is a command to display disk space usage. The -H flag is used to show the sizes in a human-readable format.
awk '{print $1 " " $5}' processes the output of the df -H command. It uses awk to print the first column (filesystem) and the fifth column (usage percentage) of the output.

➡️ Disk Usage with 'cut' command💽

Script that uses the cut command along with df -H and awk, along with the corresponding output:

#!/bin/bash

# Display usage percentage using cut, df, and awk
df -H | awk '{print $5 " " $1}' | while read output;
do
    usage=$(echo $output | awk '{print $1}' | cut -d'%' -f1)
    echo $usage
done

Expected Output (sample output):

Explanation:

The script starts with a shebang line #!/bin/bash to specify the interpreter.
df -H is a command to display disk space usage. The -H flag is used to show the sizes in a human-readable format.
awk '{print $5 " " $1}' processes the output of the df -H command. It uses awk to print the fifth column (usage percentage) and the first column (filesystem) of the output.
The output is read line by line using the while read output loop.
usage=$(echo $output | awk '{print $1}' | cut -d'%' -f1) extracts the usage percentage from the output. It uses awk to print the first column, and then cut is used to remove the '%' character and keep the numeric part.
echo $usage prints the extracted usage percentage.

The script will execute the commands and display the usage percentages extracted from the df -H command's output. The output will look similar to the sample output provided above.

➡️ Disk Usage Check for Critical Condition⚠️

Script that checks disk usage and outputs a message for filesystems with usage above 90%, along with the corresponding output:

#!/bin/bash

# Disk Check Script
df -H | awk '{print $5 " " $1}' | while read output;
do
    usage=$(echo $output | awk '{print $1}' | cut -d'%' -f1)
    file_sys=$(echo $output | awk '{print $2}')
    if [ $usage -ge 90 ]; then
        echo "CRITICAL for $file_sys"
    fi
done

Expected Output (sample output):

CRITICAL for /dev/sdb1

Explanation:

The script starts with a shebang line #!/bin/bash to specify the interpreter.
df -H is a command to display disk space usage. The -H flag is used to show the sizes in a human-readable format.
awk '{print $5 " " $1}' processes the output of the df -H command. It uses awk to print the fifth column (usage percentage) and the first column (filesystem) of the output.
The output is read line by line using the while read output loop.
usage=$(echo $output | awk '{print $1}' | cut -d'%' -f1) extracts the usage percentage from the output. It uses awk to print the first column, and then cut is used to remove the '%' character and keep the numeric part.
file_sys=$(echo $output | awk '{print $2}') extracts the filesystem name from the output using awk.
The if [ $usage -ge 90 ] condition checks if the usage percentage is greater than or equal to 90.
If the condition is true, the script outputs a "CRITICAL" message for the filesystem with usage above 90%.

The script will execute the commands, check disk usage, and display a "CRITICAL" message for filesystems with usage above 90%. The output will look similar to the sample output provided above.

⏰ Cron and Crontab

💠Cron

Cron is a time-based job scheduler in Unix-like operating systems. It allows users to schedule tasks (commands or scripts) that need to be executed at specific intervals, repeatedly or only once. Cron is particularly useful for automating routine tasks, backups, system maintenance, and other repetitive actions.

The cron daemon (cron for short) is a background process that continually monitors the system's cron tables and executes scheduled tasks when their specified time conditions are met. It ensures that tasks are executed reliably and consistently according to the schedule.

💠Crontab

Crontab is short for "cron table," and it refers to a configuration file that contains a list of cron job entries. Each entry defines a scheduled task along with the timing specification. The crontab command is used to manage and edit these cron tables.

💠Crontab Syntax

A crontab entry consists of six fields, each representing a time unit and separated by spaces:

[Minute] [Hour] [Day of Month] [Month] [Day of Week] [Command]

Minute: The minute within the hour (0-59).
Hour: The hour of the day (0-23).
Day of Month: The day of the month (1-31).
Month: The month (1-12) or textual month names (e.g., Jan, Feb).
Day of Week: The day of the week (0-6, where 0 is Sunday) or textual day names (e.g., Sun, Mon).
Command: The command or script to be executed.

💠Crontab Commands

crontab -l: List all cron jobs in the current user's crontab.
crontab -e: Edit the current user's crontab using the default text editor.
crontab -r: Remove (delete) the current user's crontab.

💠Examples

Here are a few examples of crontab entries:

Run a script every day at 3:30 AM:
```
 30 3 * * * /path/to/script.sh
```
Perform a backup every Sunday at midnight:
```
 0 0 * * 0 /path/to/backup-script.sh
```
Clean temporary files every day at 2:00 AM:
```
 0 2 * * * /path/to/cleanup-script.sh
```

💠Important Notes

Cron jobs are executed with the user's environment variables, so ensure that commands/scripts have appropriate paths set.
Be cautious when scheduling tasks, especially frequent ones, to avoid overloading the system.
Logging the output of cron jobs can help troubleshoot issues; consider redirecting output to a log file.

📍Conclusion

In this comprehensive exploration, we've progressed from shell scripting fundamentals to advanced techniques, crafting real-world scripts for diverse tasks. Our mastery extends to cron and crontab, enabling precise task scheduling. Equipped with this robust skill set, we're empowered to automate tasks efficiently, from backups to maintenance, in the dynamic world of shell scripting.

Chekit's Blog

Chekit's Blog

📜Shell Scripting Basic to Advanced

Table of contents

📍Introduction

⭐ Basic

➡️ Bash Script

➡️ Variable and User Input

➡️ User Arguments

➡️ If-Else

➡️ Elif

⭐⭐ Advanced

➡️ For Loop

➡️While Loop

➡️ Functions

⭐⭐⭐ Real World Scripts

➡️ System Backup🔄

➡️ Disk Usage💽

➡️ Disk Usage with 'cut' command💽

➡️ Disk Usage Check for Critical Condition⚠️

⏰ Cron and Crontab

📍Conclusion