Introduction

What is a shell?

Linux shell is a command line interpreter program. It is an interface that lets users interact with the operating system by writing commands. Users enter commands and the shell executes them. This must be allowed for infinite turns until the user enters the exit command or closes the shell program by other means.

How does a shell program actually work?

A shell program, at its core, is just an infinite loop of three steps: Read, Parse, and Execute. Let's understand them one by one.

Read: The shell program waits for the user to enter a command. And when the user does enter a command, the shell program reads and captures it using the getline function.

char *lineptr = NULL;    //variable to store the command
size_t n = 10;    //assumed size of the command in bytes. If the buffer is not large enough
if (getline(&lineptr, &n, stdin) == -1)    // to hold the line, getline() resizes it
{
   return NULL;
}

Parse: The captured command is then broken up in smaller units called tokens. This tokenization process is called parsing. These tokens are crucial for the next step, which is ‘Execute’.

    int len = strlen(command);
    char *delimeter = " \t\n\r";
    char *token = strtok(command, delimeter);    // to split the command into tokens

    int capacity = 2;
    char **tokens = (char **)malloc(sizeof(char *) * capacity);
    if (tokens == NULL)
    {
        perror("Failed to allocate memory");
        exit(EXIT_FAILURE);
    }
    int i = 0;

    while (token != NULL)
    {
        if (i >= capacity)
        {
            // Resize the tokens array
            capacity *= 2;
            tokens = realloc(tokens, sizeof(char *) * capacity);
            if (tokens == NULL)
            {
                perror("Failed to reallocate memory");
                exit(EXIT_FAILURE);
            }
        }

        tokens[i] = malloc(strlen(token) + 1);    
        if (tokens[i] == NULL)
        {
            perror("Failed to allocate memory for token");
            exit(EXIT_FAILURE);
        }

        // assigning the token(s) to the tokens array.
        strcpy(tokens[i], token);
        i++;
        token = strtok(NULL, delimeter);
    }

Execute: There can be various kinds of commands users want to execute. Hence, It is not pragmatic to write code for each kind individually in the shell program. We need to create their individual executable programs that we can just run from our code. Because if a new command is to be added, we can do that without touching the shell program code, by just creating a new executable for it. The good thing is, Linux provides us with these executable programs. We use the ‘execvp’ function to run them.

The ‘execvp’ function requires the name of the executable program and the tokens we created in the previous step. It searches for the filepath of the executable and then runs it.

The issue with the ‘execvp’ function is that it replaces the very process it was called in with a totally new one. Which means that the shell program will also be stopped as it is also a part of the current process.

To handle this issue, we need to create a new child process for command execution. So that our parent process remains intact. We use the ‘fork’ function to create the child process. The new child process inherits almost all of the resources from its parent process. In this child process, we then call the ‘execvp’ function using the tokens.

        pid_t pid;
        pid = fork();

        if (pid < 0)    // failed to fork
        {
            perror("Failed! to create a fork!\n");
        }
        else if (pid == 0)    //child process
        {
            int exec_val = execvp(tokens[0], tokens);

            if (exec_val == -1)
            {
                perror("Command Execution Failed!\n");
            }
        }

        waitpid(pid, NULL, 0);

After all three of these steps the command finally gets executed.

Built-in commands

In the previous section we discussed a process to execute user-entered commands. But a few commands such as ‘cd’, ‘exit’, etc can’t be executed this way. These are called built-in commands.

Linux provides us with executable programs except for a few commands like cd, exit, help etc. Let’s understand why with the help of an example of cd.

Let’s understand why these commands are different from others with the example ‘cd’. The ‘cd’ command is used to change directory. We know that almost all resources in the child process are similar to its parent process. Except that the file path contexts are different in both processes. But the ‘cd’ command needs the correct file path context for its execution. Hence it is essential to execute ‘cd’ in the parent process itself. So, using ‘execvp’ is not an option for these built-in commands.

And since these are just a handful of commands, we can handle them individually. For example, to change directory we can use the ‘chdir’ function.

    if (strcmp(args[0], "cd") == 0)
    {
        builtinExecution(args);
        if (args[1] == NULL)
        {
            perror("No filepath defined!");
        }
        else
        {
            if (chdir(args[1]) != 0)
            {
                perror("Failed to change directory!");
            }
        }
    }

Advanced Features

Multiple Commands

There should be a provision to group multiple commands separated by ‘;’ symbol. The grouped commands must be executed sequentially. That means when the current execution in the sequence finishes then only the next execution starts. e.g.

ls -la ; pwd

To implement this, the captured command is split by the delimiter ‘;’. The split function returns an array of smaller units. A for loop is then run on these units to execute them sequentially.

    char *dels = ";";
    int length = 0;

    char **subcommands = tokenize2(command, &length, dels);
    for (int i = 0; i < length; i++)
    {
        // code to execute the command 
    }

Parallel Commands

Here too, the user groups multiple commands into one, similar to the previous feature. Except that the commands get executed parallelly rather than sequentially. Also the commands are separated by ‘&’ symbol.

ls -la & pwd

We need multithreading capability to implement this. For that we can use the thread.h library. It allows us to execute each command in a separate thread. Parallelly.

    void *threadExecution(void *vargp)
    {
        // code to execute the command
    }


    char *dels = "&";
    int length = 0;

    char **subcommands = tokenize2(command, &length, dels);

    pthread_t threads[length]; // array of threads for individual execution

    for (int i = 0; i < length; i++)
    {
        void *com = (void *)subcommands[i];
        pthread_create(&threads[i], NULL, threadExecution, com); //reating new thread
    }

    for (int i = 0; i < length; i++)
    {
        pthread_join(threads[i], NULL);    // joining threads
    }

File & Command Redirection

Redirection is, when the output from the current execution is sent to the next command or file in the sequence. There can be only two types of redirections: command redirection and file redirection. The commands in a sequence are separated by ‘|’ symbol for command redirection and by ‘>’ or ‘>>’ for file redirection. Multiple redirection is allowed only for command redirection.

ps aux | grep 'nginx' >> disk_usage.txt

In case of file redirection, the file gets created if it doesn’t exist already. The content of the file gets appended if the ‘>>’ symbol is used. But it gets overwritten if ‘>’ is used instead.

Since there will be only one file redirection, It is much simpler to first split the sequence by ‘>’ or ‘>>’ delimiters. This returns an array of length two. The command redirection sequence will be at the zeroth index of this array. And the file path for final redirection will be at the first index. But all of the command redirection needs to be handled before the file redirection. So, We further split the command sequence by ‘|’ delimiter. This returns another array of all the commands in the sequence. Then we iterate over those commands to execute them one by one.

    char *arrowDel = ">";
    char *pipeDel = "|";
    int arrowLen = 0;
    char **subcommands = tokenize2(command, &arrowLen, arrowDel);

    int pipeLen = 0;
    char **pipecommands = tokenize2(subcommands[0], &pipeLen, pipeDel);

The key here is to connect the inputs and outputs of the consecutive commands through something called a pipe. Now for the redirection to happen, we need to connect the output of the current execution to the input of the next one. We do this by using something called a pipe. A pipe is simply an array of two file descriptors. First descriptor is used to read inputs and the second one is used to write outputs.

        int pipes[pipeLen - 1][2];
        for (int i = 0; i < pipeLen - 1; i++)
        {
            pipe(pipes[i])
        }

        pid_t pids[pipeLen];

        for (int i = 0; i < pipeLen; i++)
        {

                // code to execute command
        }
        // closing all the opened resources
        for (int i = 0; i < pipeLen - 1; i++)
        {
            close(pipes[i][0]);
            close(pipes[i][1]);
        }
        for (int i = 0; i < pipeLen; i++)
        {
            waitpid(pids[i], NULL, 0);
        }

On the last iteration of the loop, right before the last command redirection, we must connect the stdout of that process to the file mentioned by the user. This ensures the file redirection at the end.

Batch File Execution

Till now we have seen only one way to read commands. That is, the user enters a command and the shell reads it, executes it. And then the user enters another command and the shell reads it, executes it. And so on. But there should be a way for the user to just throw a bunch of commands at once. The shell program does this by reading commands from a file called batch file. The user can just order the program to read commands from a file and then just wait for everything to be executed. The shell does this till the end of the file using the ‘fgets’ function.

./dash file.txt            # dash is the name of this custom shell

    char *filename = filepath;
    FILE *file = fopen(filename, "r");    // opening the file
    char line[256];

    if (file != NULL)
    {
        while (fgets(line, sizeof(line), file))
        {
            // code to execute command
        }
    }

Sigint

It is a signal to terminate the ongoing command execution. It is triggered when the user enters CTRL + C keys. We use the sigaction function from signal.h library to capture the interrupt signal. Once the signal is captured, a function is triggered that stops the ongoing command execution

int main()
{
    struct sigaction sa;
    sa.sa_handler = intHandler;
    sigemptyset(&sa.sa_mask);
    sa.sa_flags = 0;

    if (sigaction(SIGINT, &sa, NULL) == -1)
    {
        perror("sigaction");
        return 1;
    }

    runShell();
    return 0;
}

void runShell()
{
    int EXITED = 0;
    while (EXITED == 0)
        EXITED = readAndExecute();
}

void intHandler(int sig)
{
    (void)sig;
    write(STDOUT_FILENO, "\n", 1);
    runShell();
}

Possible Future feature additions

• Command history: Navigation through the command history using the arrow keys.

• Environment Variables: Storing and retrieving of the environment variables.