How to use Pipes "|" in GNU/Linux (and programming)"

Brief History of pipe in Unix

The pipe was introduced around the year 1973. It was used to glue together small commands so as to produce an efficient command to do a task. Instead of creating a monolithic command that was only good in one task, you get this powerful way of making custom commands. This proved quite effective considering the storage back then, you could not afford to put the results of one in a file before starting another command. The pipe uses the Inter-process communication under the hood. When the pipe is used, two processes are created, one on the left and on the right. The output of one process (usually the left one) becomes the input of the other one (usually the right one). This ensures that there is no need for extra space for storage the output of the commands before the process is complete. You can read more about pipes (or pipeline) here

Pipes In programming

Pipes are widely used in computer science for different purposes. It is used in most programming languages to show the logical or for boolean operations. Here is an example in some of the popular programming languages.

An example in C

# include <stdio.h>

int main (void)
{
     /* Initialize the three variables */
    int a = 5, b = 9, c = 14;

    /* Compare them to each other */
    if (a > b || c > b)
        printf("The value of a or c is greater than b");
    else
        printf("The value of b is greater thab a and c");
}

Another usage is bitwise operation. Here is an example in python

# Define two binary numbers
>>> num1 = 0b1010
>>> num2 = 0b1100

# Perform bitwise OR operation using the | operator
>>> result = num1 | num2

# Print the result in binary form
>>> print(f"Result of {bin(num1)} | {bin(num2)} = {bin(result)}")

# Result of 0b1010 | 0b1100 = 0b1110

Note that IPC can be used in programming too. If interested you can start here

Pipes in GNU/Linux

In GNU/Linux and other *nix operating systems, they use pipes almost the same way, to glue together commands. So here is an example of a command that uses a pipe

ls ~/Documents/ | grep "hi"

In this example, we list the files and directories in Documents using the ls. The results we have got from ls is passed to grep for further processing. grep looks at the results and searches for the characters hi, when it encounters it, it passes the results to the stdout which in this case is the screen.

Here are other examples of pipe where it is used in a sorting algorithm

cat list | grep "zs" | sort | uniq

Here we look at a file called list, the result is sent to grep which filters out words without zs and sends the rest to uniq remove repeated text, the the results are sorted and sent to the stdout.

There is no limit on how many pipes you can chain together. Feel free to play with them. The pipe is not limited only to GNU/Linux, it can be used in other *nix OSes like MacOS and can even be used in Windows!

It's worth noting that pipes can be a powerful tool, but they can also be a source of performance problems if used improperly. For example, chaining together many commands with pipes can lead to a lot of context switching and overhead, which can slow down the system. In general, it's a good idea to use pipes judiciously and to test command sequences to make sure they're performing as expected.