The default input method for UNIX and Linux commands is the standard input (keyboard). Outputs from these commands are directed by default to the standard output (screen). In this article, we will learn how to redirect commands’ inputs and outputs to something other than those defaults. Wish you nice reading.
In Piping, the output from a command is passed as the input to another command.
COMMAND1 | COMMAND2 [| COMMAND3 …]
Where the output from COMMAND1 is redirected (instead of being sent to the screen) to be the input of COMMAND2.
Piping is mainly used for processing the output from a command line tool by one or more other tools. Common uses of piping are:
- Counting the files and/or directories in a directory (in general, or those matching specific pattern).
- Sorting output from the first command.
- Suppressing (removing) duplicates in the output.
- Extracting the first n lines of the output (using the head command), or the last n lines (using the tail command).
- Using the output to be the body of an email.
- Sending the output to a printer.
- Identifying the lines of output that match a search pattern (using the grep command).
- Extract specific fields (columns) of a formatted output (using cut, sed, or awk).
- To count the files in the current directory, use the command:
ls | wc –l
- To list the most recent 15 files in a directory:
- To send the filesystems utilization info by email to the admin:
df –h | mail –s "Filesystems Utilization" root
Before talking about I/O Redirection, we first need to understand the concept of standard Input stream STDIN, standard output STDOUT, and the standard error STDERR. Together, the three streams constitute the channels through which information flow into and out from a program.
The Standard Input STDIN: this is where commands take their inputs. The standard input defaults to the computer keyboard.
The Standard Output STDOUT: this channel is where the successful command output goes. The default destination for STDOUT is the screen.
The Standard Error STDERR: this is where error messages go. Like the STDOUT, the default destination for STDERR is the terminal screen.
Output and Error Redirection
A command output and any resulting error messages may be redirected to files.
NoteSo far, we have learned two methods to create files: using the touch command, and using the vi/vim editing tool. The redirection of STDOUT and STDERR presents a third way to create files.
To redirect the command output to a file (instead of the terminal), creating a new file if the file doesn’t exist, or overwriting it (if it already exists):
COMMAND > FILENAME
To redirect the command output to a file (instead of the terminal), creating a new file, or appending to an existing one:
COMMAND >> FILENAME
To redirect the standard error to a file, creating a new file if none exists, or overwriting it (if it already exists):
COMMAND 2> FILENAME
To redirect the standard error to a file (instead of the terminal), creating a new file, or appending to an existing one:
COMMAND 2>> FILENAME
To redirect both standard output and standard error to the same file:
COMMAND &> FILENAME
Starting with the following ls command:
- The following redirects the output only to the file outfile:
- The following redirects the standard error only to a file errfile:
- To redirect both to the same file:
Instead of the taking input from the keyboard, the standard input STDIN can be redirected to accept input from a file.
Input redirection is rarely done (compared to STDOUT and STDERR redirections). Input redirection is used mainly to form email messages with the mail and sendmail commands.
Consider having to send a notification email with high priority to the administrator when a specific event occurs. Importance and priority are specified in the mail header:
To: [email protected] Subject: CRITICAL: Database is DOWN Content-Type: text/plain X-Priority: 1 (Highest) X-MSMail-Priority: High CRITICAL: The database is DOWN
To send such email, we need to put the above mail header and body in a file, and redirect the standard input to the sendmail command to take input from the file instead of the keyboard:
sendmail -t < /home/admin/critical_mail
The tee Command
Now, what if we need to make both things: display a command output on the terminal, and redirect that output to a file (may be for further processing, analysis or something)?
Is this doable?!
Another case: Can we write the output to a file while piping it to another command?
Can we do this or that ? And how?!
The tee command has the answer.
COMMAND | tee FILE
COMMAND1 | tee FILE | COMMAND2
- The following illustrates how to write the output of the ls command to the file /tmp/ls.out while displaying the output on the terminal:
- The following line stores the output of the ls command in a file and pipes the output as an input to the command wc -l
In this article, we have learned the I/O Redirection and Piping.
- In Piping, the output from a command is taken to be the input to another command.
- The standard output and standard error redirection means writing the output, error messages, or both to a file instead of the default destination (terminal screen).
- Input redirection means accepting input from a file instead of the default (keyboard).
- The tee command copies the standard input or (the output from a command) to a file and also to the standard output.
I hope you find this article useful. See you in the next article.