Learn the Concept of Streams in Node.js

In the last article, we discussed the concept of buffers in Node.js. In this article, we are going to discuss the concept of streams in Node.Js and the various operations on the streams along with the suitable examples.

Streams in Node.js
In Node.js, streams are the objects which help the developer read data from the file or write date to file present at a particular location on machine in a regular fashion. There are four such types of streams in Node.js as follows:

• Readable – The readable stream is used for the read operation.
• Writable – The writable stream is used for write operation.
• Duplex – The Duplex stream can be used to execute both read and write operations.
• Transform – The transform stream is a type of duplex stream where we can compute an output based on an input.

Here is the interesting part about the above streams. All of the above streams are EventEmitter instances which can throw several events at different instance at the time of their execution. The following are some of such commonly used events.

• ‘data’ – The ‘data’ event is fired when there is data available to read from the file.
• ‘end’ – The ‘end’ event is fired when system has finished reading the data from the file or when there is no data to read from the file.
• ‘error’ – The ‘error’ event is fired whenever there is any error while receiving or writing data to the file.
• ‘finish’ – The ‘finish’ event is fired when all the data from the file has been flushed successfully to the underlying system.

Let’s understand each of the above events in detail.

Reading from a Stream
In this case, we are going to read data from the file ‘inputFile.txt’ which is present at a particular location on the machine. This file has the following data to read.

Welcome to Streams in Node.js

Now, create a Node.js project in eclipse with the name as ‘StreamsNodeJSApplication’ and add a js file with the name as ‘file-stream-reader.js’.

var fileSystem = require("fs");
var fileData = '';

// Creation of a readable stream.
var fileReaderStream = fileSystem.createReadStream('C:\\odesk\\Abhishek Thakur\\NodeJS\\StreamsNodeJSApplication\\inputFile.txt');

// Setting the encoding to the utf8 format. 
fileReaderStream.setEncoding('UTF8');

// Handling the stream events for data, end and error.
fileReaderStream.on('data', function(dataChunk) {
	fileData += dataChunk;
});

fileReaderStream.on('end',function(){
   console.log(fileData);
});

fileReaderStream.on('error', function(error){
   console.log(error.stack);
});

console.log ("Program execution has ended successfully!");

Explanation of the code

• Firstly, we are importing the required module by using the required directive “fs” in order to load the “fs” module and storing the returned “fs” instance into a fileSystem variable as shown above.
• Next, we are creating a fileReaderStream instance by using ‘fileSystem.createReadStream (‘file path’)’ where the instance is returned into fileReaderStream variable as shown above.
• Next, we have set the encoding of data as ‘UTF8’.
• Next, we are using ‘fileReaderStream’ instance for handling the reading stream events for data, end and error where we are taking some actions as shown above.
• Lastly, we are logging a predefined message ‘Program execution has ended successfully!’ on to the console.

Output
When we execute the code present in the ‘file-stream-reader.js’ file by using the node command, we can observe the following output.

Writing to a Stream
To write to a stream into a file ‘’,we are going to use the same Node.js project ‘StreamsNodeJSApplication’ and add a js file with the name as ‘file-stream-writer.js’. The following are the contents of this file.

var fileSystem = require("fs");
var fileData = 'Node.js is an open-source, cross-platform JavaScript runtime environment for developing a diverse variety of tools and applications.';

// Creation of a readable stream
var fileWriterStream = fileSystem.createWriteStream('C:\\odesk\\Abhishek Thakur\\NodeJS\\StreamsNodeJSApplication\\outputFile.txt');

// Writing data to the stream with encoding to be utf8
fileWriterStream.write(fileData,'UTF8');

// Marking the end of file
fileWriterStream.end();

// Handling the stream events i.e. finish and error events
fileWriterStream.on('finish', function() {
    console.log ("The write operation on the file has been completed.");
});

fileWriterStream.on('error', function(error){
   console.log(error.stack);
});

console.log ("Program execution has ended successfully!");

Explanation of the code

• Firstly, we are importing the required module by using the required directive “fs” in order to load the “fs” module and storing the returned “fs” instance into a fileSystem variable as shown above.
• Next, we are creating a fileWriterStream instance by using ‘fileSystem.createWriteStream (‘file path’)’ where the instance is returned into fileWriterStream variable as shown above.
• Next, we have set the encoding of data as ‘UTF8’ followed by marking the end of the file.
• Next, we are using ‘fileWriterStream’ instance for handling the writing stream events for finish and error where we are taking some actions as shown above.
• Lastly, we are logging a predefined message ‘Program execution has ended successfully!’ on to the console.

Output
When we execute the code present in the ‘file-stream-writer.js’ file by using the node command, we can observe the following output.

Also, we can check ‘fileOutput.txt’ file contents, which we have written using the above program as shown below.

Node.js is an open-source, cross-platform JavaScript runtime environment for developing a diverse variety of tools and applications.

Piping the Streams
Piping can be defined as a mechanism through which we can write data to a stream that can act as an input to the other stream. Therefore, we can pass the data from one to the other stream. There is no limit on the number of such piping operations. The following is the demonstration for the piping mechanism.

Let’s create a js file with the name as ‘piping.js’ file.

var fileSystem = require("fs");

// Creation of a readable stream
var fileReaderStream = fileSystem.createReadStream('inputData.txt');

// Creation of a writable stream
var fileWriterStream = fileSystem.createWriteStream('outputData.txt');

// Here we are Piping the read and write operations
fileReaderStream.pipe(fileWriterStream);

console.log ("Program execution has ended successfully!");

Explanation of the code

• Firstly, we are importing the required module by using the required directive “fs” in order to load the “fs” module and storing the returned “fs” instance into a fileSystem variable as shown above.
• Next, we are creating a fileReaderStream instance by using ‘fileSystem.createReadStream (‘file path’)’ where the instance is returned into fileReaderStream variable as shown above.
• Next, we are creating a fileWriterStream instance by using ‘fileSystem.createWriteStream (‘file path’)’ where the instance is returned into fileWriterStream variable as shown above.
• Next, we are piping both the streams using this method ‘fileReaderStream.pipe (fileWriterStream)’. As a result, whatever data is written on the ‘inputData.txt’ will be copied as it is on the ‘outputData.txt’ file. This mechanism is known as piping.
• Lastly, we are logging a predefined message ‘Program execution has ended successfully!’ on to the console.

Output
When we execute the code present in the ‘piping.js’ file by using the node command, we can observe the following output.

Chaining the Streams
Chaining can be defined as a mechanism through which we can connect the output of one stream to another stream and further create a chain of such multiple stream operations. It is generally used with piping operations. The following is the demonstration example for the piping mechanism.

Let’s create a js file with the name as ‘chaining.js’ file.

var fileSystem = require("fs");

var zlib = require('zlib');

//Chaining the stream to generate a compressed file out of inputData.txt file date.
fileSystem.createReadStream('inputData.txt').pipe(zlib.createGzip()).pipe(fileSystem.createWriteStream('inputData.txt.gz'));
// Creation of a readable stream

console.log("File has compressed successfully!");

Explanation of the code

• Firstly, we are importing the required module by using the required directive “fs” in order to load the “fs” module and storing the returned “fs” instance into a fileSystem variable as shown above.
• Next, we are creating a fileReaderStream instance by using ‘fileSystem.createReadStream (‘file path’)’ where the instance is returned into fileReaderStream variable which we have piped into a chain to ‘pipe(zlib.createGzip ())’ followed by ‘pipe(fileSystem.createWriteStream()’ to generate a compressed file.
• Lastly, we are logging a predefined message ‘File has compressed successfully!’ on to the console.

Output
When we execute the code present in the ‘chaining.js’ file by using the node command, we can observe the following output.

Source code for Streams of NodeJS Application

Conclusion: –
In this article, we have discussed about the reader and the writer streams of Node.js along with their frequently emitted events which are emitted by these streams. Later, we have used these streams to demonstrate piping and chaining mechanisms in Node.js.