Reading Files Line by Line in Node.js

Reading files line by line is a common requirement in many applications, especially when dealing with large files that don’t fit into memory. In this tutorial, we’ll explore how to read files line by line in Node.js using the built-in readline module.

Introduction to Readline

The readline module is a built-in Node.js module that allows you to read input from a readable stream one line at a time. It’s commonly used for reading files, but can also be used with other types of streams, such as stdin.

Creating a Readable Stream

To read a file line by line, we need to create a readable stream using the fs.createReadStream() method. This method returns a ReadStream object that can be passed to the readline.createInterface() method.

const fs = require('fs');
const readline = require('readline');

const fileStream = fs.createReadStream('input.txt');

Creating a Readline Interface

Once we have a readable stream, we can create a Readline interface using the readline.createInterface() method. This method returns a Readline object that emits events for each line read from the stream.

const rl = readline.createInterface({
  input: fileStream,
  crlfDelay: Infinity
});

The crlfDelay option is used to recognize all instances of CR LF (\r\n) in the input as a single line break.

Reading Lines

To read lines from the stream, we can use the for await...of loop or attach event listeners to the Readline object.

// Using for await...of loop
async function processLineByLine() {
  for await (const line of rl) {
    console.log(`Line from file: ${line}`);
  }
}

processLineByLine();

// Attaching event listeners
rl.on('line', (line) => {
  console.log(`Line from file: ${line}`);
});

rl.on('close', () => {
  console.log('All done');
});

Example Use Cases

Here are a few example use cases for reading files line by line in Node.js:

• Reading a large log file and processing each line individually
• Parsing a CSV file and performing actions based on the data
• Reading a configuration file and applying settings to an application

Best Practices

When reading files line by line in Node.js, keep the following best practices in mind:

• Always handle errors that may occur when creating the readable stream or reading from it
• Use the crlfDelay option to recognize all instances of CR LF (\r\n) as a single line break
• Consider using the for await...of loop for asynchronous iteration over the lines

By following these best practices and using the readline module, you can efficiently read files line by line in Node.js and perform actions based on the data.