Unit 1: Getting Started with Node.JS & Handling Data I/O in Node.js

1. Introducing Node.js

What is Node.js?

Node.js is an open-source, cross-platform JavaScript runtime environment that executes JavaScript code outside a web browser. It is built on Chrome's V8 JavaScript engine.

Key Characteristics

• Asynchronous and Event-Driven: All APIs of the Node.js library are asynchronous (non-blocking). The server never waits for an API to return data.
• Single-Threaded but Scalable: Node.js uses a single-threaded model with event looping. This allows it to handle a vast number of concurrent connections with high throughput, contrasting with traditional servers that create limited threads to handle requests.
• No Buffering: Node.js applications output data in chunks rather than buffering the whole data.
• High Performance: Being built on the V8 engine (written in C++), code execution is extremely fast.

Architecture: The Event Loop

Node.js uses "Libuv" to handle asynchronous operations. The Event Loop is the mechanism that allows Node.js to perform non-blocking I/O operations by offloading operations to the system kernel whenever possible.

2. Installing Node.js

Prerequisites

• Basic knowledge of JavaScript.
• Command Line Interface (CLI) access (Terminal/Command Prompt).

Installation Steps

1. Download: Visit the official website (nodejs.org).
2. Select Version:
   • LTS (Long Term Support): Recommended for most users and enterprise applications (Stability focus).
   • Current: Contains the latest features (Experimental focus).
3. Install: Run the installer package for your OS (Windows, macOS, Linux).
4. Verify Installation: Open a terminal and type:
   
   BASH

       node -v
       npm -v
       

3. Using Node.js Read Evaluate Print Loop (REPL)

REPL is a built-in interactive shell to process Node.js expressions. It is useful for testing simple JavaScript code and debugging.

The Four Components

1. Read: Reads user input, parses it into JavaScript data-structure, and stores it in memory.
2. Evaluate: Takes and evaluates the data structure.
3. Print: Prints the result.
4. Loop: Loops the above command until the user exits (Ctrl+C twice).

Using REPL

To start REPL, type node in your terminal.

$ node
> 10 + 20
30
> console.log("Hello REPL")
Hello REPL
undefined
> .help

Common REPL Commands

• .help: Lists all dot commands.
• .break: Exits the current multi-line expression.
• .editor: Enter editor mode.
• .exit: Close the I/O stream and exit REPL.
• .save filename: Saves the current REPL session to a file.
• .load filename: Loads a file into the current REPL session.

4. Node Package Manager (npm)

NPM is the default package manager for Node.js. It consists of two parts:

1. A CLI (Command Line Interface) tool for publishing and downloading packages.
2. An online repository that hosts JavaScript packages.

Key Functions

• Managing libraries/dependencies for a project.
• Running scripts (start, test, build).
• Versioning control using Semantic Versioning (SemVer).

5. Initializing a Node.js Project (npm init)

To create a Node.js project, you must create a package.json file. This file acts as the manifest for the project.

Commands

• Interactive Mode: Prompts the user for details (name, version, description, entry point, etc.).
  
  BASH

      npm init
      

• Default Mode (Flag -y): Skips the questionnaire and creates a default package.json.
  
  BASH

      npm init -y
      

Structure of package.json

{
  "name": "my-app",
  "version": "1.0.0",
  "main": "index.js",
  "scripts": {
    "start": "node index.js"
  },
  "dependencies": {
    "express": "^4.17.1"
  }
}

6. NPM Modules

In Node.js, a module is a block of code (functions, objects, variables) encapsulated in a file.

A. Core Modules

Built-in modules that come with Node.js installation. No installation is required, only importing.

• http: To create a server.
• fs: To handle the file system.
• path: To handle file paths.
• os: Operating system information.

Example:

const http = require('http'); // No ./ required

B. Local Modules

Modules created by the developer within the application.

1. Create module (math.js):
   
   JAVASCRIPT

       const add = (a, b) => a + b;
       module.exports = { add };
       

2. Import module (index.js):
   
   JAVASCRIPT

       const math = require('./math'); // Requires relative path
       console.log(math.add(2, 3));
       

C. Third-Party Modules

Modules developed by the community and available via NPM.

1. Install: npm install <package-name> (e.g., npm install lodash).
2. Storage: Stored in the node_modules folder.
3. Import:
   
   JAVASCRIPT

       const _ = require('lodash');
       

7. EventEmitter in Node.js

Node.js core API is based on an asynchronous event-driven architecture. Objects (Emitters) emit named events which cause Function objects (Listeners) to be called.

The events Module

const EventEmitter = require('events');
const myEmitter = new EventEmitter();

// 1. Create an Event Listener
myEmitter.on('greet', (name) => {
  console.log(`Hello, ${name}!`);
});

// 2. Emit the Event
myEmitter.emit('greet', 'Alice');

Key Methods

• on(event, listener): Adds a listener to the end of the listeners array for the specified event.
• emit(event, [args]): Synchronously calls each of the listeners registered for the event.
• once(event, listener): Adds a one-time listener.
• removeListener(event, listener): Removes a specific listener.

8. Callbacks in Node.js

A callback is a function passed as an argument to another function, which is then invoked inside the outer function to complete some kind of routine or action. This is the primary way Node.js handled asynchronous data before Promises.

Synchronous vs Asynchronous

• Blocking (Sync): const data = fs.readFileSync('file.txt');
• Non-Blocking (Async Callback):
  
  JAVASCRIPT

      const fs = require('fs');
  
      // The arrow function is the callback
      fs.readFile('input.txt', 'utf8', (err, data) => {
          if (err) return console.error(err);
          console.log(data);
      });
      console.log("Program Ended"); // This prints BEFORE the file data
      

Error-First Callback Pattern

Node.js callbacks usually take the error object as the first parameter. If error is null, the operation was successful.

9. Working with the fs (File System) Module

The fs module enables interacting with the file system.

Common Operations

1. Reading Files

const fs = require('fs');

// Asynchronous read
fs.readFile('demo.txt', 'utf8', (err, data) => {
    if (err) throw err;
    console.log(data);
});

2. Writing Files

Overwrites the file content or creates a new file if it doesn't exist.

fs.writeFile('demo.txt', 'Hello Node', (err) => {
    if (err) throw err;
    console.log('File Saved!');
});

3. Appending Files

Adds data to the end of the file.

fs.appendFile('demo.txt', '\nNew Line', (err) => {
    if (err) throw err;
    console.log('Data appended!');
});

4. Deleting Files

fs.unlink('demo.txt', (err) => {
    if (err) throw err;
    console.log('File deleted!');
});

10. Working with JSON

JSON (JavaScript Object Notation) is the standard for data exchange in Node.js.

Handling JSON Data

1. JSON.stringify(): Converts a JavaScript object into a JSON string (for sending/writing).
2. JSON.parse(): Converts a JSON string into a JavaScript object (for reading/using).

Reading/Writing JSON Files

const fs = require('fs');

const user = {
    name: "John",
    age: 30
};

// Writing JSON
const jsonData = JSON.stringify(user);
fs.writeFile('user.json', jsonData, (err) => {
    if(!err) console.log("JSON saved");
});

// Reading JSON
fs.readFile('user.json', 'utf8', (err, data) => {
    if(!err) {
        const userObj = JSON.parse(data);
        console.log(userObj.name); // Output: John
    }
});

11. Using Stream Module to Stream Data

Streams are objects that let you read data from a source or write data to a destination in continuous chunks. This is more memory-efficient than reading whole files into memory.

Types of Streams

1. Readable: Used for reading operations (e.g., fs.createReadStream).
2. Writable: Used for writing operations (e.g., fs.createWriteStream).
3. Duplex: Can be used for both reading and writing (e.g., Network sockets).
4. Transform: A type of duplex stream where the output is computed based on input (e.g., zlib).

Piping Streams

The pipe() method attaches a readable stream to a writable stream.

const fs = require('fs');

// Create a readable stream
const readerStream = fs.createReadStream('input.txt');

// Create a writable stream
const writerStream = fs.createWriteStream('output.txt');

// Pipe the read and write operations
readerStream.pipe(writerStream);

console.log("File copy initiated via streams");

Stream Events

Streams emit events like data, end, error, and finish.

readerStream.on('data', (chunk) => {
   console.log("Received chunk:", chunk);
});

12. Compressing and Decompressing Data with Zlib

The zlib module provides compression functionality (Gzip, Deflate, etc.). It is often used with Streams.

Compressing a File (Gzip)

const fs = require('fs');
const zlib = require('zlib');

const gzip = zlib.createGzip();
const input = fs.createReadStream('input.txt');
const output = fs.createWriteStream('input.txt.gz');

// Pipeline: Read -> Compress -> Write
input.pipe(gzip).pipe(output);

Decompressing a File (Gunzip)

const fs = require('fs');
const zlib = require('zlib');

const gunzip = zlib.createGunzip();
const input = fs.createReadStream('input.txt.gz');
const output = fs.createWriteStream('input_restored.txt');

// Pipeline: Read Compressed -> Decompress -> Write
input.pipe(gunzip).pipe(output);

13. Promises and Async/Await

To solve the "Callback Hell" (nested callbacks becoming unreadable), Node.js utilizes Promises and the modern async/await syntax.

Promises

A Promise represents the eventual completion (or failure) of an asynchronous operation and its resulting value. It has three states: Pending, Resolved (Fulfilled), and Rejected.

const myPromise = new Promise((resolve, reject) => {
    const success = true;
    if (success) {
        resolve("Operation Successful");
    } else {
        reject("Operation Failed");
    }
});

myPromise
    .then(result => console.log(result))
    .catch(error => console.error(error));

Async/Await

async and await are syntactic sugar built on top of Promises, making asynchronous code look and behave like synchronous code.

• async: Declares a function that automatically returns a Promise.
• await: Pauses the execution of the async function until the Promise is resolved.

Using fs.promises (Node.js v10+):

const fs = require('fs').promises;

async function readFileData() {
    try {
        const data = await fs.readFile('input.txt', 'utf8');
        console.log("File content:", data);
        
        // We can await another operation sequentially
        await fs.writeFile('output.txt', data);
        console.log("File written");
        
    } catch (error) {
        console.error("Error reading file:", error);
    }
}

readFileData();