Unit6 - Subjective Questions

Apache Cassandra is an open-source, highly scalable, distributed NoSQL database designed to handle large amounts of structured and semi-structured data across many commodity servers, providing high availability with no single point of failure.

Core Features:

• Distributed and Decentralized: Every node in the cluster is identical. There is no master-slave architecture, meaning no single point of failure.
• Highly Scalable: Allows for seamless scaling horizontally by simply adding more hardware without downtime.
• Fault Tolerant: Data is automatically replicated to multiple nodes. If a node goes down, data can still be served from replicas.
• Tunable Consistency: Allows developers to choose between strong consistency and high availability based on the application's needs.
• High Performance: Provides lightning-fast write speeds and can handle extremely high volumes of concurrent operations.

Cassandra is designed with a peer-to-peer distributed architecture, represented as a token ring. It avoids the traditional master-slave model to eliminate single points of failure.

Key Components of Cassandra Architecture:

• Node: The basic infrastructure component where data is stored.
• Data Center: A collection of related nodes configured together. It can be a physical or virtual data center.
• Cluster: A component that contains one or more data centers spanning across multiple physical locations.
• Commit Log: A crash-recovery mechanism. Every write operation is written here first.
• Memtable: A memory-resident data structure. Data is written to the memtable after being written to the commit log.
• SSTable (Sorted String Table): A disk file to which the memory-resident data is flushed when the memtable is full.
• Bloom Filter: Quick, non-deterministic algorithms testing whether an element is a member of a set. Used to check if an SSTable has data for a particular row.

Cassandra's data model is based on a wide-column store concept, though it is often queried using CQL (Cassandra Query Language) which resembles SQL.

Hierarchy of the Data Model:

• Keyspace: The outermost container for data in Cassandra (similar to a database in an RDBMS). It contains replication settings.
• Column Family / Table: A container for an ordered collection of rows. Each row is an ordered collection of columns (similar to a table in RDBMS).
• Row: A collection of columns identified by a primary key.
• Column: The basic data construct, consisting of a name, a value, and a timestamp.

Types of Columns:

• Partition Key: Determines which node stores the data.
• Clustering Columns: Determines how the data is sorted within the partition.

Cassandra vs. RDBMS:

• Architecture: RDBMS uses a Master-Slave architecture, whereas Cassandra uses a Peer-to-Peer architecture (no single point of failure).
• Data Model: RDBMS uses a structured, table-based schema. Cassandra uses a wide-column store or partitioned row store which is flexible.
• Scalability: RDBMS relies heavily on vertical scaling (adding more power to a single server). Cassandra excels in horizontal scaling (adding more commodity servers to the cluster).
• Query Language: RDBMS uses SQL with support for complex joins, subqueries, and aggregations. Cassandra uses CQL (Cassandra Query Language) which does not support JOINs or complex aggregations to ensure performance.
• ACID vs BASE: RDBMS complies strictly with ACID properties (Atomicity, Consistency, Isolation, Durability). Cassandra follows the BASE property (Basically Available, Soft state, Eventual consistency), offering tunable consistency.

The Gossip Protocol is a peer-to-peer communication mechanism used by nodes in a Cassandra cluster to discover and share state and location information about themselves and other nodes.

How it works:

• Every second, each node communicates with one to three other nodes and exchanges information about the state of the cluster.
• Nodes quickly learn about all other nodes in the cluster through this continuous, localized exchange of information.
• This protocol operates in time, where is the number of nodes, meaning information propagates very rapidly across the entire cluster.
• It helps in failure detection by keeping track of which nodes are up, down, or re-joining the cluster.

Cassandra is optimized for extremely fast write operations. The write path avoids random disk seeks.

Steps in the Write Process:

1. Commit Log: When a write request is received, Cassandra immediately writes the data to the Commit Log on disk. This ensures durability; if the node crashes, data can be recovered.
2. Memtable: Simultaneously, the data is written to a memory-resident structure called the Memtable. Once written here, the write operation is considered successful and an acknowledgment is sent back to the client.
3. Flushing: When the Memtable reaches a pre-configured threshold, its contents are flushed sequentially to disk into an immutable file called an SSTable (Sorted String Table).
4. Compaction: Over time, multiple SSTables are created. Cassandra periodically runs a compaction process in the background to merge these SSTables and discard overwritten or deleted data.

Reads in Cassandra are more complex than writes because data for a single row may be spread across a Memtable and multiple SSTables.

Steps in the Read Process:

1. Memtable Check: Cassandra first checks the Memtable. If all required data is found, it is returned.
2. Row Cache / Key Cache: If not entirely in the Memtable, it checks the Row Cache (if enabled) and the Key Cache to find the exact location of the partition key in the SSTable.
3. Bloom Filter: Cassandra checks the Bloom Filter associated with each SSTable. The Bloom filter quickly determines if an SSTable might contain the requested row, preventing unnecessary disk I/O.
4. Partition Summary & Index: If the Bloom Filter indicates a match, Cassandra checks the Partition Summary and then the Partition Index to locate the exact offset on disk.
5. SSTable Read & Merge: Data is retrieved from the relevant SSTables on disk. The results from the Memtable and SSTables are merged, with the latest timestamp taking precedence, and returned to the client.

Memtable:
A Memtable is an in-memory data structure in Cassandra. When a write occurs, data is stored in the Memtable. It holds data temporarily before it is flushed to disk. Data in a Memtable is sorted by key.

SSTable (Sorted String Table):
An SSTable is an immutable data file on the disk. Once a Memtable reaches a certain size threshold, its contents are flushed to an SSTable. Because SSTables are immutable, a write operation never overwrites an existing SSTable.

Interaction:
Writes go to the Memtable. Upon reaching capacity, the Memtable is flushed to create a new SSTable on disk. During a read operation, Cassandra merges data from the current Memtable and multiple SSTables to return the most up-to-date row to the user.

The Commit Log is an append-only log file on disk used by Cassandra to ensure data durability and for crash recovery.

Role and Importance:

• Durability: Before a write is applied to the in-memory Memtable, it is written to the Commit Log. This guarantees that if a node crashes or loses power before the Memtable is flushed to an SSTable, the data is not lost.
• Fast Writes: Because writing to the Commit Log is an append-only operation, it does not require random disk seeks. This sequential I/O ensures writes are extremely fast.
• Recovery: Upon node restart, Cassandra replays the Commit Log to restore the state of the Memtables to exactly what it was before the crash.

Compaction is the background process in Cassandra that merges multiple SSTables into a single, larger SSTable.

Why it is necessary:

• SSTable Immutability: Because SSTables are immutable, updates and deletes are simply appended as new records (or tombstones for deletes). Over time, a single row's data can be fragmented across many SSTables.
• Read Performance: Without compaction, a read operation would have to scan an ever-increasing number of SSTables, severely degrading read performance.
• Disk Space Reclaim: Compaction merges data, retains only the latest version of a record based on timestamps, purges deleted data (tombstones) once their grace period expires, and reclaims disk space.

Cassandra offers different compaction strategies, such as SizeTieredCompactionStrategy (STCS) for write-heavy workloads and LeveledCompactionStrategy (LCS) for read-heavy workloads.

In Cassandra, data is not deleted immediately from the disk because SSTables are immutable. Instead, Cassandra uses a mechanism called Tombstones.

How they work:

• When a delete command is issued, Cassandra writes a special marker called a tombstone. This marker includes the deleted column/row and a timestamp.
• During a read operation, if Cassandra encounters a tombstone with a newer timestamp than the actual data, it knows the data has been deleted and suppresses it from the results.
• Tombstones are kept in the system for a configurable period called gc_grace_seconds (default is 10 days). This grace period allows the tombstone to be propagated to all replicas in the cluster, even if some nodes are temporarily down.
• During compaction, if a tombstone is older than the gc_grace_seconds, the tombstone and the underlying data are finally purged from the disk.

The CAP theorem states that a distributed data store can only guarantee two out of the three following properties: Consistency (C), Availability (A), and Partition Tolerance (P).

Application to Cassandra:

• Cassandra is fundamentally designed as an AP (Available and Partition Tolerant) system.
• Partition Tolerance: It easily handles network partitions. If nodes cannot communicate, they continue to operate independently.
• Availability: Replicas allow the system to remain highly available. The system will continue to accept reads and writes even if some nodes are down.
• Eventual Consistency: Instead of strict Consistency, Cassandra offers Eventual Consistency. Over time, all replicas will converge to the same value.
• Tunable Consistency: Cassandra allows developers to configure the desired level of consistency per query (e.g., ONE, QUORUM, ALL), effectively allowing it to act more like a CP system when strongly required, at the cost of some availability.

Replication Factor (RF):
The Replication Factor determines the total number of replicas (copies) of data stored across the cluster. An RF of 3 means there are three copies of each row stored on three different nodes.

Replica Placement Strategies:
Cassandra uses strategies to determine exactly which nodes receive the replicated data.

1. SimpleStrategy: Used for a single data center. It places the first replica on the node determined by the partitioner. Additional replicas are placed on the next consecutive nodes clockwise along the token ring.
2. NetworkTopologyStrategy: Used for multiple data centers. It allows you to specify a different replication factor for each data center. It attempts to place replicas on different racks within a data center to maximize availability in case a whole rack fails.

Consistency Level:
Consistency Level (CL) in Cassandra specifies how many replicas must acknowledge a read or write operation before it is considered successful by the client.

Tunable Consistency:
Cassandra allows developers to adjust the consistency level on a per-query basis, trading off between consistency and latency/availability. This is called Tunable Consistency.

Common Consistency Levels:

• ANY: (Write only) Highly available. A write is successful if written to the commit log of at least one node, or even just stored as a hint if no replicas are reachable.
• ONE: Acknowledgment from only one replica is required. Very fast, but low consistency.
• QUORUM: A majority of replicas () must acknowledge. Provides strong consistency when used for both reads and writes.
• ALL: All replicas must acknowledge. Highest consistency but lowest availability (fails if even one replica is down).

To achieve strong consistency, the condition must hold true (where is read CL, is write CL, and is Replication Factor).

Primary Key:
The Primary Key uniquely identifies a row in a Cassandra table. It can be made up of one or more columns. It is divided into two parts: the Partition Key and the Clustering Key.

Partition Key:

• It is the first part of the Primary Key.
• It dictates exactly which node (or nodes, depending on RF) will store the data.
• Cassandra applies a hash function to the partition key to determine the token, which decides the node placement.

Clustering Key (or Clustering Column):

• It is the second part of the Primary Key (optional).
• It determines how the data is sorted and stored within the partition on the physical disk.
• It allows for efficient range queries within a specific partition.

Example: PRIMARY KEY ((user_id), created_at) -> user_id is the partition key, created_at is the clustering key.

A Snitch in Cassandra determines which data center and rack a node belongs to. It informs Cassandra about the network topology, ensuring that requests are routed efficiently and that replicas are distributed properly (e.g., across different racks to prevent failure if a single switch goes down).

Types of Snitches:

1. SimpleSnitch: The default. Does not recognize data centers or racks. Only suitable for single-datacenter deployments.
2. GossipingPropertyFileSnitch: The recommended snitch for production. It uses a local configuration file to define the node's rack and datacenter, and gossips this information to the rest of the cluster.
3. Ec2Snitch / Ec2MultiRegionSnitch: Designed for AWS deployments, extracting region and availability zone information directly from the EC2 API.
4. RackInferringSnitch: Infers rack and datacenter from IP addresses.

A Bloom filter is a space-efficient, probabilistic data structure used by Cassandra during the read process to test whether an element is a member of a set.

Usage in Cassandra:
Since SSTables are immutable, data for a row might be distributed across many SSTables. Searching every SSTable on disk would require expensive I/O operations. To avoid this, Cassandra maintains an in-memory Bloom filter for each SSTable.

Efficiency and Behavior:

• When a read request arrives, Cassandra checks the Bloom filter first.
• If the Bloom filter returns "Definitely Not", Cassandra knows the data is not in that SSTable and skips it entirely, saving a disk read.
• If the Bloom filter returns "Probably Yes", Cassandra will read from the SSTable. False positives are possible, meaning it might occasionally read an SSTable only to find the data isn't there, but it guarantees no false negatives.
• The false positive chance is configurable, balancing memory usage against I/O reduction.

CQL stands for Cassandra Query Language. It is the primary interface for communicating with Cassandra. Its syntax is very similar to SQL (Structured Query Language), making it easier for developers transitioning from relational databases, though it lacks operations like JOINs to enforce performance scalability.

Examples:

1. Creating a Keyspace:
sql
CREATE KEYSPACE UserData
WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 3};

2. Creating a Table:
sql
USE UserData;
CREATE TABLE users (
user_id uuid,
username text,
email text,
created_at timestamp,
PRIMARY KEY (user_id)
);

Apache Cassandra is optimized for write-heavy workloads, massive scalability, and continuous availability. Its primary use cases include:

• IoT and Time-Series Data: Excellent for ingesting massive amounts of telemetry, sensor data, or log data at high speeds.
• User Activity Tracking: Storing browsing history, shopping cart data, and clickstreams for high-traffic e-commerce or media websites.
• Messaging Systems: Serving as the backend for messaging apps and email providers due to its ability to handle immense write throughput.
• Personalization and Recommendations: Providing real-time personalization by serving user preference data quickly.
• Fraud Detection: In the financial sector, handling high-velocity transaction streams to analyze and detect fraudulent activity in real-time.

These are mechanisms Cassandra uses to ensure eventual consistency across replicas.

Hinted Handoff:

• If a node responsible for receiving a write is temporarily down, the coordinator node will store the write locally as a "hint".
• This hint contains the data and the target node's ID.
• When the coordinator detects via Gossip that the target node is back online, it hands off the hinted data to the node, restoring consistency quickly. It is an availability feature.

Anti-Entropy Repair (Manual Repair):

• Hinted handoff is for short-term outages. If a node is down for an extended period, hints are dropped.
• Anti-entropy repair is a background/manual process (using nodetool repair) that compares data across replicas to catch inconsistencies.
• It uses Merkle Trees (cryptographic hash trees) to efficiently compare data between nodes over the network. If a mismatch is found, the out-of-date replica is updated with the latest data.