Unit 2: Introduction to Programming in Scala

I. Scala Orientation

Scala is a statically typed, general-purpose programming language created by Martin Odersky and first released publicly in 2004. Its name means “scalable language”: the same language supports concise scripts, object-oriented applications, functional programs, concurrent systems, and distributed frameworks such as Apache Spark.

• Governing principle: Scala combines object-oriented programming, where values and operations are represented through objects, with functional programming, where computation is expressed through functions and immutable data.
• Platform: Scala commonly compiles to Java Virtual Machine (JVM) bytecode, enabling it to use Java libraries such as java.time and frameworks such as Hadoop.
• Static typing: Types are checked during compilation; for example, assigning "cluster" to an Int variable causes a compile-time error.
• Concise syntax: Type inference, higher-order functions, and expression-oriented control structures reduce boilerplate without removing type safety.
• Expression orientation: Constructs such as if, match, and blocks return values; therefore, they can appear on the right side of an assignment.
• Programming convention: Immutable values declared with val are generally preferred over mutable variables declared with var, especially in concurrent and distributed programs.
• Entry point: A traditional executable program defines a main method or extends App; newer Scala versions also support @main.

object ClusterApp {
  def main(args: Array[String]): Unit = {
    println("Scala on the JVM")
  }
}

II. Scala Language Foundations — Syntax, Values, and Types

Scala’s language foundations determine how programs represent values, perform operations, and establish types before collection processing or distributed computation begins.

A. Features of Scala

Scala is designed to express complex programs through interoperable, composable, and type-safe abstractions.

• Object-oriented structure: Every value is treated as an object; even 5 supports methods such as 5.toDouble.
• Functional programming: Functions can be stored in variables, passed as arguments, and returned from other functions.
• JVM interoperability: Scala can instantiate Java classes directly, as in new java.util.ArrayList[String]().
• Strong static type system: The compiler prevents incompatible operations, such as adding an Int directly to a Boolean.
• Pattern matching: match selects behavior by value, type, or structure and is more expressive than a simple switch.

val status = 200
val message = status match {
  case 200 => "Success"
  case 404 => "Not found"
  case _   => "Other"
}

• Traits: A trait defines reusable fields and methods and can be mixed into multiple classes, avoiding the limitations of single implementation inheritance.
• Case classes: case class Node(id: Int, host: String) automatically provides useful methods such as equals, hashCode, toString, and copy.
• Concurrency support: Immutable data and functional transformations reduce shared-state problems in parallel programs.
• Scalability: Operators, classes, functions, and domain-specific APIs allow small expressions such as data.map(_ * 2) to scale into larger systems.

B. Basic data types and literals used in Scala

Scala provides value types for numbers, characters, and logical values, together with reference types such as strings and collections.

• Integer types:
  • Byte: 8-bit signed integer, from −128 to 127.
  • Short: 16-bit signed integer.
  • Int: 32-bit signed integer; 42 is an Int literal by default.
  • Long: 64-bit signed integer; the suffix L appears in 5000000000L.
• Floating-point types:
  • Float: 32-bit IEEE 754 value; 3.5F requires the F suffix.
  • Double: 64-bit IEEE 754 value; 3.5 is a Double by default.
• Character and text types: Char uses single quotes, as in 'S', while String uses double quotes, as in "Scala".
• Boolean type: Boolean has only the literals true and false.
• Unit type: Unit represents the absence of a meaningful result; methods returning it use the value ().
• Special types: Any is the root of Scala’s type hierarchy, while Nothing is the subtype of every type and represents computations that never return normally.
• Literal forms: Hexadecimal integers use 0x, such as 0xFF; strings may contain escapes such as "\n" or use triple quotes for multiline text.
• Interpolation: Prefixing a string with s inserts expressions marked by $.

val nodes: Int = 8
val load: Double = 0.75
val active: Boolean = true
val label = s"$nodes nodes at $load load"

C. Operators and methods used in Scala

Scala operators are method calls written in operator notation, so a + b is generally interpreted as a.+(b).

• Arithmetic operators: +, -, *, /, and % perform arithmetic; 7 / 2 produces 3 because both operands are integers.
• Relational operators: <, <=, >, and >= return a Boolean, as in load >= 0.80.
• Equality operators: == and != perform null-safe value comparison; eq and ne test reference identity for reference types.
• Logical operators: &&, ||, and ! implement AND, OR, and NOT; && and || short-circuit.
• Assignment-related operators: x += 1 updates a mutable variable and corresponds conceptually to x = x + 1.
• Bitwise operators: &, |, ^, and ~ operate on integral bit patterns; shifts use <<, >>, and >>>.
• Method notation: A method with one parameter may use infix notation, so 1 to 5 means 1.to(5).
• Operator precedence: Precedence is based primarily on the first character of the method name; multiplication binds more tightly than addition in 2 + 3 * 4.
• Common methods: "scala".toUpperCase, 42.toString, and List(1, 2).contains(2) show that operations are invoked on values.

val total = 10 + 5       // 10.+(5)
val range = 1 to 4       // 1.to(4)
val valid = total > 10 && range.contains(3)

D. Introduction to type inference

Type inference allows the compiler to determine a value’s type from its expression while preserving compile-time checking.

• Local inference: In val count = 12, the literal causes count to be inferred as Int.
• Collection inference: List(1, 2, 3) becomes List[Int], while List(1, 2.5) receives a common numeric supertype.
• Function inference: The expected context may determine parameter types, as in List(1, 2).map(x => x * 2), where x is inferred as Int.
• Explicit annotations: A declaration such as val rate: Double = 2 documents intent and widens the integer literal to 2.0.
• Method boundaries: Public method return types are often written explicitly for clarity, although the compiler can infer many non-recursive method results.
• Parameters: Named method parameters require declared types: def square(x: Int) = x * x.
• Limitations: Inference does not mean dynamic typing; once val size = 4 is inferred as Int, it cannot later hold "four".

val workers = 16                    // Int
val names = List("n1", "n2")        // List[String]
val doubled = workers * 2           // Int

def utilization(used: Double, total: Double): Double =
  used / total

III. Functions and Computation — Reusable Program Behavior

Scala functions encapsulate transformations and can be composed or supplied to collection operations, making them central to data-parallel programming.

A. Functions in Scala

A function maps input values to an output value and may be defined as a named method, anonymous function, or function-valued variable.

• Named methods: def introduces a method with parameter lists and an optional explicit return type.

def add(a: Int, b: Int): Int = a + b

• Anonymous functions: (x: Int) => x * x is a function literal with one Int parameter and an Int result.
• Function types: Int => Int denotes a one-argument function; (Int, Int) => Int denotes a two-argument function.
• Higher-order functions: A function is higher-order when it accepts or returns another function. Collection methods such as map, filter, and reduce use this principle.
• Shorthand syntax: Placeholder notation replaces simple parameters; numbers.map(_ * 2) is equivalent to numbers.map(x => x * 2).
• Multiple parameter lists: def multiply(a: Int)(b: Int) = a * b supports partial application and currying.
• Default and named arguments: def connect(host: String, port: Int = 7077) permits connect(host = "master").
• Recursion: A recursive function calls itself and should have a termination condition; @annotation.tailrec verifies tail-recursive optimization.
• Closures: A function may capture a value from its surrounding scope, such as val factor = 3; val scale = (x: Int) => x * factor.
• Collection example: A transformation pipeline selects even values and squares them.

val data = List(1, 2, 3, 4, 5)
val result = data.filter(_ % 2 == 0).map(x => x * x)
// List(4, 16)

IV. Scala Collections — Storage and Data Transformation

Scala collections organize groups of values through common operations such as map, filter, fold, and iteration, while differing in mutability, ordering, lookup behavior, and evaluation strategy.

A. Mutable vs. immutable collections

Immutable collections return new collections after an operation, whereas mutable collections may alter their existing internal state.

1. Immutable collections:

   • Default availability: List, Set, and Map usually refer to scala.collection.immutable implementations.
   • Update behavior: val ys = xs :+ 4 creates a new sequence; xs remains unchanged.
   • Concurrency advantage: Unchanging values can be safely shared among tasks without locks for collection updates.
   • Structural sharing: Implementations may reuse unchanged internal nodes, so a new collection does not necessarily copy every element.

2. Mutable collections:

   • Explicit import: Types commonly come from scala.collection.mutable, such as ArrayBuffer and HashMap.
   • Update behavior: Operations including +=, -=, and update modify the collection.
   • Use case: Repeated local accumulation may be efficient when mutation is controlled and not shared.
   • Risk: Concurrent updates to the same mutable collection may produce races or require synchronization.

val fixed = List(1, 2)
val extended = fixed :+ 3

val changing = scala.collection.mutable.ArrayBuffer(1, 2)
changing += 3

B. Lists in Scala

A Scala List is an immutable, ordered, singly linked sequence optimized for access and insertion at its beginning.

• Construction: List(10, 20, 30) creates a List[Int]; Nil represents the empty list.
• Cons operator: 5 :: List(10, 20) prepends 5, producing List(5, 10, 20).
• Core components: head returns the first element and tail returns the remaining list, but both require a non-empty list.
• Safe inspection: headOption returns Some(value) or None, avoiding an exception on Nil.
• Traversal operations: map, filter, flatMap, and foldLeft process elements without modifying the original list.
• Performance: Prepending with :: is constant time, while indexed access such as items(500) is linear because links must be traversed.
• Pattern matching: A list can be decomposed through head :: tail and Nil.

val nodes = List("n1", "n2", "n3")
val first = nodes.headOption
val upper = nodes.map(_.toUpperCase)

C. Maps in Scala

A Map stores key-value associations in which each key identifies at most one value.

• Construction: Map("master" -> 7077, "worker" -> 8081) creates string keys associated with integer ports.
• Lookup: ports.get("master") returns Option[Int], making a missing key explicit through None.
• Direct access: ports("master") returns the value but throws an exception if the key is absent.
• Safe defaults: ports.getOrElse("history", 18080) supplies 18080 when "history" is not present.
• Immutable update: ports + ("history" -> 18080) returns a new map; adding an existing key replaces its associated value in the result.
• Removal: ports - "worker" creates a map without that key.
• Traversal: map, foreach, and pattern matching can process key-value pairs.

val loads = Map("node1" -> 0.70, "node2" -> 0.85)
val busy = loads.filter { case (_, load) => load >= 0.80 }
// Map("node2" -> 0.85)

D. Streams in Scala

A stream is an immutable sequence whose elements are evaluated lazily, allowing potentially large or infinite data sequences to be represented.

• Lazy evaluation: Elements are computed only when demanded by operations such as take, head, or conversion to a strict collection.
• Infinite sequence: Stream.from(1) represents all positive integers without constructing them simultaneously.
• Construction: In Scala 2, #:: prepends an element lazily and Stream.empty terminates a finite stream.
• Memoization: Once evaluated, stream elements are generally retained, which avoids recomputation but may consume memory if references to the beginning remain.
• Finite extraction: Stream.from(1).map(_ * 2).take(4).toList produces List(2, 4, 6, 8).
• Modern replacement: Scala 2.13 deprecated Stream in favor of LazyList, which provides clearer lazy-tail behavior and similar operations.
• Cluster relevance: Language-level streams model lazy in-memory sequences; they are distinct from distributed streaming systems such as Spark Structured Streaming.

val naturals = LazyList.from(1)
val multiplesOfFive = naturals.map(_ * 5)
val firstFour = multiplesOfFive.take(4).toList
// List(5, 10, 15, 20)