Unit 6: Network Security

I. Foundations of Network Security

Network security protects data, communicating entities, and network services against unauthorized access, modification, disclosure, and disruption. Its design commonly follows the OSI security architecture defined by ITU-T X.800, which distinguishes security attacks, services, and mechanisms.

• Confidentiality: Prevents unauthorized disclosure of information, usually through encryption such as AES.
• Integrity: Detects unauthorized modification, insertion, deletion, or replay of data through hashes, MACs, or digital signatures.
• Authentication: Establishes the identity of an entity or confirms the source of a message.
• Authorization: Determines which resources an authenticated entity may access.
• Non-repudiation: Provides evidence that prevents a sender or receiver from falsely denying an action.
• Availability: Keeps systems and services accessible despite failures or attacks such as denial of service.
• Layered security: Places controls at application, transport, and network layers; each layer protects a different communication scope.
• Cryptographic basis:
  • Symmetric encryption uses one shared secret key and is efficient for bulk data.
  • Asymmetric encryption uses public-private key pairs for signatures, authentication, and key establishment.
  • Hash functions produce fixed-length digests without requiring a key.

II. Application and Email Security

Application-layer security protects particular services and can apply controls according to the meaning and structure of application data.

A. Security at Application Layer

Security at the application layer embeds protection directly into protocols or programs such as email, web services, messaging systems, and domain-name services.

• Scope: Protection is application-specific; PGP protects email content, while Kerberos authenticates users and network services.
• End-to-end operation: Data may be encrypted by the sending application and decrypted only by the receiving application, remaining protected from intermediate hosts.
• Content awareness: An application can protect selected fields, attachments, commands, or records because it understands their semantics.
• Typical mechanisms:
  • Encryption supplies confidentiality.
  • MACs and digital signatures supply integrity and origin authentication.
  • Certificates or passwords support entity authentication.
  • Timestamps and nonces help prevent replay.
• Advantages: Security policies can closely match business requirements, such as digitally signing an invoice while leaving routing information visible.
• Limitations: Each application requires separate implementation and key management; metadata in lower-layer headers may remain exposed.

B. Email System and PGP

An email system transfers messages through multiple servers, while Pretty Good Privacy (PGP) supplies end-to-end confidentiality, authentication, compression, and email-compatible encoding.

• Email components:
  • A Mail User Agent creates and reads messages.
  • A Mail Transfer Agent relays mail using SMTP.
  • A Mail Delivery Agent places mail in the recipient’s mailbox.
  • POP3 or IMAP allows mailbox access.
• Security problem: Ordinary SMTP messages can be stored or relayed as readable text, and the basic protocol does not prove the sender’s identity.
• PGP authentication: The sender hashes the message and signs the digest with the sender’s private key; the receiver verifies it using the sender’s public key.
• PGP confidentiality: PGP generates a random session key, encrypts the message symmetrically, and encrypts the session key with the recipient’s public key.
• Processing order: A typical protected message is signed, compressed, encrypted, and converted to Radix-64 text for transmission.
• Trust model: PGP commonly uses a decentralized web of trust in which users certify associations between identities and public keys.
• Worked flow:
  
  TEXT

  signature = Sign(sender_private_key, Hash(message))
  ciphertext = Encrypt(session_key, Compress(message || signature))
  key_packet = Encrypt(recipient_public_key, session_key)

  • message is the email content, session_key is a random symmetric key, and key_packet securely carries that key.

III. Secure Transport

Transport-layer protection establishes a secure logical channel between communicating processes, especially between web clients and servers.

A. Security service at transport layer

A transport-layer security service protects application traffic carried over a transport protocol without requiring each application to implement its own cryptography.

• Position: The service operates above TCP and below application protocols such as HTTP, SMTP, and IMAP.
• Confidentiality: Symmetric encryption protects application bytes exchanged after session keys have been established.
• Integrity: A MAC or authenticated-encryption tag detects modification of protected records.
• Authentication: Server authentication is common; optional client certificates provide mutual authentication.
• Application independence: Multiple TCP-based applications can use the same secure-channel design.
• Boundary: Protection applies between transport endpoints; data becomes plaintext after reaching the endpoint unless application-layer protection is also used.

B. SSL architecture

Secure Sockets Layer organizes secure communication into sessions, connections, cryptographic state, and layered protocols; SSL is historically important but has been superseded by TLS.

• SSL session: A relatively long-lived association created by the Handshake Protocol and identified by parameters such as a session identifier, peer certificate, cipher suite, and master secret.
• SSL connection: A transient peer-to-peer transport relationship associated with one session; several connections may reuse session parameters.
• Record-layer state: Each direction maintains separate encryption keys, MAC secrets, initialization values, and sequence numbers.
• Cipher suite: Specifies the key-exchange, authentication, bulk-encryption, and integrity algorithms negotiated by the peers.
• Record processing:
  
  TEXT

  application data -> fragment -> optional compression
                   -> integrity protection -> encryption -> SSL header

• Security dependency: SSL relies on certificate validation, unpredictable random values, secure algorithms, and correct handling of protocol state.

C. Four SSL protocols

SSL consists of four cooperating protocols that negotiate security, report state changes, signal errors, and protect application data.

1. SSL Record Protocol: Fragments higher-layer data, optionally compresses it, adds integrity protection, encrypts it, and prepends a record header.
2. Handshake Protocol: Negotiates the SSL version and cipher suite, authenticates the server and optionally the client, and establishes shared key material through messages such as ClientHello, ServerHello, and Finished.
3. Change Cipher Spec Protocol: Contains a one-byte message that instructs the peer to copy the pending cryptographic state into the current state.
4. Alert Protocol: Communicates warnings, fatal errors, and orderly closure; a fatal alert terminates the connection and invalidates related state.

• Protocol relationship: Handshake, Change Cipher Spec, Alert, and application messages are all encapsulated as Record Protocol payloads.
• Handshake confirmation: The Finished messages authenticate the negotiated exchange, allowing each peer to detect handshake tampering.

D. Transport layer security

Transport Layer Security (TLS) is the standardized successor to SSL and provides authenticated, confidential, integrity-protected communication over reliable transport.

• Standardization: TLS was developed through the IETF; TLS 1.2 is defined by RFC 5246 and TLS 1.3 by RFC 8446.
• TLS 1.3 design: It removes obsolete algorithms, encrypts more handshake information, and normally completes a new handshake in one round trip.
• Key establishment: Ephemeral Diffie-Hellman exchanges, commonly ECDHE, provide forward secrecy because later server-key compromise does not reveal recorded session traffic.
• Record protection: TLS 1.3 uses authenticated encryption with associated data, such as AES-GCM or ChaCha20-Poly1305.
• Certificate role: A server proves possession of the private key associated with a certificate whose identity, validity period, and certification path must be checked.
• Use: HTTPS is HTTP carried through TLS, conventionally using TCP port 443.
• Limitation: TLS does not protect compromised endpoints, and traffic metadata such as IP addresses remains observable.

IV. IP-Layer Security

IP-layer security protects packets independently of applications and can secure host-to-host, host-to-gateway, or gateway-to-gateway communication.

A. IP Security (IPSec)

IP Security (IPSec) is an IETF protocol suite that supplies authentication, integrity, anti-replay protection, and optional confidentiality at the network layer.

• Protocol independence: IPSec can protect TCP, UDP, ICMP, and other payloads carried by IP.
• Main protocols: Authentication Header (AH) supplies authentication and integrity, while Encapsulating Security Payload (ESP) additionally supports confidentiality.
• Key management: Internet Key Exchange negotiates algorithms, authenticates peers, and creates security associations.
• VPN use: Gateways can protect traffic between private networks without modifying individual applications.
• Version support: IPSec is available for IPv4 and is an integral part of the IPv6 protocol suite.

B. Modes of IPSec

IPSec operates in transport mode or tunnel mode, distinguished by how much of the original IP packet is protected.

1. Transport mode: Protects the IP payload while retaining the original IP header for routing; it is mainly used between end hosts.
2. Tunnel mode: Protects the entire original IP packet by encapsulating it inside a new packet with a new outer IP header; it is common for VPN gateways.

• Packet forms:
  
  TEXT

  Transport: [IP header][AH or ESP][transport payload]
  Tunnel:    [new IP header][AH or ESP][original IP packet]

• Trade-off: Tunnel mode hides internal addresses and supports security gateways, but adds more header overhead than transport mode.

C. AH security protocol

Authentication Header provides connectionless integrity, data-origin authentication, and anti-replay protection, but it does not encrypt data.

• Integrity coverage: AH protects the payload and immutable or predictable IP-header fields.
• Authentication data: A keyed integrity algorithm computes an Integrity Check Value over the protected packet.
• Anti-replay field: A monotonically increasing sequence number allows the receiver’s sliding window to reject duplicated packets.
• IP identification: AH uses IP protocol number 51.
• Limitation: Because AH authenticates selected IP-header fields, address translation by NAT can invalidate verification; AH is therefore uncommon compared with ESP.

D. ESP security protocol

Encapsulating Security Payload provides confidentiality and can also provide integrity, origin authentication, and anti-replay protection.

• Encryption scope: ESP encrypts its payload, padding, padding length, and next-header field, but not the outer IP header.
• Authentication scope: When enabled, integrity protection covers the ESP header and protected payload.
• Packet fields: Important fields include Security Parameters Index, sequence number, payload data, padding, next header, and authentication data.
• IP identification: ESP uses IP protocol number 50.
• Modern algorithms: Authenticated-encryption modes such as AES-GCM combine encryption and integrity protection.
• Practical role: ESP is preferred for most VPNs because it supplies confidentiality and works more readily with NAT traversal.

E. Security associations

A security association (SA) is a one-way logical relationship that defines how IPSec protects traffic sent to a particular destination.

• Identification: An SA is selected using the Security Parameters Index, destination IP address, and AH or ESP protocol identifier.
• Stored parameters: It includes mode, algorithms, keys, sequence counters, anti-replay window, lifetime, and tunnel endpoints.
• Directionality: Bidirectional communication requires at least two SAs, one for each direction.
• Databases: Active SAs are held in the Security Association Database.
• SA bundles: Multiple SAs may be applied sequentially, such as end-to-end transport protection inside a gateway tunnel.

F. Security policy

An IPSec security policy determines which packets must be protected, bypassed without IPSec, or discarded.

• Policy database: Rules are stored in the Security Policy Database and matched against selectors such as source, destination, protocol, and port.
• Outbound processing: A matching rule chooses PROTECT, BYPASS, or DISCARD; protected traffic is mapped to a suitable SA.
• Inbound processing: The receiver verifies that IPSec processing and the resulting packet satisfy the applicable policy.
• Policy-SA distinction: A policy states what protection is required, while an SA contains the keys and algorithms that implement it.
• Example rule: Traffic from an internal subnet to a remote corporate subnet may require ESP tunnel mode, while public web traffic may bypass IPSec.