Unit 2: Log and Network Traffic Analysis

I. Orientation — Security Evidence and Behavioral Visibility

Log and network traffic analysis converts recorded system activity and communications into evidence for detection, investigation, and response. Logs describe events at hosts, applications, identities, and cloud services; traffic analysis reveals communication patterns, protocol behavior, and data movement.

• Governing principle: Security monitoring compares observed activity with known rules, expected baselines, threat intelligence, and surrounding context.
• Core evidence:
  • Logs: Authentication records, process creation, API calls, database audits, and cloud control-plane events.
  • Network data: Packets, flow records, DNS transactions, TLS metadata, and HTTP requests.
• Common pipeline: Collection → parsing → normalization → enrichment → correlation → detection → investigation → response.
• Essential properties: Accurate timestamps, consistent schemas, sufficient retention, integrity protection, and traceable evidence handling.
• Analytical caution: A single alert is rarely conclusive; confidence rises when independent host, identity, cloud, and network observations agree.

II. Log and Event Analysis — From Recorded Events to Attack Evidence

Log and event analysis examines timestamped records to identify misuse, correlate activity across systems, and reconstruct incidents.

A. Log and Event Analysis: role of log and event analysis

Log analysis provides operational visibility and a historical record of security-relevant actions.

• Detection: Failed logins, privilege changes, malware execution, and unusual API calls can indicate attacks; Windows Event ID 4625, for example, records a failed logon.
• Investigation: Events establish who acted, what resource was affected, when it occurred, and from which host or address.
• Compliance and operations: Audit trails support accountability, while error and performance logs help distinguish attacks from ordinary failures.
• Limit: Absence of a log is not proof that an event did not occur; logging may have been disabled, bypassed, or misconfigured.

B. Log and Event Analysis: advanced log analysis techniques

Advanced analysis finds relationships and patterns that isolated rule matching can miss.

• Correlation: Join events by fields such as user, source IP, hostname, process ID, or session ID within a defined time window.
• Statistical analysis: Baselines use event frequency, rarity, seasonality, and peer-group behavior; ten midnight logins may be normal for one administrator but abnormal for an office user.
• Sequence analysis: Ordered events such as document.exe → powershell.exe → network connection are more informative than any event alone.
• Search methods: Regular expressions, field aggregations, clustering, and timeline queries expose repeated indicators and event chains.

C. Log and Event Analysis: detecting anomalies

Anomaly detection identifies activity that differs materially from expected behavior.

• Baseline: Expected values should reflect time, role, asset type, and business cycle; weekend traffic must not automatically be compared with weekday traffic.
• Standard score:

z = (x - μ) / σ

• x is the observed value, μ the baseline mean, and σ the standard deviation. A large absolute z indicates unusual behavior, not necessarily an attack.
  • Examples: Impossible travel, first-time administrative tools, sudden data exports, and an account accessing hundreds of hosts.
  • Risk: Changing workloads create concept drift, so thresholds and models require regular tuning.

D. Log and Event Analysis: integrating log analysis

Integrating analysis means combining heterogeneous sources into one correlated security view.

• Architecture: Agents, syslog, message queues, and cloud connectors deliver events to a SIEM or data lake.
• Normalization: Vendor fields such as src_ip, sourceAddress, and client.ip should map to a canonical source-address field.
• Enrichment: Asset criticality, identity role, geolocation, and vulnerability status add context before rules execute.
• Time alignment: NTP synchronization and consistent UTC storage prevent false event ordering across systems.

E. Log and Event Analysis: enhancing log data security

Log data must remain confidential, complete, and resistant to alteration.

• Integrity: Append-only storage, restricted deletion, digital signatures, and hash chaining can reveal tampering.
• Confidentiality: Encrypt logs in transit with TLS and at rest with managed keys; mask passwords, tokens, personal data, and session cookies.
• Access control: Apply least privilege and separate log administrators from monitored system administrators.
• Availability: Forward records off-host, replicate critical archives, and define retention periods based on investigative and legal needs.

F. Log and Event Analysis: reconstructing the attack chain

Attack-chain reconstruction orders correlated evidence to explain an adversary’s progression.

• Timeline: Normalize timestamps, deduplicate records, and link events by identity, process, host, IP address, and file hash.
• Typical progression: Phishing login → new cloud session → privilege escalation → persistence → discovery → collection → exfiltration.
• Causality: Parent-child process data can connect winword.exe to a script interpreter and then to an outbound connection.
• Validation: Separate observed facts from inference and preserve original event IDs, timestamps, and raw records as evidence.

G. Log and Event Analysis: APIs for advanced threat detection

APIs allow detection platforms to obtain context and automate analytical actions.

• Data acquisition: REST APIs can retrieve cloud audit events, endpoint telemetry, identity risk, and sandbox verdicts using authenticated requests.
• Enrichment: A suspicious hash can be submitted to a reputation service; the response may provide prevalence, malware family, and first-seen time.
• Automation: SOAR workflows can query an API, raise case severity, disable a token, or isolate an endpoint.
• Controls: Protect API keys, enforce scopes, respect rate limits, validate responses, and log every automated action.

H. Log and Event Analysis: cross-platform log analysis

Cross-platform analysis correlates evidence despite differences among operating systems, applications, and cloud providers.

• Sources: Windows Event Logs, Linux journald or syslog, macOS unified logs, container logs, and cloud audit services use different formats.
• Canonical schema: Normalize timestamp, actor, action, object, outcome, source, destination, and device while retaining the raw event.
• Identity mapping: Relate local usernames, directory accounts, service principals, and cloud identities to one entity.
• Portability: Detection rules should target normalized behavior, such as “new privileged account,” rather than one vendor-specific message.

I. Log and Event Analysis: spotting cloud cryptojacking

Cloud cryptojacking uses compromised resources to mine cryptocurrency at the victim’s expense.

• Control-plane indicators: New high-compute instances, unusual regions, disabled monitoring, changed quotas, and newly created access keys.
• Workload indicators: Sustained CPU or GPU usage near capacity, mining-pool DNS queries, unfamiliar containers, and repeated outbound connections.
• Correlation: Link instance creation to the initiating identity, source IP, API token, and billing spike.
• Response: Revoke exposed credentials, isolate workloads, preserve snapshots and audit logs, and check persistence mechanisms before termination.

J. Log and Event Analysis: integration of log analysis

Operational integration embeds log findings into broader detection and response processes.

• SOC workflow: SIEM alerts should create cases containing evidence, severity, affected assets, ownership, and response playbooks.
• Endpoint and network linkage: A suspicious login gains confidence when followed by endpoint process execution and command-and-control traffic.
• Automation boundary: Low-risk enrichment may be automatic, whereas account deletion or production isolation normally requires approval.
• Feedback loop: Incident conclusions should update rules, allowlists, threat intelligence, and telemetry requirements.

K. Log and Event Analysis: evaluating log analysis

Evaluation determines whether logging and detections provide useful, reliable coverage.

• Coverage: Map required data sources and rules to assets, attack techniques, and critical business processes.
• Quality: Measure missing fields, parsing failures, duplicate events, clock drift, ingestion delay, and collection outages.
• Effectiveness: Track true-positive rate, false-positive rate, mean time to detect, and mean time to respond.
• Validation: Controlled simulations should generate known events and confirm collection, correlation, alert creation, and analyst interpretation.

III. Network Traffic Analysis — Understanding Communications and Protocol Behavior

Network traffic analysis inspects packet contents or communication metadata to identify malicious relationships, policy violations, and deviations from expected protocol use.

A. Network Traffic Analysis: traffic segmentation and normalization

Segmentation separates traffic into meaningful zones, while normalization makes observations consistently analyzable.

• Segmentation: Distinguish internet edge, user, server, management, operational technology, and cloud networks; inter-zone traffic provides valuable choke points.
• Traffic units: Packets may be grouped into flows using source IP, destination IP, source port, destination port, and transport protocol—the five-tuple.
• Normalization: Reassemble fragments and streams, standardize timestamps and addresses, and resolve ambiguous protocol encodings.
• Security value: Normalization prevents attackers from exploiting differences between how sensors and destination systems interpret malformed traffic.

B. Network Traffic Analysis: threat intelligence integration

Threat intelligence enriches observed communications with knowledge about known adversary infrastructure and behavior.

• Indicators: Compare IP addresses, domains, URLs, certificates, and file hashes with curated intelligence feeds.
• Context: Record confidence, source, first-seen time, last-seen time, and expiration; shared cloud IP addresses may otherwise create false positives.
• Behavioral intelligence: Tactics and techniques, such as periodic beaconing or domain generation, remain useful after individual indicators change.
• Prioritization: A match involving a critical server and a current high-confidence indicator warrants greater severity than a stale match.

C. Network Traffic Analysis: contextual protocol analysis

Contextual analysis evaluates whether a protocol is being used appropriately, not merely whether its syntax is valid.

• Role context: DNS from an approved resolver is expected; direct external DNS from a database server may violate policy.
• Semantic context: HTTP methods, DNS record types, TLS certificates, SMB operations, and authentication results reveal intent.
• Encrypted traffic: Analysts can use destination, timing, byte counts, TLS version, certificate metadata, and connection frequency without decrypting content.
• Example: Long, high-entropy DNS labels sent repeatedly may indicate tunnelling rather than ordinary name resolution.

D. Network Traffic Analysis: NIDS/NIPS

A network intrusion detection system alerts on suspicious traffic, whereas a network intrusion prevention system can block it inline.

1. NIDS: Passive monitoring reduces operational risk but cannot directly stop traffic; it commonly uses signature, anomaly, and protocol rules.
2. NIPS: Inline inspection can drop packets, reset sessions, or rate-limit sources, but false positives may interrupt legitimate services.
   • Placement: Sensors belong at internet gateways, data-centre boundaries, cloud inspection points, and key east-west segments.
   • Constraint: Encryption, packet loss, asymmetric routing, and high throughput reduce visibility and detection accuracy.

E. Network Traffic Analysis: vulnerability validation

Traffic evidence can validate whether a vulnerability is reachable, attempted, or successfully exploited.

• Exposure: Observed service banners, protocol versions, and listening communications supplement scanner results.
• Exploit evidence: Requests containing exploit patterns indicate attempts, while a subsequent shell connection or abnormal process provides stronger success evidence.
• Prioritization: Combine vulnerability severity with asset importance, network reachability, exploit availability, and observed attack traffic.
• Safety: Passive validation avoids disruption; active testing must be authorized and carefully scoped.

F. Network Traffic Analysis: analyzing jarring signals

Jarring signals are abrupt or internally inconsistent observations that violate normal network expectations.

• Abrupt changes: Sudden traffic spikes, new ports, large outbound transfers, or rapid fan-out across many hosts may indicate scanning or exfiltration.
• Contradictions: An internal address appearing externally, HTTP on an unusual port, or a workstation behaving as a server demands investigation.
• Correlation: Confirm whether maintenance, backups, failover, or software deployment explains the deviation.
• Triage: Rank signals by rarity, duration, affected asset, destination reputation, and agreement with host logs.

G. Network Traffic Analysis: modelling protocol behavior

Protocol models describe valid states and expected transitions so that abnormal sequences can be detected.

• State model: A TCP conversation normally progresses through establishment, data exchange, and termination; impossible transitions may indicate malformed or evasive traffic.
• Statistical model: Learn typical message sizes, request rates, response codes, session duration, and client-server ordering.
• Transition probability:

P(next_state | current_state)

• P is the learned probability of the next protocol state given the current state; very low-probability transitions receive higher anomaly scores.
  • Limitation: Models require protocol-aware parsing and retraining when applications, encryption, or legitimate usage patterns change.