Unit 6: Analysis of Malware

I. Orientation — The Nature of Malicious Software

Malware is software or code intentionally used to compromise confidentiality, integrity, availability, or authorized control of systems and data. Malware analysis combines digital forensics, reverse engineering, and behavioral observation to determine what a suspicious artifact does, how it operates, and what evidence it leaves.

A. Malware Definition

Malware is defined by malicious purpose or unauthorized effect rather than by a particular file format, programming language, or delivery mechanism.

• Core objectives: Malware commonly performs theft, espionage, disruption, extortion, unauthorized access, or resource hijacking.
  • Confidentiality: Credential stealers and spyware disclose protected information.
  • Integrity: File infectors and destructive malware alter data or system settings.
  • Availability: Ransomware and wipers prevent legitimate access to resources.
• Typical components: A campaign may contain a delivery document, downloader, payload, persistence mechanism, and command-and-control channel rather than one self-contained executable.
• Major categories:
  • Virus: Attaches to another file and replicates when the host executes.
  • Worm: Self-propagates across systems or networks.
  • Trojan: Appears legitimate while performing concealed actions.
  • Ransomware: Encrypts, locks, or steals data to support extortion.
  • Spyware: Collects information such as credentials, browser data, or keystrokes.
  • Rootkit: Conceals malicious activity or provides privileged access.
  • Bot: Places a device under remote control as part of a botnet.
• Important distinction: A potentially unwanted application may be intrusive without meeting the stronger intent normally associated with malware; analysis must rely on observed capability and context.

II. Malware Distribution — Initial Access and Propagation

Attackers spread malware by exploiting human trust, technical vulnerabilities, weak authentication, and trusted software relationships. Delivery should be distinguished from execution: receiving a malicious attachment does not compromise a system until code or an exploit activates.

A. Common Techniques Used by Attackers to Spread Malware

Malware distribution techniques attempt to persuade users, applications, or services to retrieve and execute an attacker-controlled payload.

• Phishing and social engineering: Messages impersonate trusted parties and deliver attachments, links, QR codes, or fake login pages. Examples include macro-enabled Office files and password-protected archives.
• Exploitation of vulnerabilities: Drive-by downloads, exposed servers, and malicious documents exploit unpatched software; the exploit establishes execution, after which a separate payload may be downloaded.
• Malvertising and compromised websites: Advertisements or injected scripts redirect victims to fraudulent updates, exploit infrastructure, or trojanized installers.
• Supply-chain compromise: Attackers modify software packages, build systems, libraries, or update channels so malicious code inherits the vendor’s apparent trust.
• Removable media: USB devices may carry infected files, deceptive shortcuts, or malicious firmware and can cross network isolation boundaries.
• Credential abuse: Stolen VPN, remote desktop, cloud, or administrative credentials allow an attacker to place malware directly on a target.
• Network propagation: Worms and human-operated intrusions move through vulnerable services, administrative shares, remote-management tools, or reused passwords.
• Defensive significance: Email filtering, patching, multifactor authentication, application control, and user awareness address different stages; no single control blocks every delivery path.

III. Malware Forensics — Evidence, Goals, and Analytical Models

Malware forensics applies repeatable forensic procedures to suspicious code and its associated host and network evidence. Its goals include classification, incident scoping, attribution support, recovery, and production of defensible indicators.

A. Malware Forensics Fundamentals

Malware forensics must preserve evidence while reconstructing the artifact’s origin, execution, capabilities, and effects.

• Evidence integrity: Original media and files should be preserved, access documented, and working copies analyzed. Cryptographic hashes such as SHA-256 identify exact byte sequences.
• Chain of custody: Records should state who collected, transferred, stored, and examined each item, including dates, tools, and evidence identifiers.
• Acquisition priorities: Volatile evidence—processes, memory, connections, logged-in users, and encryption keys—may disappear at shutdown and is therefore often collected before disk imaging.
• Analytical questions:
  • How did the malware arrive and execute?
  • What persistence, privilege, discovery, and lateral-movement actions occurred?
  • What files, accounts, systems, and data were affected?
  • Which indicators can support detection and containment?
• Indicators of compromise: Hashes, filenames, registry paths, mutexes, domains, IP addresses, certificates, and behavioral patterns can identify related activity. Hashes are precise but fragile because one byte change produces a different value.
• Timeline correlation: File-system timestamps, event logs, endpoint telemetry, browser history, and network captures should be aligned using a known time zone and checked for clock drift.
• Safe handling: Analysis uses isolated virtual machines, controlled networking, snapshots, and nonproduction accounts; suspicious artifacts should not be opened on ordinary workstations.

B. Types of Malware Analysis

Malware analysis methods differ in whether they inspect code without execution, observe execution, or reconstruct detailed program logic.

1. Static approaches:
   • Basic static analysis: Examines hashes, file type, metadata, strings, headers, imports, signatures, and embedded resources without running the sample.
   • Advanced static analysis: Uses disassembly and decompilation to study control flow, algorithms, configuration data, and anti-analysis logic.
2. Dynamic approaches:
   • Basic dynamic analysis: Executes a sample in a sandbox and records processes, files, registry changes, and connections.
   • Advanced dynamic analysis: Uses debuggers, memory inspection, API tracing, and controlled breakpoints to examine particular execution paths.
   • Hybrid analysis: Combines static and dynamic evidence because packed code may hide statically, while dormant functionality may not appear during one execution.
   • Limitation: A clean sandbox run does not prove safety; malware may require user actions, command-line arguments, elapsed time, geographic conditions, or a live command server.

IV. Static Examination — Analyzing Artifacts Without Execution

Static examination extracts structural and semantic evidence while avoiding execution of the suspected malware. It is efficient for triage, but packing, encryption, obfuscation, and malformed files can restrict visibility.

A. Static Analysis of Malware

Static analysis identifies a file’s true format, probable capabilities, and useful indicators from its stored bytes and structure.

• Identification: Determine file type from magic bytes and internal structure rather than filename extension; an executable renamed .jpg still retains an executable header.
• Hashing: Calculate SHA-256 for evidence tracking and exact-match searching. Fuzzy or similarity hashes may help group related samples but are not substitutes for cryptographic integrity hashes.
• Strings: ASCII and Unicode strings can reveal URLs, commands, paths, error messages, registry keys, or campaign identifiers; encoded or encrypted strings require further inspection.
• Executable structure: For Windows Portable Executable files, analysts inspect sections, imports, exports, compilation metadata, resources, and entry point.
  • Imports such as networking, process creation, or registry functions suggest capabilities but do not prove malicious use.
• Packing indicators: High-entropy sections, unusual section names, a small import table, or an entry point in an unexpected section may indicate compression or encryption.
• Code analysis: Disassemblers translate machine instructions into assembly; decompilers produce higher-level approximations. Analysts trace branches, function calls, and data references rather than treating decompiled output as original source.
• Constraint: Static properties establish potential behavior; dynamic or contextual evidence is needed to show that a capability actually executed.

B. Analysis of Suspicious Word and PDF Documents

Suspicious documents are container formats that may hold active content, embedded objects, links, exploits, or social-engineering instructions.

• Word documents: Analysts distinguish modern Open XML containers such as .docx and .docm from older compound binary formats.
  • ZIP-based Office files can expose relationships, embedded objects, external templates, and macro projects without opening the document in Word.
  • VBA macros should be reviewed for auto-execution routines, obfuscated strings, shell commands, and network retrieval behavior.
• PDF documents: Examination covers the header, objects, cross-reference information, streams, embedded files, forms, actions, and scripts.
  • Suspicious features include automatic actions, JavaScript, launch behavior, embedded executables, or links to newly created domains.
• Deobfuscation: Encoded strings and compressed streams should be decoded only in controlled tools; analysts retain both original and derived artifacts and hash each one.
• Metadata: Author names, producer software, template paths, and creation times can support correlation but may be forged or inherited.
• Safe workflow: First hash and identify the document, then parse it with specialized tools, extract active content, inspect links and objects, and use sandbox execution only when static evidence is insufficient.

V. Dynamic Examination — Observing Execution and Effects

Dynamic analysis executes malware under controlled conditions to reveal actual behavior. A reproducible laboratory normally uses disposable virtual machines, snapshots, instrumentation, restricted connectivity, and synchronized host and network logging.

A. Dynamic Malware Analysis Fundamentals and Approaches

Dynamic analysis records actions produced by a sample while controlling the conditions that influence those actions.

• Baseline and comparison: Capture system state before execution, monitor the run, and compare the final state to distinguish malware changes from pre-existing artifacts.
• Execution context: Record the sample hash, command line, privileges, operating-system version, time, user actions, and network configuration.
• Instrumentation: Process monitors, API tracers, debuggers, memory-capture tools, packet analyzers, and simulated network services provide complementary evidence.
• Controlled approaches:
  1. Automated sandboxing: Offers rapid, standardized reports across many samples but may miss delayed or environment-sensitive behavior.
  2. Manual analysis: Allows interaction, breakpoint control, and alternate execution paths but requires more time and expertise.
• Anti-analysis behavior: Malware may detect virtual hardware, analysis tools, short uptime, missing user activity, or disabled internet access and then terminate or behave benignly.
• Repeatability: Analysts restore clean snapshots between runs and vary one condition at a time, such as privilege level or network availability.

B. Real-time Analysis of Malware Behavior on System Properties

Host monitoring determines how malware changes processes, files, configuration, persistence, memory, and security controls during execution.

• Processes and threads: Record parent-child relationships, command lines, integrity levels, loaded modules, injected memory, and unexpected process termination.
• File system: Observe created, modified, deleted, or renamed files, including temporary directories, startup locations, and alternate data streams.
• Configuration: Monitor registry keys, services, scheduled tasks, startup folders, event subscriptions, browser settings, and firewall rules.
• Persistence evidence: A new service or scheduled task is stronger evidence when linked by time, command path, and process lineage to the sample.
• Memory behavior: Process memory may expose unpacked code, decrypted configuration, credentials, or network endpoints absent from the disk file.
• Security interference: Attempts to disable logging, endpoint protection, backups, or recovery services indicate defense evasion or destructive intent.
• Interpretation rule: Correlate multiple observations; a file creation alone may be normal, whereas creation followed by execution, persistence, and outbound communication forms a meaningful behavioral chain.

C. Real-time Analysis of Malware Behavior on Networks

Network analysis identifies external communication, command infrastructure, propagation attempts, and possible data exfiltration.

• Data sources: Packet captures provide payload-level detail, while DNS, proxy, firewall, TLS, and flow logs support broader and longer-term correlation.
• Observable behavior: Analysts examine DNS queries, destination addresses, ports, protocols, HTTP methods, TLS metadata, connection frequency, and transferred volume.
• Command and control: Regular beacon intervals, repeated failed connections, algorithmically generated domains, unusual user agents, or encrypted traffic to rare destinations may indicate remote control.
• Exfiltration indicators: Large uploads, unusual archive transfers, long DNS labels, or outbound traffic after sensitive-file access deserve correlation with host events.
• Containment in laboratories: Simulated DNS, web, and other services can elicit behavior without exposing the sample to the public internet.
• Limitation: Encryption hides content but not all metadata; timing, destination, certificate characteristics, and byte counts remain available. Network indicators should be contextualized because shared cloud addresses may also serve legitimate users.

VI. Memory-Resident Threats — Abuse of Trusted System Facilities

Fileless malware minimizes conventional executable files on disk by operating through memory, scripts, system utilities, registry data, or remote administration mechanisms. “Fileless” does not mean evidence-free: scripts, logs, memory structures, persistence records, and network traces may remain.

A. Fileless Malware Attacks and How They Happen

Fileless attacks use trusted interpreters and administrative components to execute malicious instructions while reducing file-based detection opportunities.

• Initial access: A phishing link, document exploit, stolen credential, or vulnerable service starts the attack; the entry artifact itself may still be a file.
• Living-off-the-land execution: Legitimate scripting engines, management frameworks, document interpreters, or signed system utilities execute commands already available on the host.
• Memory residence: Payloads may be decoded directly into memory, injected into another process, or loaded reflectively without a conventional installed executable.
• Persistence: Attackers may store encoded commands in registry values, management subscriptions, scheduled tasks, or shortcut parameters.
• Typical sequence:
  1. A user or vulnerable application launches a script or command.
  2. The command retrieves or reconstructs code.
  3. Code executes in memory under a trusted process.
  4. Persistence and command-and-control mechanisms maintain access.
• Forensic detection: Valuable evidence includes process command lines, script-block or interpreter logs, memory captures, parent-child process anomalies, registry modifications, authentication records, and network telemetry.
• Defensive implication: Effective detection emphasizes behavior—such as an Office process spawning an interpreter—alongside application control, restricted scripting, least privilege, detailed logging, memory analysis, and rapid credential containment.