Unit 1: Introduction and Static Malware Analysis

INT251 — Malware Analysis And Cyber Defence 11 min read

I. Orientation

Malware analysis is the disciplined examination of malicious software to determine what it is, how it operates, what systems it affects, and how defenders can detect or contain it. Static analysis studies a file without executing it, using observable properties such as file structure, hashes, strings, imports, metadata, and embedded resources.

Governing principle: Treat malware as untrusted code and preserve the original sample before analysis.
Primary objectives: Identify the sample, classify its behavior, extract indicators of compromise, and support detection or response.
Static-analysis assumption: Useful evidence can be obtained from file contents and structure without running the program.
Evidence convention: Record hashes, filenames, paths, timestamps, tools, and conclusions so findings can be reproduced.
Safety boundary: Analysis should occur in isolated systems with controlled storage and no unnecessary network access.
Analytical limitation: Packing, encryption, anti-analysis techniques, and missing execution context may hide behavior from static inspection.

II. Malware

Malware is software intentionally designed to compromise confidentiality, integrity, availability, authentication, or system control. It may be a standalone executable, script, document, macro, library, boot component, or code delivered through another program.

A. Introduction to malware

This subsection establishes malware’s purpose, delivery, and effects as the foundation for later classification.

Definition: Malware includes malicious programs such as a backdoor that enables unauthorized access or ransomware that encrypts files.
Security impact: A keylogger threatens confidentiality by capturing keystrokes; a destructive wiper threatens availability and integrity by deleting data.
Delivery methods: Common vectors include phishing attachments, drive-by downloads, vulnerable public-facing services, removable media, and compromised software updates.
Execution forms: A malicious payload may run as an .exe, PowerShell script, Office macro, DLL loaded by another process, or JavaScript file.
Malicious intent: Intent may include financial extortion, espionage, credential theft, sabotage, botnet recruitment, or unauthorized resource use.
Indicators of compromise: Examples include a known hash, suspicious domain, unusual persistence location, or a process creating encrypted files rapidly.

B. Types of malware

This subsection distinguishes malware by its propagation method, access mechanism, or primary effect.

Virus: Attaches to a host file and replicates when the host executes; a file-infecting virus may modify executable entry points.
Worm: Self-propagates across systems without requiring a host file; network scanning and exploitation of a service vulnerability are typical signs.
Trojan: Pretends to be legitimate software while performing hidden actions, such as a fake installer that deploys a remote-access tool.
Ransomware: Blocks access to data or systems, commonly by encrypting files with a generated key and demanding payment.
Spyware and keylogger: Collect information covertly; a keylogger may read keyboard events and send captured credentials to a command server.
Rootkit: Hides processes, files, drivers, or registry entries, often by operating at kernel or boot level.
Bot and backdoor: Gives an operator remote control; infected hosts may receive commands for denial-of-service attacks or data theft.
Adware and potentially unwanted software: Displays unwanted advertising or changes browser behavior, sometimes without the transparency expected of legitimate software.
Fileless malware: Uses trusted interpreters such as PowerShell or Windows Management Instrumentation, leaving limited executable content on disk.

C. Malware analysis

This subsection explains malware analysis as an evidence-driven process for understanding a sample and producing defensive outcomes.

Identification: Determine whether a file is malicious by combining hashes, reputation, structure, and behavior rather than relying on one signal.
Capability discovery: Establish whether the sample can persist, communicate, inject code, steal credentials, encrypt files, or alter security settings.
Indicator extraction: Record domains, IP addresses, mutex names, registry keys, filenames, paths, user-agent strings, and cryptographic hashes.
Detection support: Findings can become antivirus signatures, YARA rules, endpoint detection logic, firewall blocks, or incident-response searches.
Sample handling: Use a copy for analysis and calculate a cryptographic hash before modifying or unpacking anything.
Analytical workflow: Triage the file, perform static inspection, escalate to controlled dynamic analysis when required, then correlate results with threat intelligence.
Professional boundary: Analysis should preserve evidence and avoid interacting with live attacker infrastructure unless specifically authorized.

D. Types of malware analysis

This subsection contrasts the major analytical approaches and clarifies when each is useful.

Static analysis: Examines a file without execution, using strings, headers, imports, sections, resources, and disassembly; it is comparatively safe and repeatable.
Dynamic analysis: Executes malware in a controlled environment and observes processes, files, registry changes, network traffic, and memory; it reveals runtime behavior.
1. Basic static versus advanced static:
- Basic static: Uses hashes, file type, strings, metadata, and antivirus results for rapid triage.
- Advanced static: Uses disassemblers and decompilers to inspect instructions, control flow, and API usage without execution.
2. Automated versus manual analysis:
- Automated: Scanners and sandboxes process samples quickly but may miss obfuscated or environment-dependent behavior.
- Manual: An analyst interprets context and intent but requires more time and technical expertise.
Hybrid analysis: Combines static clues with controlled execution, such as using imports to form hypotheses and network traces to verify them.

III. Static Malware Analysis Techniques

Static analysis uses file artifacts as evidence. The following techniques form a practical triage sequence for identifying, fingerprinting, and comparing malware.

A. Determining file type

This subsection identifies the actual format of a file rather than trusting its filename or extension.

Magic bytes: File signatures at the beginning of a file provide strong format clues; Windows PE files commonly begin with MZ (4D 5A in hexadecimal).
Extension unreliability: A file named invoice.pdf.exe may be hidden by operating-system settings, while a renamed executable such as photo.jpg remains executable.
Header confirmation: For a PE file, the DOS header points to the PE signature, normally PE\0\0, through the e_lfanew field.
MIME and format tools: Utilities such as file, TrID, or Detect It Easy compare signatures and structural patterns to identify formats.
Container awareness: A document may contain an embedded executable, archive, script, or macro; identifying only the outer container can miss the payload.
Example: A file with a .docx extension that is actually a PE image should be renamed only in an analysis copy and investigated as an executable.

B. Fingerprinting malware

This subsection creates stable identifiers and descriptive characteristics for tracking a sample across tools and investigations.

Cryptographic hashes: SHA-256 is preferred for exact-file identification; even one changed byte produces a different digest.
Hash command:
TEXT
```
  sha256sum sample.bin
```
sample.bin is the examined file, and the output is its hexadecimal SHA-256 digest.
MD5 and SHA-1: These remain common in legacy threat reports and databases but are unsuitable as modern integrity guarantees because collision attacks exist.
Fuzzy hashes: Similarity algorithms such as ssdeep can indicate related files even when small sections differ; they are similarity indicators, not proof of identity.
Structural fingerprint: Record file size, compile timestamp, section names, imports, resources, entropy, and embedded certificates alongside the hash.
Behavioral fingerprint: A sample’s persistence path, mutex, command-and-control pattern, or encryption routine may link variants from the same family.
Reproducibility: Store the hash with the analyst’s source, tool version, and date because external reputation results can change.

C. Multiple anti-virus scanning

This subsection uses several antivirus engines to obtain a broad reputation signal while recognizing its evidential limits.

Purpose: Multi-engine services compare a sample against vendors with different signatures, heuristics, and cloud intelligence.
Interpretation: A result such as 35/70 detections suggests broad suspicion, but the count is not a probability that the file is malicious.
Detection names: Names can reveal family clues, such as Ransom, Trojan, or a vendor-specific family label, but naming conventions are inconsistent.
False positives: Legitimate administration tools, packed software, and custom binaries may trigger heuristic detections.
False negatives: New, private, encrypted, or heavily obfuscated malware may receive zero detections.
Privacy concern: Uploading a confidential file can disclose proprietary data or an incident sample; check organizational policy before using public services.
Correlation rule: Compare scanner output with file structure, strings, imports, and hashes instead of treating antivirus consensus as final proof.

D. Extracting strings

This subsection recovers readable text that can expose functionality, configuration, and indicators without executing the sample.

Useful content: Strings may include URLs, IP addresses, registry paths, command-line options, error messages, filenames, mutex names, or ransom notes.
ASCII and Unicode: Windows malware frequently stores text as UTF-16LE, so extraction should include both ASCII and wide-character strings.
Command example:
TEXT
```
  strings -a -n 6 sample.exe
  strings -a -el -n 6 sample.exe
```
-a scans the file, -n 6 selects strings of at least six characters, and -el searches little-endian 16-bit text.
Analytical grouping: A URL beside WinHttpOpenRequest supports a network-communication hypothesis; Run beside a registry path suggests possible persistence.
Limitations: Packed malware may contain only loader text, while encrypted configuration produces no meaningful strings until decoded.
Example: http://198.51.100.24/update is an indicator requiring validation; its presence alone does not prove that the program contacts that address.

E. Determining file obfuscation

This subsection assesses whether the sample deliberately hides code or configuration from scanners and analysts.

Purpose of packing: Packers compress or encrypt the original code and provide a small loader that restores it at runtime.
Entropy: High entropy, approaching 8 bits per byte, can indicate compressed or encrypted data; ordinary code and text usually have lower, varied entropy.
Structural clues: A single unusually large section, names such as UPX0 and UPX1, a very small import table, or an entry point in a loader section can indicate packing.
Import scarcity: A packed sample may import only LoadLibraryA and GetProcAddress, resolving APIs dynamically after execution.
Other obfuscation: It may encode strings, flatten control flow, use junk instructions, split configuration across resources, or detect virtual machines.
Caution: High entropy also occurs in legitimate archives, encrypted data, and certificates; entropy is a clue rather than a verdict.
Defensive consequence: Obfuscation increases the value of controlled unpacking, memory capture, and later dynamic analysis.

F. Inspecting PE header information

This subsection examines the Portable Executable structure used by Windows executables, DLLs, drivers, and related files.

DOS header: Begins with MZ; e_lfanew locates the later PE header and helps identify malformed or unusual files.
COFF header: Contains the machine type, number of sections, timestamp, and characteristics such as executable or DLL status.
Optional header: Despite its name, it is required for normal PE images and contains the entry point, image base, section alignment, subsystem, and data-directory locations.
Sections: .text usually contains code, .rdata read-only data, .data writable data, .rsrc resources, and .reloc relocation information; malware may rename or repurpose them.
Imports: DLL and API names reveal possible capabilities, such as CreateProcess, VirtualAlloc, WinInet, or RegSetValueEx.
Exports and resources: Exports can identify DLL interfaces, while resources may contain icons, configuration, scripts, or embedded payloads.
Example: Imports of VirtualAlloc, WriteProcessMemory, and CreateRemoteThread together support a code-injection hypothesis, but require contextual confirmation.
Tools: PE-bear, CFF Explorer, Detect It Easy, objdump, and Python PE-parsing libraries can display headers without running the file.

G. Comparing and classifying malware

This subsection combines static evidence to decide whether samples are related and to assign a defensible malware classification.

Comparison dimensions: Examine hashes, fuzzy similarity, section layout, compiler artifacts, imports, strings, resources, configuration format, and embedded certificates.
Family classification: Samples may belong to a family when they share code, naming conventions, persistence mechanisms, command formats, and distinctive infrastructure.
Variant distinction: A changed payload, packer, configuration, or minor code revision can create a new hash while retaining family-level similarities.
1. Signature-based classification: Matches a known hash, byte pattern, YARA rule, or vendor family name; it is fast but weak against modification.
2. Feature-based classification: Uses combined features such as high-entropy sections, suspicious imports, encoded URLs, and persistence references; it generalizes better but can produce false positives.
Confidence statement: A defensible conclusion separates observation from inference: “The sample imports registry APIs and contains a Run key string” supports, but does not alone prove, persistence.
Operational result: Classification should produce actionable outputs, including hashes for blocking, extracted indicators for hunting, and behavioral hypotheses for sandbox or reverse-engineering follow-up.