Unit 2: Dynamic Analysis and Assembly Language - Subjective Questions

Dynamic analysis involves executing malware in a controlled environment and observing its behavior. A typical workflow is:

1. Prepare an isolated laboratory:

   • Use a virtual machine with snapshots.
   • Disable shared folders, clipboard sharing, and unnecessary network access.
   • Install monitoring and debugging tools.

2. Record the initial state:

   • Capture running processes, services, files, registry keys, and network configuration.
   • Calculate cryptographic hashes such as SHA-256 for the sample.

3. Perform baseline monitoring:

   • Observe normal system and network activity before executing the sample.
   • This helps distinguish malware activity from background noise.

4. Execute the sample:

   • Run the sample with appropriate privileges.
   • Provide simulated command-line arguments, files, or network services if required.

5. Monitor behavior:

   • Record process creation, file operations, registry modifications, API calls, mutexes, services, scheduled tasks, and network traffic.

6. Interact with the sample:

   • Trigger buttons, documents, timers, or simulated server responses to reveal hidden behavior.

7. Collect artifacts:

   • Save packet captures, memory dumps, dropped files, logs, screenshots, and process dumps.

8. Analyze persistence and impact:

   • Determine whether the malware establishes persistence, steals data, modifies security settings, or communicates with a command-and-control server.

9. Restore the environment:

   • Revert the virtual machine to a clean snapshot.

10. Document findings:

    • Prepare a timeline, indicators of compromise, behavioral summary, and defensive recommendations.

A safe malware analysis laboratory should prevent the sample from affecting production systems while still allowing its behavior to be observed.

• Virtualization: Execute malware inside a dedicated virtual machine using a hypervisor. Take a clean snapshot before analysis.
• Network isolation: Use host-only or isolated networking. Internet access should be blocked or simulated with tools that provide fake DNS, HTTP, SMTP, and other services.
• Disable integration features: Turn off shared folders, drag-and-drop, clipboard synchronization, USB passthrough, and automatic host file access.
• Monitoring tools: Install process, file, registry, API, memory, and packet-monitoring utilities.
• Controlled privileges: Begin with normal privileges and elevate only when required to test privilege-dependent behavior.
• Reproducibility: Record the operating system version, installed software, sample hash, execution time, and test inputs.
• Containment: Do not connect the laboratory to organizational networks or use real credentials and personal data.
• Recovery: Revert to a trusted snapshot after every experiment and securely preserve only the required evidence.

These controls reduce the risk of accidental infection, data leakage, and uncontrolled command-and-control communication.

The major behavioral indicators include:

• Process activity: New processes, terminated processes, process trees, command-line arguments, and suspicious parent-child relationships.
• File-system activity: Created, modified, deleted, renamed, encrypted, or hidden files. Analysts should also examine temporary directories and dropped executables.
• Registry activity: Changes to startup keys, security settings, file associations, and application configuration.
• Persistence mechanisms: Services, scheduled tasks, startup folders, logon scripts, browser extensions, and registry run keys.
• Network activity: DNS queries, remote IP addresses, domain names, protocols, ports, beacon intervals, downloaded payloads, and encrypted traffic.
• Memory behavior: Injected code, unpacked payloads, suspicious memory permissions, decrypted strings, and process hollowing.
• API usage: Calls related to file access, networking, process creation, memory allocation, cryptography, and privilege manipulation.
• System changes: New users, altered firewall rules, disabled security services, or modified boot settings.
• Anti-analysis behavior: Delays, virtual-machine checks, debugger detection, environment checks, and rapid termination.

These observations can be converted into indicators of compromise and detection rules.

Static analysis examines a malware sample without executing it, whereas dynamic analysis observes the sample while it is running.

Advantages of static analysis:

• Safe initial examination.
• Useful for identifying file format, architecture, imports, and embedded data.
• Does not depend on successful execution.

Limitations of static analysis:

• Obfuscation, encryption, and packing can hide functionality.
• Assembly-level analysis may be time-consuming.

Advantages of dynamic analysis:

• Reveals actual runtime behavior.
• Captures unpacked payloads and generated data.
• Helps identify persistence and network communication.

Limitations of dynamic analysis:

• Malware may detect the laboratory or remain dormant.
• Only triggered execution paths are observed.
• Improper isolation may create security risks.

The two approaches are complementary and should normally be used together.

A DLL cannot always be analysed by simply opening it because it is normally loaded by another process. A systematic procedure is:

1. Perform preliminary inspection:

   • Verify the Portable Executable format and architecture.
   • Identify hashes, sections, imports, exports, resources, and suspicious strings.

2. Identify an execution method:

   • Determine whether the DLL exports callable functions.
   • Identify whether it is intended for a specific application, service, or loader.
   • Use a controlled loader only when the expected interface is understood.

3. Prepare monitoring:

   • Record the initial system state.
   • Enable process, file, registry, API, and network monitoring.

4. Load the DLL:

   • Execute it through the expected host program or a controlled DLL-loading utility.
   • Supply required export names, ordinal values, or arguments.

5. Observe loader activity:

   • Monitor DLL loading, dependency resolution, entry-point execution, and child-process creation.

6. Debug important functions:

   • Place breakpoints on the DLL entry point, exported functions, memory-allocation APIs, process-creation APIs, and networking APIs.

7. Dump runtime content:

   • If the DLL is packed or decrypted, dump the unpacked image or relevant memory regions.

8. Document behavior:

   • Record persistence, injection, network communication, dropped files, and indicators of compromise.

Care must be taken because incorrectly calling an exported function with the wrong arguments or calling convention may cause a crash rather than reveal meaningful behavior.

• DLL entry point: The entry point is commonly invoked by the operating-system loader when the DLL is attached to or detached from a process or thread. Analysts inspect it for initialization, unpacking, environment checks, or malicious setup. Complex operations inside the entry point may also indicate suspicious design.

• Exported functions: These are functions made available to other modules. Their names may reveal capabilities, but malware may use misleading names or export functions only by ordinal. Analysts determine which exports contain the main malicious logic and what arguments they expect.

• Imported functions: Imports identify external APIs required by the DLL. Networking, process injection, cryptography, service creation, and file-manipulation imports can suggest possible behavior. Dynamically resolved imports may not appear in the normal import table.

• Ordinals: An ordinal is a numeric identifier used to export or import a function without a textual name. Malware may use ordinals to reduce readable information and complicate analysis.

These elements help the analyst determine how the DLL is loaded, how its code is activated, and which operating-system capabilities it may use.

Assembly language is a low-level representation of machine instructions that uses symbolic mnemonics, register names, labels, and memory operands. It provides a close correspondence between program instructions and processor operations.

A typical assembly instruction contains:

• Mnemonic: Specifies the operation, such as mov, add, xor, or jmp.
• Destination operand: Identifies where the result is written.
• Source operand: Supplies the input value.
• Optional size information: Indicates whether the operand is a byte, word, double word, or quad word.
• Addressing expression: Calculates the address of a memory operand.

For example, in Intel syntax:

mov eax, [rbx+4]

• mov is the mnemonic.
• eax is the destination register.
• [rbx+4] is the source memory operand.

Common operand types are immediate values, register operands, and memory operands. Assembly syntax varies among assemblers; for example, Intel and AT&T syntax use different operand orders and notation.

The major x86 register categories are:

• General-purpose registers: Registers such as EAX, EBX, ECX, EDX, ESI, and EDI hold operands, addresses, counters, and temporary results. Their exact roles depend on the program and calling convention.
• Stack pointer: ESP points to the current top of the stack.
• Base or frame pointer: EBP may identify the current stack frame, although optimized code can use it as a general-purpose register.
• Instruction pointer: EIP contains the address of the next instruction to execute.
• Flags register: EFLAGS stores status and control flags such as zero, carry, sign, and overflow.
• Segment registers: CS, DS, SS, ES, FS, and GS support segmented addressing and access to thread- or system-related structures.
• Floating-point and vector registers: The x87, MMX, and XMM registers support floating-point, multimedia, cryptographic, and vector operations.

During malware analysis, register values reveal function arguments, return values, memory addresses, loop counters, decoded data, and branch conditions. Tracking changes to EIP, ESP, and status flags is especially important when debugging control flow.

The x64 architecture extends x86 while preserving substantial backward compatibility.

Major features include:

• Wider registers: General-purpose registers are 64 bits wide, such as RAX, RBX, and RCX.
• Additional registers: x64 adds R8 through R15, reducing dependence on memory and the stack.
• Register subdivisions: For example, RAX contains EAX, AX, AH, and AL. Writing to EAX clears the upper 32 bits of RAX.
• 64-bit instruction pointer: RIP controls execution. RIP-relative addressing is widely used for position-independent access to code and data.
• Larger address space: x64 can address much more memory than the theoretical -byte address space of 32-bit x86.
• Calling conventions: x64 calling conventions pass several arguments in registers. Under the Microsoft x64 convention, the first four integer or pointer arguments are generally passed in RCX, RDX, R8, and R9.
• Stack alignment: Calling conventions normally require stack alignment, commonly on a 16-byte boundary at specified points.
• Instruction encoding: A REX prefix may be used to select 64-bit operand size and access the additional registers.

For malware analysis, the analyst must identify the correct architecture, calling convention, pointer size, stack layout, and register usage. A 32-bit debugger or loader cannot directly treat a 64-bit sample as ordinary 32-bit code.

Data transfer instructions copy, exchange, load, or store values without normally changing the logical meaning of the data.

• mov: Copies data from a source to a destination. A direct memory-to-memory move is generally not permitted in a basic mov instruction.
• lea: Loads an effective address rather than the value stored at that address. It is also frequently used for arithmetic calculations.
• push: Places a value on the stack and updates the stack pointer.
• pop: Removes a value from the stack and updates the stack pointer.
• xchg: Exchanges the values of two operands.
• movzx: Moves a smaller value into a larger destination and fills the upper bits with zeros.
• movsx: Moves a smaller signed value into a larger destination and replicates the sign bit.

For an address expression such as [base+index*scale+displacement], the effective address is:

In malware, these instructions are used to prepare API arguments, traverse data structures, copy decoded values, manipulate the stack, and access configuration data.

Common arithmetic instructions include:

• add: Adds the source operand to the destination.
• sub: Subtracts the source operand from the destination.
• inc and dec: Increment or decrement an operand by one.
• imul: Performs signed multiplication.
• mul: Performs unsigned multiplication.
• idiv and div: Perform signed and unsigned division respectively.
• neg: Replaces an operand with its two's-complement negative.
• cmp: Performs subtraction for flag calculation but discards the result.

Important status flags are:

• Zero Flag: Set when the result is zero.
• Sign Flag: Reflects the most significant bit of the result.
• Carry Flag: Indicates an unsigned carry or borrow.
• Overflow Flag: Indicates that a signed result is outside the representable range.
• Parity Flag: Indicates even parity in the low-order byte.
• Auxiliary Carry Flag: Records a carry between lower nibbles.

For an -bit unsigned value, the representable range is to . For an -bit signed two's-complement value, the range is to . Correctly interpreting carry and overflow is essential when reconstructing comparisons and conditions.

Common bitwise operations include:

• and: Produces 1 only when both corresponding bits are 1. It is used for masking and alignment.
• or: Produces 1 when at least one corresponding bit is 1. It is used to set selected bits.
• xor: Produces 1 when corresponding bits differ. It is often used for reversible encoding and to clear a register, as in xor eax, eax.
• not: Inverts every bit.
• test: Performs a bitwise AND for flag calculation without storing the result.
• shl or sal: Shifts bits left and generally multiplies an unsigned value by powers of two when no significant bits are lost.
• shr: Performs a logical right shift and fills high bits with zeros.
• sar: Performs an arithmetic right shift and preserves the sign bit.
• rol and ror: Rotate bits around the operand rather than discarding shifted bits.

Malware may use these operations to encode strings, decrypt configuration data, calculate hashes, manipulate permission flags, align addresses, hide constants, and implement custom cryptographic routines. A sequence of XOR, rotate, and add operations is commonly encountered in API hashing and data-obfuscation algorithms.

Decision-making is implemented by changing the instruction pointer according to status flags.

• Unconditional branch: jmp always transfers execution to its target.
• Comparison: cmp subtracts operands for flag calculation without preserving the result.
• Bit test: test performs a bitwise AND for flag calculation without storing the result.
• Conditional branches: Instructions such as je, jne, jg, jl, ja, and jb branch only when particular flag conditions are satisfied.

Examples of conditions include:

• je or jz: branch when the Zero Flag is set.
• jne or jnz: branch when the Zero Flag is clear.
• ja and jb: unsigned above and below comparisons.
• jg and jl: signed greater-than and less-than comparisons.

Signed and unsigned conditions must not be confused. Signed comparisons use relationships among the Sign, Overflow, and Zero Flags, whereas unsigned comparisons mainly use the Carry and Zero Flags.

Malware uses conditional branches for password checks, environment detection, anti-debugging tests, command selection, error handling, and payload activation. Analysts reconstruct high-level if, else, and switch statements by following these branches and their flag-producing instructions.

A loop repeatedly executes a block of instructions until a termination condition is satisfied. Assembly loops usually contain:

1. Initialization: A counter, pointer, or condition value is initialized.
2. Body: The repeated operation is performed.
3. Update: The counter or pointer is modified.
4. Condition check: A comparison or test updates flags.
5. Backward branch: A conditional jump returns to the beginning when repetition should continue.

Example:

mov ecx, 10

loop_start: add eax, [rsi]

add rsi, 4

dec ecx

jnz loop_start

This loop processes ten 4-byte elements. The dec instruction updates the counter, and jnz repeats the loop while the counter is nonzero.

The dedicated loop instruction can also decrement a count register and branch, although compilers often prefer explicit decrement-and-jump sequences. Malware uses loops to process arrays, decode strings, encrypt files, scan memory, calculate hashes, and retry network connections.

A function is a reusable code block entered with a call and normally exited with a return.

• Function call: call stores the return address and transfers execution to the target function.
• Function return: ret retrieves the saved return address and resumes execution at the caller.
• Stack frame: A function may save a frame pointer, allocate space for local variables, and preserve required registers.
• Arguments: Depending on the calling convention, arguments are placed on the stack, in registers, or in both.
• Return value: Integer and pointer return values are commonly placed in EAX on x86 or RAX on x64.
• Register preservation: Calling conventions define which registers the caller must save and which the callee must preserve.

A traditional x86 function prologue may save EBP, copy ESP to EBP, and subtract from ESP to allocate local storage. Its epilogue reverses these operations. Optimized functions may omit the frame pointer.

Under the Microsoft x64 convention, the first four integer or pointer arguments are generally passed through RCX, RDX, R8, and R9, and the caller reserves stack space for register arguments. Under the System V AMD64 convention, a different register order is used.

Identifying the calling convention is essential in malware analysis because it allows the analyst to reconstruct API arguments, local variables, function boundaries, and returned results.

An array stores elements of the same type in contiguous memory. To access a one-dimensional array, the address of element is calculated as:

For example, if rsi contains the base address of an array of 4-byte integers and rcx contains index , [rsi+rcx*4] refers to element .

For a row-major two-dimensional array with columns, the address of is:

Assembly code may calculate this address using multiplication, shifts, additions, or scaled-index addressing. A pointer may also be incremented by the element size instead of recalculating every address.

During malware analysis, array access patterns may indicate lookup tables, substitution tables, lists of process identifiers, encrypted byte buffers, or collections of function pointers. Bounds checking is usually not automatically performed by assembly instructions.

A string is stored as a sequence of character codes in memory. Common representations include:

• Null-terminated strings: End with a zero byte or zero word.
• Length-prefixed strings: Store the length before the character data.
• Fixed-length buffers: Reserve a predefined number of bytes or words.
• Narrow and wide strings: Narrow strings often use one byte per character, while UTF-16 strings commonly use two-byte code units.

Strings can be processed using ordinary load, compare, and branch instructions or specialized string instructions such as movs, cmps, scas, lods, and stos. Repeat prefixes such as rep, repe, and repne can repeat string operations according to a counter and flag condition.

A string-processing loop generally maintains:

• A source pointer.
• A destination pointer when copying or transforming data.
• A counter or terminator check.
• An operation such as comparison, XOR decoding, or character conversion.

Malware commonly encodes strings to conceal URLs, file paths, registry keys, commands, and API names. Dynamic analysis can reveal these strings after they are decoded in memory.

A structure is a group of related fields stored at fixed offsets from a base address. Assembly language normally accesses a field by adding its offset to the structure's base pointer.

If a structure contains a 4-byte identifier at offset 0, an 8-byte pointer at offset 8, and a status byte at offset 16, the fields may be accessed as [rbx], [rbx+8], and [rbx+16] respectively.

An analyst can reconstruct an unknown structure by:

• Identifying a register that consistently acts as the base pointer.
• Recording every constant offset used with that register.
• Determining access size, such as byte, word, double word, or quad word.
• Observing whether each field is read, written, compared, or dereferenced.
• Following allocation calls to estimate the total structure size.
• Examining loops to identify embedded arrays.
• Matching field usage with known operating-system or library structures.
• Considering padding and alignment between fields.

For example, repeated dereferencing of [rbx+8] suggests that the field may be a pointer, while comparisons of byte [rbx+16] suggest a flag or Boolean field. Reconstructed structures make decompiled malware code significantly easier to understand.

The instructions execute as follows:

1. mov eax, 5 places 5 in EAX.
2. mov ecx, 3 places 3 in ECX.
3. add eax, ecx computes:

4. cmp eax, 8 conceptually computes for flag calculation. The result is zero, so the Zero Flag is set.
5. jne not_equal branches only if the Zero Flag is clear. Since it is set, the branch is not taken.
6. xor edx, edx clears EDX, making its value zero.
7. jmp finish skips the instruction at not_equal.
8. nop performs no data operation.

The final relevant register values are:

• EAX = 8
• ECX = 3
• EDX = 0

At a high level, the sequence adds two values, tests whether the result equals 8, and stores zero in EDX for equality or one for inequality. It is equivalent to setting a Boolean result based on the comparison.

An integrated strategy should combine safe execution, debugging, memory analysis, and network observation.

1. Initial triage:

   • Calculate the DLL's hashes.
   • Confirm that it is a 64-bit Portable Executable.
   • Examine sections, entropy, imports, exports, resources, and signatures.
   • High entropy and limited imports may indicate packing.

2. Determine activation requirements:

   • Identify exported functions, ordinals, expected arguments, and the intended host process.
   • Avoid calling unknown exports with arbitrary parameters.

3. Prepare an isolated x64 environment:

   • Use a 64-bit virtual machine, debugger, process monitor, packet capture tool, and simulated network services.
   • Record a clean snapshot and baseline.

4. Trace execution:

   • Break at the DLL entry point and relevant exported functions.
   • Monitor memory allocation, memory-protection changes, thread creation, process opening, and remote memory operations.

5. Identify unpacking:

   • Look for data copied or decrypted into writable memory followed by a change to executable permissions.
   • Set breakpoints when execution transfers to the newly unpacked region.
   • Dump the unpacked image and rebuild imports if necessary.

6. Investigate process injection:

   • Identify the target process.
   • Record allocated addresses, written buffers, thread start addresses, mapped sections, and process-hollowing behavior.
   • Dump the injected code from the target process.

7. Analyse network communication:

   • Capture DNS, IP addresses, ports, protocols, request patterns, encryption routines, and beacon intervals.
   • Simulate responses to reveal additional commands and payloads.

8. Apply x64 knowledge:

   • Trace arguments in the correct calling-convention registers.
   • Account for 8-byte pointers, stack alignment, additional registers, and RIP-relative addressing.

9. Produce defensive output:

   • Document hashes, domains, IP addresses, mutexes, file paths, registry keys, target processes, memory patterns, and behavioral detections.
   • Recommend network blocks, endpoint rules, memory-scanning signatures, and monitoring for the observed injection chain.

This approach reveals both the unpacked code and the malware's actual effects while preserving containment and evidential quality.