Back to Subject

Unit 3: Learning, Adaptation and Fine-Tuning for LLMs and Agents - Practice Quiz

CSE473 — Large Language Models And Agentic Ai 60 Questions
0 Correct 0 Wrong 60 Left
0/60

1 What is the main purpose of supervised fine-tuning for a large language model?

Supervised fine-tuning and instruction tuning Easy
A. To convert text into image pixels
B. To adapt the model using labeled examples
C. To remove all parameters from the model
D. To eliminate the need for training data

2 What does instruction tuning primarily teach a language model to do?

Supervised fine-tuning and instruction tuning Easy
A. Replace its tokenizer during every request
B. Store training examples without processing them
C. Follow natural-language instructions more effectively
D. Compress every response into one word

3 Which data format is commonly used for instruction tuning?

Supervised fine-tuning and instruction tuning Easy
A. Instructions paired with suitable responses
B. Random numbers paired with timestamps
C. Images paired with file sizes
D. Passwords paired with account names

4 Supervised fine-tuning normally updates a model by comparing its output with what?

Supervised fine-tuning and instruction tuning Easy
A. An unrelated model size
B. A labeled target answer
C. A fixed hardware temperature
D. A random network address

5 What does LoRA stand for in large language model adaptation?

Parameter-efficient techniques (LoRA) Easy
A. Layer Rotation Algorithm
B. Local Response Analysis
C. Long-Range Attention
D. Low-Rank Adaptation

6 What is a key advantage of LoRA?

Parameter-efficient techniques (LoRA) Easy
A. It converts every model into a search engine
B. It removes the need for a base model
C. It trains fewer parameters than full fine-tuning
D. It guarantees perfect responses after one update

7 During typical LoRA training, what happens to most original model weights?

Parameter-efficient techniques (LoRA) Easy
A. They remain frozen
B. They are randomly replaced
C. They are permanently deleted
D. They become text tokens

8 What is quantization in the context of deploying language models?

Quantization for deployment Easy
A. Adding more layers to every model block
B. Using lower-precision numbers for model values
C. Sorting training examples by sentence length
D. Translating all prompts into another language

9 Which deployment benefit is commonly provided by quantization?

Quantization for deployment Easy
A. A larger vocabulary
B. Lower memory usage
C. More training labels
D. Longer input sentences

10 What is a possible trade-off of aggressive quantization?

Quantization for deployment Easy
A. Unlimited model context length
B. Automatic creation of labels
C. Slightly reduced model accuracy
D. Perfect output consistency

11 In Q-learning, what does a Q-value estimate?

Reinforcement learning foundations (Q-learning basics) Easy
A. The expected return for a state-action pair
B. The physical size of the learning agent
C. The length of an instruction-tuning dataset
D. The number of states in an environment

12 What is the basic goal of a Q-learning agent?

Reinforcement learning foundations (Q-learning basics) Easy
A. To minimize the number of states
B. To avoid every available action
C. To copy a fixed sequence forever
D. To maximize cumulative reward

13 What does an action represent in reinforcement learning?

Reinforcement learning foundations (Q-learning basics) Easy
A. A label assigned before training
B. A file containing model weights
C. A choice made by the agent
D. A parameter stored in a tokenizer

14 Why might a Q-learning agent explore unfamiliar actions?

Reinforcement learning foundations (Q-learning basics) Easy
A. To discover actions with better rewards
B. To erase its previous learning
C. To reduce the available state space
D. To prevent rewards from being recorded

15 What is reward shaping?

Reward shaping Easy
A. Removing all feedback from the environment
B. Converting rewards into training instructions
C. Adding guidance through modified reward signals
D. Changing the model's physical hardware

16 Why is reward shaping often used during agent training?

Reward shaping Easy
A. To make every action equally valuable
B. To provide more frequent learning guidance
C. To replace the environment with a dataset
D. To prevent the agent from observing states

17 What can happen if a shaped reward is poorly designed?

Reward shaping Easy
A. The model will automatically increase precision
B. The environment will lose all possible states
C. The agent will always find the ideal policy
D. The agent may exploit unintended shortcuts

18 What type of information do humans commonly provide when guiding an agent through feedback?

Learning from human feedback for agent behavior Easy
A. Preferences between possible responses
B. Hardware instructions for memory allocation
C. Random changes to model architecture
D. Exact values for every model weight

19 What is the role of a reward model in learning from human feedback?

Learning from human feedback for agent behavior Easy
A. To predict which outputs humans prefer
B. To store the entire internet locally
C. To tokenize every prompt twice
D. To select the model's numeric precision

20 What is a common goal of learning from human feedback?

Learning from human feedback for agent behavior Easy
A. Aligning agent behavior with human preferences
B. Increasing model size after every response
C. Replacing language data with random values
D. Removing all safety constraints from the agent

21 A model fine-tuned only on a small customer-support dataset becomes less capable on general language tasks. Which strategy most directly reduces this problem?

Supervised fine-tuning and instruction tuning Medium
A. Increase the learning rate throughout all training epochs
B. Remove instruction prompts from the fine-tuning examples
C. Train repeatedly on the smallest support-data category
D. Mix representative general examples into the fine-tuning data

22 During instruction tuning, each training example contains an instruction, optional context, and target response. Which tokens should usually contribute most directly to the supervised loss?

Supervised fine-tuning and instruction tuning Medium
A. Primarily the tokens in the target response
B. Only padding tokens after the target response
C. Primarily separator tokens between sequence fields
D. Only instruction tokens before the context

23 Two instruction-tuning datasets give conflicting answers to nearly identical prompts. What is the most appropriate action before training?

Supervised fine-tuning and instruction tuning Medium
A. Increase generation temperature during model training
B. Resolve conflicts using a consistent annotation policy
C. Duplicate both answers to increase their frequency
D. Randomly remove half of all training prompts

24 A team wants to estimate whether instruction tuning generalizes to unseen user requests. Which evaluation split is most informative?

Supervised fine-tuning and instruction tuning Medium
A. Training responses evaluated without their original prompts
B. Unseen prompts representing the intended task distribution
C. Training prompts repeated with shuffled answer order
D. Random tokens sampled from the model vocabulary

25 A LoRA adapter uses updates of the form . What is the main effect of increasing the rank ?

Parameter-efficient techniques (LoRA) Medium
A. It changes every base-model weight into an integer
B. It freezes more layers in the original transformer
C. It reduces adapter capacity and trainable parameter count
D. It increases adapter capacity and trainable parameter count

26 A service must switch rapidly among adapters for legal, medical, and technical tasks while sharing one base model. Which deployment design best supports this?

Parameter-efficient techniques (LoRA) Medium
A. Combine all adapters into random transformer layers
B. Delete the base weights after training each adapter
C. Load one shared base model and swap task adapters
D. Retrain a separate base model for every request

27 Before exporting a model with one fixed LoRA adapter, the team merges the adapter into the base weights. What is the primary deployment benefit?

Parameter-efficient techniques (LoRA) Medium
A. It removes separate adapter operations during inference
B. It permits switching adapters without storing their weights
C. It restores the base model to random initialization
D. It automatically converts all weights to four bits

28 LoRA is applied to attention projection matrices while the base model remains frozen. Which parameters receive gradient-based updates?

Parameter-efficient techniques (LoRA) Medium
A. Only the tokenizer's fixed vocabulary identifiers
B. The low-rank adapter matrices attached to projections
C. Every parameter in the frozen transformer backbone
D. Only the optimizer's non-trainable configuration values

29 Ignoring metadata and scaling factors, approximately how does converting model weights from FP32 to INT8 affect weight storage?

Quantization for deployment Medium
A. It leaves storage approximately unchanged
B. It reduces storage to about one half
C. It reduces storage to about one eighth
D. It reduces storage to about one quarter

30 A post-training quantized model loses substantial accuracy because a few activation values are extreme outliers. Which response is most appropriate?

Quantization for deployment Medium
A. Increase the tokenizer vocabulary without recalibration
B. Use calibrated ranges or selective higher-precision operations
C. Map every activation to one shared integer value
D. Remove all scaling factors from quantized computations

31 A model must run at low precision, but post-training quantization causes unacceptable degradation. Which technique can adapt the model to simulated quantization error?

Quantization for deployment Medium
A. Instruction tuning without any quantization simulation
B. Reward shaping with a constant terminal reward
C. Tokenization-aware decoding with unlimited precision
D. Quantization-aware training with fake quantization

32 For a Q-learning update, let , , , , and . What is the updated value?

Reinforcement learning foundations (Q-learning basics) Medium
A.
B.
C.
D.

33 An agent receives reward and enters a terminal state. If and , what is the updated value when terminal states have no bootstrap value?

Reinforcement learning foundations (Q-learning basics) Medium
A.
B.
C.
D.

34 A Q-learning agent uses an -greedy policy. If is reduced too quickly near the start of training, what is the most likely consequence?

Reinforcement learning foundations (Q-learning basics) Medium
A. The agent may exploit poor actions before exploring alternatives
B. The agent will update every action with identical rewards
C. The environment will become fully deterministic during training
D. The discount factor will automatically increase above one

35 Which shaping reward is known to preserve the optimal policy under standard conditions when added to the environment reward?

Reward shaping Medium
A.
B.
C.
D.

36 A delivery agent earns a small reward whenever it moves closer to a package. It learns to oscillate between two locations and repeatedly collect reward. What does this demonstrate?

Reward shaping Medium
A. Quantization error caused by low-precision activations
B. Reward hacking caused by an exploitable shaping signal
C. Policy preservation caused by terminal-state bootstrapping
D. Catastrophic forgetting caused by supervised fine-tuning

37 A navigation task gives a reward only when the destination is reached, so successful experiences are rare. Which shaping approach is most useful?

Reward shaping Medium
A. Penalize the agent only after successful episodes
B. Remove all observations related to the destination
C. Add progress rewards based on reduced goal distance
D. Replace every reward with the same positive constant

38 Human reviewers consistently choose the better of two agent trajectories. How is this feedback commonly converted into a trainable learning signal?

Learning from human feedback for agent behavior Medium
A. Copy every preferred trajectory into the tokenizer vocabulary
B. Assign rewards using only each trajectory's token count
C. Quantize both trajectories and compare their memory sizes
D. Train a reward model to predict pairwise preferences

39 During feedback-based policy optimization, an agent begins producing unnatural outputs that score highly under the learned reward model. Which technique most directly limits excessive policy drift?

Learning from human feedback for agent behavior Medium
A. Remove the reference model from all evaluations
B. Increase reward-model scores by a fixed constant
C. Train only on the highest-scoring generated token
D. Add a KL penalty relative to a reference policy

40 An assistant can call external tools, and reviewers care about both answer correctness and safe tool use. What should the preference data contain?

Learning from human feedback for agent behavior Medium
A. Complete trajectories showing answers and tool-call decisions
B. Only tool names without arguments or resulting observations
C. Random prompts without reviewer comparisons or ratings
D. Only final answers with all tool actions removed

41 An instruction-tuning dataset concatenates multiple examples into each fixed-length sequence. Which attention and loss configuration best prevents unintended cross-example learning?

Supervised fine-tuning and instruction tuning Hard
A. Use causal attention globally and compute loss on every response and padding token
B. Use causal attention globally and compute loss only on each example's instruction tokens
C. Use block-diagonal attention and compute loss only on each example's response tokens
D. Use bidirectional attention and compute loss on every non-padding token

42 A model is fine-tuned with teacher forcing and obtains low token-level validation loss, but its long generated answers frequently drift after an early mistake. Which explanation best accounts for this discrepancy?

Supervised fine-tuning and instruction tuning Hard
A. Teacher forcing evaluates predictions under gold prefixes rather than model-generated prefixes
B. Teacher forcing optimizes sequence-level accuracy rather than token-level likelihood
C. Teacher forcing removes causal masking from response tokens during optimization
D. Teacher forcing trains recovery only from prefixes sampled by the model itself

43 Two instruction datasets contain identical response tokens, but one repeats long prompts before every response. Training loss is averaged over all non-padding tokens without prompt masking. What is the most likely consequence?

Supervised fine-tuning and instruction tuning Hard
A. All examples retain equal influence because token averaging normalizes each sequence
B. Prompt length affects memory use but cannot affect the optimization objective
C. Short-prompt examples receive disproportionate influence through response-only gradients
D. Long-prompt examples receive disproportionate influence through prompt-reconstruction gradients

44 For multi-turn assistant fine-tuning, the loss is applied to every assistant turn but masked on system and user turns. What behavior does this objective directly train?

Supervised fine-tuning and instruction tuning Hard
A. Reconstruct the complete dialogue using bidirectional context from later turns
B. Predict each user turn conditioned only on the preceding assistant response
C. Predict each assistant turn conditioned on all preceding visible conversation tokens
D. Optimize only the final assistant turn while treating earlier turns as labels

45 A frozen linear layer has . LoRA represents its update as with rank . Ignoring biases, how many trainable parameters are added?

Parameter-efficient techniques (LoRA) Hard
A.
B.
C.
D.

46 Two independently trained rank- LoRA adapters for the same frozen layer are merged by adding both updates. What is the tightest general upper bound on the rank of the combined update?

Parameter-efficient techniques (LoRA) Hard
A.
B.
C.
D. regardless of

47 A LoRA implementation uses . If rank changes from to while and the typical scale of entries in and remain fixed, what is the purpose of the divisor ?

Parameter-efficient techniques (LoRA) Hard
A. It exactly preserves the matrix rank of the update
B. It guarantees identical gradients for every possible rank
C. It moderates update magnitude as more rank components are added
D. It converts the low-rank update into an orthogonal projection

48 A weight matrix contains a few extreme outlier values confined to certain output channels. Under the same bit width, which scheme should usually reduce quantization error most effectively?

Quantization for deployment Hard
A. One symmetric scale shared by the entire model
B. One affine scale shared by the entire matrix
C. A single scale estimated only from the median weight
D. Separate scales for individual channels or small groups

49 Post-training activation quantization is calibrated only on short, formal prompts, but deployment uses long conversational prompts. What is the central technical risk?

Quantization for deployment Hard
A. Deployment activations may exceed calibrated ranges and undergo clipping
B. Deployment weights will change because activation scales are recalculated
C. Long prompts necessarily increase the numerical precision of each activation
D. Conversational tokens cannot be represented by the original tokenizer

50 An autoregressive decoder stores key-value tensors for every generated token. Which change most directly reduces KV-cache memory growth while leaving model weights unchanged?

Quantization for deployment Hard
A. Merge LoRA adapters into the frozen projection weights
B. Quantize cached keys and values to a lower bit width
C. Quantize only the embedding matrix to a lower bit width
D. Increase the group size used for weight quantization

51 For a non-terminal transition , tabular Q-learning uses learning rate and discount . Which update is correct?

Reinforcement learning foundations (Q-learning basics) Hard
A.
B.
C.
D.

52 A transition ends an episode because the agent reaches a genuine terminal state and receives reward . What target should standard Q-learning use for that transition?

Reinforcement learning foundations (Q-learning basics) Hard
A.
B.
C.
D.

53 Why can the operator in noisy Q-learning estimates cause systematic overestimation?

Reinforcement learning foundations (Q-learning basics) Hard
A. It forces all state-action values to converge to the same fixed point
B. It tends to select actions whose estimation errors are unusually positive
C. It removes rewards associated with actions not chosen by the behavior policy
D. It averages positive and negative estimation errors before action selection

54 An LLM agent observes only the latest user message, while optimal action choice depends on an unobserved earlier constraint. Why may tabular Q-learning on the latest message fail even with unlimited visits?

Reinforcement learning foundations (Q-learning basics) Hard
A. The observation aliases histories requiring different optimal actions
B. Off-policy updates require rewards to be differentiable functions
C. The discount factor prevents learning from any delayed reward
D. Q-learning cannot represent deterministic policies in finite action spaces

55 In a discounted MDP, which shaping reward preserves the set of optimal policies under the standard potential-based shaping result?

Reward shaping Hard
A.
B.
C.
D.

56 An agent receives each time it calls a diagnostic tool and for completing the task. It learns to call the tool repeatedly without finishing. Which redesign best addresses the reward-hacking mechanism?

Reward shaping Hard
A. Remove the completion reward but retain the per-call reward
B. Increase the reward for every diagnostic tool call equally
C. Increase the discount factor while leaving rewards unchanged
D. Reward verified information gain and penalize redundant tool calls

57 A designer applies while the learner discounts returns using . What guarantee is generally lost?

Reward shaping Hard
A. Existence of any finite state-action value function
B. Preservation of the original MDP's optimal policies
C. Convergence of every policy to uniform random actions
D. Ability to represent rewards using scalar values

58 A preference model uses . Which transformation of leaves every predicted preference probability unchanged?

Learning from human feedback for agent behavior Hard
A. for a constant
B. for every completion
C. for every completion
D. for every completion

59 An RLHF objective maximizes . What is the primary role of the KL term?

Learning from human feedback for agent behavior Hard
A. Eliminate all sampling variance from policy-gradient estimates
B. Constrain policy drift from the reference while optimizing learned reward
C. Force the policy to assign equal probability to every response
D. Guarantee that the learned reward equals true human utility

60 Human raters evaluate only final outcomes of long agent trajectories, but many intermediate tool decisions affect success. Which learning challenge is most directly created?

Learning from human feedback for agent behavior Hard
A. Matrix-rank selection for the reward model's adapters
B. Tokenization ambiguity in the human preference labels
C. Temporal credit assignment across intermediate actions
D. Integer overflow in the policy's quantized embeddings
Previous MCQ
Unit 2
Next MCQ
Unit 4