Back to Subject

Unit 4: Foundations of Reinforcement Learning - Practice Quiz

INT423 — Machine Learning-Ii 50 Questions
0 Correct 0 Wrong 50 Left
0/50

1 What is the primary goal of an agent in Reinforcement Learning?

A. To maximize the cumulative reward over time
B. To classify data into distinct categories
C. To minimize the error in prediction
D. To cluster similar data points together

2 Which of the following elements is NOT a core component of a Reinforcement Learning system?

A. Reward Signal
B. Agent
C. Environment
D. Supervisor Labels

3 In the context of RL, what does the 'Markov Property' imply about the state?

A. The state is independent of the actions taken.
B. The future depends on the past history of all states.
C. The state transition is always deterministic.
D. The future depends only on the current state and action, not the history.

4 What does a 'Policy' represent in Reinforcement Learning?

A. A mapping from perceived states to actions to be taken
B. The probability of moving from one state to another
C. The calculation of total future reward
D. The immediate reward received after an action

5 In an MDP, what does the discount factor (gamma, γ) determine?

A. The importance of future rewards relative to immediate rewards
B. The probability of choosing a random action
C. The learning rate of the algorithm
D. The magnitude of the transition probability

6 Which tuple represents a finite Markov Decision Process (MDP)?

A. (S, A, R, γ)
B. (S, A, P, R)
C. (S, P, R, γ)
D. (S, A, P, R, γ)

7 What is the difference between a Value Function V(s) and an Action-Value Function Q(s, a)?

A. V(s) includes the action taken, while Q(s, a) does not.
B. There is no mathematical difference.
C. V(s) is for continuous spaces, Q(s, a) is for discrete spaces.
D. V(s) estimates the return of a state, while Q(s, a) estimates the return of taking an action in a state.

8 The Bellman Equation expresses the relationship between:

A. The agent and the environment
B. The current reward and the previous reward
C. The value of a state and the values of its successor states
D. The exploration rate and the exploitation rate

9 Which method requires the completion of an entire episode before updating the value estimates?

A. Dynamic Programming
B. Temporal Difference Learning
C. Q-Learning
D. Monte Carlo Learning

10 What is 'Bootstrapping' in the context of Temporal Difference (TD) learning?

A. Updating an estimate based on another estimate
B. Running multiple episodes in parallel
C. Using random weights for initialization
D. Restarting the learning process from scratch

11 In the Exploration vs. Exploitation trade-off, what does 'Exploitation' refer to?

A. Randomly selecting actions
B. Choosing the action currently believed to be the best
C. Trying new actions to find better rewards
D. Ignoring the reward signal

12 What is the Epsilon-Greedy strategy?

A. Choosing the action with the lowest value
B. Choosing the best action most of the time, but a random action with probability epsilon
C. Always choosing a random action
D. Always choosing the best action

13 Which equation represents the Bellman Optimality Equation for V*(s)?

A. V*(s) = max_a (R)
B. V(s) = max_a Σ P(s'|s,a) [R + γV(s')]
C. V(s) = Σ P(s'|s,a) [R + γV(s')]
D. V(s) = R + γV(s')

14 In Monte Carlo learning, what is the difference between 'First-visit' and 'Every-visit' MC?

A. Every-visit is for continuous tasks; First-visit is for episodic tasks.
B. First-visit uses bootstrapping; Every-visit does not.
C. First-visit updates only the first time a state is visited in an episode; Every-visit updates for all visits.
D. First-visit is faster; Every-visit is slower.

15 What is the TD(0) update rule for V(s)?

A. V(s) ← V(s) + α [R + γV(s') - V(s)]
B. V(s) ← V(s) + α [Gt - V(s)]
C. V(s) ← max(Q(s, a))
D. V(s) ← R + γV(s')

16 Which of the following describes a 'Model-Free' RL approach?

A. The agent requires a supervisor to model the environment.
B. The agent plans by simulating future states.
C. The agent learns the transition probabilities and reward function explicitly.
D. The agent learns a policy or value function directly from experience without knowing the environment's dynamics.

17 What is the return (Gt) in Reinforcement Learning?

A. The total discounted sum of future rewards
B. The final reward at the terminal state
C. The average reward of the episode
D. The immediate reward received

18 If the discount factor γ is 0, the agent is:

A. Myopic (cares only about immediate reward)
B. Far-sighted (cares only about long-term reward)
C. Optimal
D. Random

19 What is an 'Episodic Task'?

A. A task that breaks interaction into subsequences called episodes which end in a terminal state
B. A task where the reward is always zero
C. A task that continues indefinitely without end
D. A task with only one state

20 Comparing MC and TD methods, which statement is true regarding variance and bias?

A. MC and TD have identical variance and bias properties.
B. TD has high variance, low bias.
C. MC has low variance, high bias.
D. MC has high variance, zero bias; TD has low variance, some bias.

21 In the context of the Bellman Equation, what is p(s', r | s, a)?

A. The dynamics function (probability of next state and reward)
B. The policy function
C. The value function
D. The discount factor

22 What is a 'Deterministic Policy'?

A. A policy that changes over time
B. A policy that maps a state to a specific, single action
C. A policy that ignores the state
D. A policy that maps a state to a probability distribution over actions

23 Which learning method performs updates step-by-step without waiting for the episode to end?

A. Temporal Difference
B. Batch Learning
C. Monte Carlo
D. Exhaustive Search

24 The term 'Greedy Action' implies:

A. Selecting a random action
B. Selecting the action with the highest estimated value
C. Selecting the action with the lowest cost
D. Selecting an action that maximizes exploration

25 What is the role of the Value Function?

A. To define the rules of the environment
B. To generate random numbers
C. To store the immediate reward
D. To predict how good it is to be in a specific state

26 Which of the following is NOT a challenge in Reinforcement Learning?

A. Credit Assignment Problem
B. Availability of labeled training data
C. Exploration vs Exploitation
D. Delayed Reward

27 In the equation Gt = R{t+1} + γR{t+2} + γ^2R{t+3} + ... , what is G_t?

A. The transition probability
B. The policy
C. The discounted return
D. The value function

28 Why is exploration necessary in Reinforcement Learning?

A. To avoid overfitting
B. To speed up the calculation of the Bellman equation
C. To minimize the discount factor
D. To discover states and actions that might yield higher rewards than the current best known options

29 Which of the following is an Off-Policy control method?

A. SARSA
B. Q-Learning
C. Standard TD Prediction
D. Monte Carlo Policy Evaluation

30 What is the 'Credit Assignment Problem' in RL?

A. Determining which past action is responsible for a current reward
B. Assigning monetary value to states
C. Calculating the computational cost of the algorithm
D. Deciding how much memory to allocate

31 An optimal policy π* is defined as:

A. A policy that is better than or equal to all other policies
B. A policy that explores every state
C. A policy that reaches the terminal state fastest
D. A policy with zero discount factor

32 In Q-Learning, the target value for the update is:

A. R + γ max_a' Q(s', a')
B. V(s')
C. R + γ Q(s', a')
D. The actual return Gt

33 Monte Carlo methods are applicable only to:

A. Deterministic environments
B. Tasks with known models
C. Continuous tasks
D. Episodic tasks

34 What does SARSA stand for?

A. State-Action-Return-State-Average
B. System-Action-Reward-System-Action
C. Search-And-Retrieve-Sorted-Arrays
D. State-Action-Reward-State-Action

35 When does the 'Optimistic Initial Values' technique encourage exploration?

A. When initial value estimates are set higher than the expected maximum reward
B. When initial value estimates are set very low
C. When the discount factor is 1
D. When epsilon is set to 0

36 Which Bellman equation is linear?

A. Both
B. Bellman Optimality Equation
C. Bellman Expectation Equation
D. Neither

37 What is the main advantage of TD learning over Monte Carlo?

A. It can learn online during an episode
B. It is unbiased
C. It requires less memory
D. It works better for non-Markov environments

38 The sequence of states and actions S0, A0, R1, S1, A1, R2... is called:

A. A Model
B. A Policy
C. A Trajectory
D. A Value Function

39 In a stochastic environment:

A. Taking an action leads to a next state based on a probability distribution
B. Taking an action always leads to the same next state
C. Rewards are not provided
D. The agent cannot learn

40 Which algorithm is considered 'On-Policy'?

A. Max-Q
B. Off-Policy MC
C. SARSA
D. Q-Learning

41 The quantity R + γV(s') is often called the:

A. Monte Carlo Return
B. TD Error
C. TD Target
D. Exploration Bonus

42 If an agent uses a pure Greedy strategy (epsilon=0), it:

A. Never explores
B. Explores 50% of the time
C. Alternates between exploration and exploitation
D. Explores randomly

43 The State-Value function V_π(s) is the expected return starting from state s and then following:

A. A random policy
B. Policy π
C. The greedy policy
D. The optimal policy

44 Dynamic Programming (DP) methods in RL assume:

A. Rewards are always positive
B. The environment is unknown
C. Monte Carlo sampling is used
D. A perfect model of the environment is available

45 What is 'Policy Improvement'?

A. Collecting more data
B. Increasing the learning rate
C. Making a new policy that is greedy with respect to the current value function
D. Calculating the value function for a policy

46 Upper Confidence Bound (UCB) is an algorithm used to handle:

A. The Exploration-Exploitation Dilemma
B. Discount Factors
C. Continuous State Spaces
D. The Bellman Equation

47 In the TD error equation δ = R + γV(s') - V(s), what does δ represent?

A. The difference between the target and the current estimate
B. The learning rate
C. The total return
D. The probability of the next state

48 A key distinction between Reinforcement Learning and Unsupervised Learning is:

A. RL is for clustering
B. RL uses labeled data
C. RL maximizes a reward signal, Unsupervised Learning finds hidden structure
D. Unsupervised Learning uses a supervisor

49 Which of the following creates a 'Continuous Task'?

A. A robot balancing only for 10 seconds
B. Go
C. An automated stock trading agent operating indefinitely
D. Chess

50 Policy Iteration consists of two alternating steps:

A. Policy Evaluation and Policy Improvement
B. Monte Carlo and TD
C. Prediction and Control
D. Exploration and Exploitation
Previous MCQ
Unit 3
Next MCQ
Unit 5