Back to Subject

Unit 4: Foundations of Reinforcement Learning - Practice Quiz

INT423 — Machine Learning-Ii 50 Questions
0 Correct 0 Wrong 50 Left
0/50

1 What is the primary goal of an agent in Reinforcement Learning?

A. To minimize the error in prediction
B. To maximize the cumulative reward over time
C. To cluster similar data points together
D. To classify data into distinct categories

2 Which of the following elements is NOT a core component of a Reinforcement Learning system?

A. Environment
B. Agent
C. Reward Signal
D. Supervisor Labels

3 In the context of RL, what does the 'Markov Property' imply about the state?

A. The state is independent of the actions taken.
B. The state transition is always deterministic.
C. The future depends only on the current state and action, not the history.
D. The future depends on the past history of all states.

4 What does a 'Policy' represent in Reinforcement Learning?

A. A mapping from perceived states to actions to be taken
B. The calculation of total future reward
C. The immediate reward received after an action
D. The probability of moving from one state to another

5 In an MDP, what does the discount factor (gamma, γ) determine?

A. The importance of future rewards relative to immediate rewards
B. The learning rate of the algorithm
C. The probability of choosing a random action
D. The magnitude of the transition probability

6 Which tuple represents a finite Markov Decision Process (MDP)?

A. (S, A, P, R, γ)
B. (S, A, P, R)
C. (S, P, R, γ)
D. (S, A, R, γ)

7 What is the difference between a Value Function V(s) and an Action-Value Function Q(s, a)?

A. V(s) estimates the return of a state, while Q(s, a) estimates the return of taking an action in a state.
B. V(s) is for continuous spaces, Q(s, a) is for discrete spaces.
C. There is no mathematical difference.
D. V(s) includes the action taken, while Q(s, a) does not.

8 The Bellman Equation expresses the relationship between:

A. The exploration rate and the exploitation rate
B. The current reward and the previous reward
C. The value of a state and the values of its successor states
D. The agent and the environment

9 Which method requires the completion of an entire episode before updating the value estimates?

A. Dynamic Programming
B. Monte Carlo Learning
C. Q-Learning
D. Temporal Difference Learning

10 What is 'Bootstrapping' in the context of Temporal Difference (TD) learning?

A. Restarting the learning process from scratch
B. Using random weights for initialization
C. Updating an estimate based on another estimate
D. Running multiple episodes in parallel

11 In the Exploration vs. Exploitation trade-off, what does 'Exploitation' refer to?

A. Randomly selecting actions
B. Ignoring the reward signal
C. Choosing the action currently believed to be the best
D. Trying new actions to find better rewards

12 What is the Epsilon-Greedy strategy?

A. Choosing the action with the lowest value
B. Choosing the best action most of the time, but a random action with probability epsilon
C. Always choosing the best action
D. Always choosing a random action

13 Which equation represents the Bellman Optimality Equation for V*(s)?

A. V*(s) = max_a (R)
B. V(s) = max_a Σ P(s'|s,a) [R + γV(s')]
C. V(s) = Σ P(s'|s,a) [R + γV(s')]
D. V(s) = R + γV(s')

14 In Monte Carlo learning, what is the difference between 'First-visit' and 'Every-visit' MC?

A. First-visit uses bootstrapping; Every-visit does not.
B. First-visit updates only the first time a state is visited in an episode; Every-visit updates for all visits.
C. First-visit is faster; Every-visit is slower.
D. Every-visit is for continuous tasks; First-visit is for episodic tasks.

15 What is the TD(0) update rule for V(s)?

A. V(s) ← R + γV(s')
B. V(s) ← V(s) + α [Gt - V(s)]
C. V(s) ← max(Q(s, a))
D. V(s) ← V(s) + α [R + γV(s') - V(s)]

16 Which of the following describes a 'Model-Free' RL approach?

A. The agent learns a policy or value function directly from experience without knowing the environment's dynamics.
B. The agent plans by simulating future states.
C. The agent learns the transition probabilities and reward function explicitly.
D. The agent requires a supervisor to model the environment.

17 What is the return (Gt) in Reinforcement Learning?

A. The average reward of the episode
B. The final reward at the terminal state
C. The immediate reward received
D. The total discounted sum of future rewards

18 If the discount factor γ is 0, the agent is:

A. Myopic (cares only about immediate reward)
B. Far-sighted (cares only about long-term reward)
C. Optimal
D. Random

19 What is an 'Episodic Task'?

A. A task that breaks interaction into subsequences called episodes which end in a terminal state
B. A task where the reward is always zero
C. A task that continues indefinitely without end
D. A task with only one state

20 Comparing MC and TD methods, which statement is true regarding variance and bias?

A. MC and TD have identical variance and bias properties.
B. MC has high variance, zero bias; TD has low variance, some bias.
C. TD has high variance, low bias.
D. MC has low variance, high bias.

21 In the context of the Bellman Equation, what is p(s', r | s, a)?

A. The value function
B. The policy function
C. The dynamics function (probability of next state and reward)
D. The discount factor

22 What is a 'Deterministic Policy'?

A. A policy that maps a state to a probability distribution over actions
B. A policy that maps a state to a specific, single action
C. A policy that changes over time
D. A policy that ignores the state

23 Which learning method performs updates step-by-step without waiting for the episode to end?

A. Temporal Difference
B. Batch Learning
C. Monte Carlo
D. Exhaustive Search

24 The term 'Greedy Action' implies:

A. Selecting the action with the lowest cost
B. Selecting the action with the highest estimated value
C. Selecting an action that maximizes exploration
D. Selecting a random action

25 What is the role of the Value Function?

A. To predict how good it is to be in a specific state
B. To store the immediate reward
C. To generate random numbers
D. To define the rules of the environment

26 Which of the following is NOT a challenge in Reinforcement Learning?

A. Availability of labeled training data
B. Credit Assignment Problem
C. Delayed Reward
D. Exploration vs Exploitation

27 In the equation Gt = R{t+1} + γR{t+2} + γ^2R{t+3} + ... , what is G_t?

A. The value function
B. The policy
C. The transition probability
D. The discounted return

28 Why is exploration necessary in Reinforcement Learning?

A. To speed up the calculation of the Bellman equation
B. To minimize the discount factor
C. To discover states and actions that might yield higher rewards than the current best known options
D. To avoid overfitting

29 Which of the following is an Off-Policy control method?

A. Q-Learning
B. Monte Carlo Policy Evaluation
C. Standard TD Prediction
D. SARSA

30 What is the 'Credit Assignment Problem' in RL?

A. Deciding how much memory to allocate
B. Determining which past action is responsible for a current reward
C. Assigning monetary value to states
D. Calculating the computational cost of the algorithm

31 An optimal policy π* is defined as:

A. A policy that explores every state
B. A policy that reaches the terminal state fastest
C. A policy that is better than or equal to all other policies
D. A policy with zero discount factor

32 In Q-Learning, the target value for the update is:

A. R + γ Q(s', a')
B. The actual return Gt
C. R + γ max_a' Q(s', a')
D. V(s')

33 Monte Carlo methods are applicable only to:

A. Deterministic environments
B. Continuous tasks
C. Tasks with known models
D. Episodic tasks

34 What does SARSA stand for?

A. State-Action-Return-State-Average
B. System-Action-Reward-System-Action
C. Search-And-Retrieve-Sorted-Arrays
D. State-Action-Reward-State-Action

35 When does the 'Optimistic Initial Values' technique encourage exploration?

A. When the discount factor is 1
B. When initial value estimates are set very low
C. When epsilon is set to 0
D. When initial value estimates are set higher than the expected maximum reward

36 Which Bellman equation is linear?

A. Both
B. Bellman Optimality Equation
C. Bellman Expectation Equation
D. Neither

37 What is the main advantage of TD learning over Monte Carlo?

A. It works better for non-Markov environments
B. It can learn online during an episode
C. It is unbiased
D. It requires less memory

38 The sequence of states and actions S0, A0, R1, S1, A1, R2... is called:

A. A Value Function
B. A Trajectory
C. A Model
D. A Policy

39 In a stochastic environment:

A. Taking an action leads to a next state based on a probability distribution
B. Taking an action always leads to the same next state
C. Rewards are not provided
D. The agent cannot learn

40 Which algorithm is considered 'On-Policy'?

A. Off-Policy MC
B. Max-Q
C. SARSA
D. Q-Learning

41 The quantity R + γV(s') is often called the:

A. Monte Carlo Return
B. TD Error
C. TD Target
D. Exploration Bonus

42 If an agent uses a pure Greedy strategy (epsilon=0), it:

A. Alternates between exploration and exploitation
B. Never explores
C. Explores 50% of the time
D. Explores randomly

43 The State-Value function V_π(s) is the expected return starting from state s and then following:

A. The greedy policy
B. Policy π
C. A random policy
D. The optimal policy

44 Dynamic Programming (DP) methods in RL assume:

A. The environment is unknown
B. Monte Carlo sampling is used
C. A perfect model of the environment is available
D. Rewards are always positive

45 What is 'Policy Improvement'?

A. Calculating the value function for a policy
B. Making a new policy that is greedy with respect to the current value function
C. Collecting more data
D. Increasing the learning rate

46 Upper Confidence Bound (UCB) is an algorithm used to handle:

A. The Bellman Equation
B. Discount Factors
C. Continuous State Spaces
D. The Exploration-Exploitation Dilemma

47 In the TD error equation δ = R + γV(s') - V(s), what does δ represent?

A. The probability of the next state
B. The difference between the target and the current estimate
C. The learning rate
D. The total return

48 A key distinction between Reinforcement Learning and Unsupervised Learning is:

A. RL uses labeled data
B. Unsupervised Learning uses a supervisor
C. RL maximizes a reward signal, Unsupervised Learning finds hidden structure
D. RL is for clustering

49 Which of the following creates a 'Continuous Task'?

A. Go
B. Chess
C. An automated stock trading agent operating indefinitely
D. A robot balancing only for 10 seconds

50 Policy Iteration consists of two alternating steps:

A. Monte Carlo and TD
B. Policy Evaluation and Policy Improvement
C. Exploration and Exploitation
D. Prediction and Control
Previous MCQ
Unit 3
Next MCQ
Unit 5