1What is the primary characteristic of a Sequence Model compared to a standard Feedforward Neural Network?
A.It assumes all inputs are independent of each other
B.It takes the order of inputs into account and can handle variable-length inputs
C.It processes inputs of fixed length only
D.It uses only Convolutional layers
Correct Answer: It takes the order of inputs into account and can handle variable-length inputs
Explanation:
Sequence models, like RNNs, are designed to handle sequential data where the current output depends on previous inputs, allowing for variable-length input sequences.
Incorrect! Try again.
2Which of the following data types is best suited for a Sequence Model?
A.Tabular housing price data
B.Static image classification
C.Sentiment analysis of movie reviews
D.Iris flower categorization
Correct Answer: Sentiment analysis of movie reviews
Explanation:
Sentiment analysis involves text, which is sequential data where the order of words matters, making it ideal for sequence models.
Incorrect! Try again.
3In a Recurrent Neural Network (RNN), what is the function of the 'hidden state'?
A.To reset the network weights after every epoch
B.To store the final output class
C.To visualize the attention weights
D.To act as a memory that captures information about previous time steps
Correct Answer: To act as a memory that captures information about previous time steps
Explanation:
The hidden state in an RNN passes information from one time step to the next, effectively acting as the network's memory of the sequence seen so far.
Incorrect! Try again.
4What is the phenomenon called when the gradients become extremely small during the backpropagation through time in an RNN, preventing weights from updating?
A.Exploding Gradient
B.Vanishing Gradient
C.Overfitting
D.Gradient Clipping
Correct Answer: Vanishing Gradient
Explanation:
The Vanishing Gradient problem occurs when gradients shrink exponentially as they propagate back through many time steps, causing earlier layers to stop learning.
Incorrect! Try again.
5Which algorithm is typically used to train Recurrent Neural Networks?
A.Random Forest
B.Backpropagation Through Time (BPTT)
C.Standard Backpropagation
D.K-Means Clustering
Correct Answer: Backpropagation Through Time (BPTT)
Explanation:
BPTT is a specific variation of backpropagation used for RNNs, where the network is unrolled for all time steps, and errors are propagated back through the sequence.
Incorrect! Try again.
6Which activation function is most commonly used for the hidden state in a simple RNN to help regulate values?
A.Linear
B.Softmax
C.Tanh
D.ReLU
Correct Answer: Tanh
Explanation:
The Tanh (Hyperbolic Tangent) function is commonly used in RNN hidden states because it keeps values between -1 and 1, preventing values from blowing up too quickly.
Incorrect! Try again.
7What is the primary architectural solution designed to solve the Vanishing Gradient problem in standard RNNs?
A.Autoencoder
B.Convolutional Neural Network (CNN)
C.Perceptron
D.Long Short-Term Memory (LSTM)
Correct Answer: Long Short-Term Memory (LSTM)
Explanation:
LSTMs introduce a cell state and gating mechanisms specifically designed to maintain long-term dependencies and mitigate the vanishing gradient problem.
Incorrect! Try again.
8In an LSTM unit, which gate is responsible for deciding what information to discard from the cell state?
A.Update Gate
B.Forget Gate
C.Input Gate
D.Output Gate
Correct Answer: Forget Gate
Explanation:
The Forget Gate uses a sigmoid layer to look at the previous hidden state and current input to decide which information to remove (output 0) or keep (output 1) in the cell state.
Incorrect! Try again.
9What represents the 'long-term memory' component in an LSTM architecture?
A.Cell State (C_t)
B.Input Gate
C.Hidden State (h_t)
D.Output Gate
Correct Answer: Cell State (C_t)
Explanation:
The Cell State runs down the entire chain of the LSTM with only minor linear interactions, acting as the conveyor belt for long-term information.
Incorrect! Try again.
10In an LSTM, what is the range of values output by the sigmoid activation function used in gates?
A.0 to 100
B.-infinity to +infinity
C.0 to 1
D.-1 to 1
Correct Answer: 0 to 1
Explanation:
Sigmoid functions output values between 0 and 1, which effectively act as a switch or percentage to let information pass through or block it.
Incorrect! Try again.
11Which task involves assigning a grammatical category (like Noun, Verb, Adjective) to every word in a sentence?
A.Sentiment Analysis
B.Named Entity Recognition
C.Machine Translation
D.Part-of-Speech (POS) Tagging
Correct Answer: Part-of-Speech (POS) Tagging
Explanation:
POS Tagging is the process of marking up a word in a text as corresponding to a particular part of speech based on its definition and context.
Incorrect! Try again.
12Named Entity Recognition (NER) is primarily concerned with identifying:
A.Real-world objects like people, organizations, and locations
B.Sentiment of the text
C.Grammatical errors
D.The translation of the text
Correct Answer: Real-world objects like people, organizations, and locations
Explanation:
NER is a subtask of information extraction that seeks to locate and classify named entities mentioned in unstructured text into pre-defined categories.
Incorrect! Try again.
13What type of Sequence problem is POS Tagging?
A.One-to-Many
B.Many-to-Many (Synced)
C.Many-to-One
D.One-to-One
Correct Answer: Many-to-Many (Synced)
Explanation:
In POS tagging, a sequence of words (input) is mapped to a sequence of tags (output) of the same length, making it a Many-to-Many synced problem.
Incorrect! Try again.
14In the context of NER, what does the 'BIO' or 'IOB' tagging scheme stand for?
A.Basic-Input-Operation
B.Binary-Input-Output
C.Beginning-Inside-Outside
D.Backward-Inward-Onward
Correct Answer: Beginning-Inside-Outside
Explanation:
BIO stands for Beginning (first token of an entity), Inside (subsequent tokens of an entity), and Outside (not part of an entity), a common format for tagging spans of text.
Incorrect! Try again.
15What is the core architecture used in Neural Machine Translation (NMT) before the introduction of Attention?
A.Random Forest
B.Encoder-Only
C.Decoder-Only
D.Encoder-Decoder (Seq2Seq)
Correct Answer: Encoder-Decoder (Seq2Seq)
Explanation:
Traditional NMT uses an Encoder-Decoder architecture where the encoder processes the source language and the decoder generates the target language.
Incorrect! Try again.
16In a traditional Seq2Seq model, what is the role of the Encoder?
A.To generate the output sequence
B.To visualize the data
C.To compress the input sequence into a fixed-length context vector
D.To calculate the loss function
Correct Answer: To compress the input sequence into a fixed-length context vector
Explanation:
The encoder reads the input sequence and summarizes the information into a final hidden state, often called the context vector, which is passed to the decoder.
Incorrect! Try again.
17What is the 'Context Vector' in a traditional RNN-based Encoder-Decoder model?
A.The average of all input vectors
B.The weights of the output layer
C.The last hidden state of the encoder
D.The first hidden state of the encoder
Correct Answer: The last hidden state of the encoder
Explanation:
In a standard Seq2Seq model, the final hidden state of the encoder RNN represents the context vector containing the summary of the entire input sequence.
Incorrect! Try again.
18Which of the following is a major bottleneck of the traditional Seq2Seq model?
A.It requires too much RAM
B.Performance degrades significantly for long sentences due to the fixed-length context vector
C.It can only translate into English
D.It cannot handle numeric data
Correct Answer: Performance degrades significantly for long sentences due to the fixed-length context vector
Explanation:
Compressing a long sentence into a single fixed-length vector causes information loss, making it difficult for the decoder to generate accurate translations for long sequences.
Incorrect! Try again.
19What is 'Teacher Forcing' in the context of training sequence models?
A.Manually setting the weights of the network
B.Using the model's predicted output as input for the next step during training
C.Forcing the model to stop training early
D.Using the actual ground truth output from the previous time step as input for the current step during training
Correct Answer: Using the actual ground truth output from the previous time step as input for the current step during training
Explanation:
Teacher forcing speeds up convergence by feeding the correct previous token to the decoder during training, rather than the token the model predicted (which might be wrong).
Incorrect! Try again.
20Which search strategy explores multiple possible output sequences simultaneously to find the most likely translation?
A.Beam Search
B.Linear Search
C.Greedy Search
D.Binary Search
Correct Answer: Beam Search
Explanation:
Beam search maintains a 'beam' of the top 'k' most probable partial sequences at each step, offering a better approximation of the global best sequence than Greedy Search.
Incorrect! Try again.
21The Attention Mechanism was primarily introduced to solve which problem?
A.The inability of RNNs to process images
B.Overfitting in CNNs
C.The information bottleneck of the fixed-length context vector in NMT
D.Slow training of Linear Regression
Correct Answer: The information bottleneck of the fixed-length context vector in NMT
Explanation:
Attention allows the decoder to look at different parts of the input sequence dynamically, removing the reliance on a single static context vector.
Incorrect! Try again.
22How does the Attention Mechanism calculate the context vector for each time step in the decoder?
A.By averaging all input words
B.By computing a weighted sum of all encoder hidden states
C.By taking the last state of the encoder only
D.By randomly selecting an encoder state
Correct Answer: By computing a weighted sum of all encoder hidden states
Explanation:
Attention computes a weighted sum of encoder states, where the weights represent the importance (alignment) of that specific encoder state to the current decoder step.
Incorrect! Try again.
23In Attention, what do the 'alignment scores' (or attention weights) represent?
A.The number of hidden layers
B.The magnitude of the gradient
C.The error rate of the model
D.How relevant a specific input word is to the word currently being generated
Correct Answer: How relevant a specific input word is to the word currently being generated
Explanation:
Alignment scores indicate how much focus the decoder should place on a specific encoder hidden state (input word) when generating the current output word.
Incorrect! Try again.
24What mathematical function is typically applied to alignment scores to convert them into probabilities that sum to 1?
A.Softmax
B.Sigmoid
C.ReLU
D.Tanh
Correct Answer: Softmax
Explanation:
The Softmax function normalizes the raw attention scores into a probability distribution, ensuring the attention weights sum to 1.
Incorrect! Try again.
25In the context of RNNs, what is 'Backpropagation Through Time' (BPTT)?
A.Unfolding the RNN across time steps and applying backpropagation
B.Using future data to predict past data
C.Training the network in reverse order
D.A method to predict future stock prices
Correct Answer: Unfolding the RNN across time steps and applying backpropagation
Explanation:
BPTT involves unrolling the recurrent network for the duration of the sequence and calculating gradients across this unrolled graph.
Incorrect! Try again.
26Which of the following is NOT a gate in a standard LSTM?
A.Attention Gate
B.Input Gate
C.Forget Gate
D.Output Gate
Correct Answer: Attention Gate
Explanation:
Standard LSTMs have Forget, Input, and Output gates. Attention is a separate mechanism usually applied externally to the RNN layers.
Incorrect! Try again.
27What is the shape of the input data for a basic RNN layer in Keras/TensorFlow?
A.(Features, Labels)
B.(Batch Size, Timesteps, Features)
C.(Timesteps, Batch Size)
D.(Batch Size, Features)
Correct Answer: (Batch Size, Timesteps, Features)
Explanation:
RNNs require 3D input tensors: the number of samples (batch size), the length of the sequence (timesteps), and the dimensionality of the data at each step (features).
Incorrect! Try again.
28Why are Bidirectional RNNs (BiRNNs) useful?
A.They use fewer parameters
B.They allow the network to have context from both the past and the future
C.They train faster than standard RNNs
D.They eliminate the need for backpropagation
Correct Answer: They allow the network to have context from both the past and the future
Explanation:
BiRNNs process the sequence in both forward and backward directions, providing the current state with context from both preceding and succeeding words.
Incorrect! Try again.
29In a Many-to-One sequence model (e.g., Sentiment Analysis), where is the output typically taken?
A.Randomly sampled
B.At every time step
C.At the last time step
D.At the first time step
Correct Answer: At the last time step
Explanation:
For Many-to-One tasks like classification, the final hidden state after processing the whole sequence is used to generate the single output label.
Incorrect! Try again.
30What does the 'candidate cell state' in an LSTM do?
A.It outputs the final prediction
B.It decides what to forget
C.It clears the memory
D.It proposes new values that could be added to the state
Correct Answer: It proposes new values that could be added to the state
Explanation:
The candidate cell state (usually created with a Tanh layer) creates a vector of new candidate values that might be added to the cell state, regulated by the input gate.
Incorrect! Try again.
31Which issue leads to the 'Exploding Gradient' problem?
If the derivatives are larger than 1, repeated multiplication during backpropagation causes the gradients to grow exponentially, leading to instability.
Incorrect! Try again.
32A solution to the Exploding Gradient problem is:
A.Gradient Clipping
B.Removing the hidden layer
C.Using ReLU
D.Increasing the learning rate
Correct Answer: Gradient Clipping
Explanation:
Gradient Clipping involves capping the gradients at a specific threshold value to prevent them from becoming too large during training.
Incorrect! Try again.
33In an Attention model, the vector c_t is often referred to as:
A.The Context Vector
B.The Bias Vector
C.The Forget Vector
D.The Noise Vector
Correct Answer: The Context Vector
Explanation:
The context vector c_t is the dynamic summary of the input sequence tailored for the specific decoding time step t via attention weights.
Incorrect! Try again.
34Sequence-to-Sequence models are most commonly associated with:
A.Cluster Analysis
B.Text Summarization
C.Linear Regression
D.Image Segmentation
Correct Answer: Text Summarization
Explanation:
Text summarization transforms an input sequence (text) into an output sequence (summary), fitting the Seq2Seq paradigm perfectly.
Incorrect! Try again.
35What is 'Global Attention'?
A.Attention that considers all hidden states of the encoder
B.Attention applied without weights
C.Attention applied to a single word
D.Attention that considers only a window of hidden states
Correct Answer: Attention that considers all hidden states of the encoder
Explanation:
Global attention computes the alignment scores against all encoder hidden states, as opposed to Local attention which focuses on a small window.
Incorrect! Try again.
36Which of the following describes 'Greedy Decoding'?
A.Choosing the word with the highest probability at each step immediately
B.Waiting until the end to choose words
C.Choosing a random word based on distribution
D.Considering all possible future sequences
Correct Answer: Choosing the word with the highest probability at each step immediately
Explanation:
Greedy decoding selects the token with the highest probability at the current step without considering how this choice impacts future probabilities.
Incorrect! Try again.
37In POS tagging, if a word is ambiguous (e.g., 'book' can be a noun or verb), how does an RNN resolve it?
A.It always picks the most common usage
B.It uses the context provided by surrounding words stored in the hidden state
C.It cannot resolve ambiguity
D.It flips a coin
Correct Answer: It uses the context provided by surrounding words stored in the hidden state
Explanation:
The RNN's hidden state captures context (e.g., 'read a book' vs 'book a flight'), allowing it to disambiguate based on the surrounding sequence.
Incorrect! Try again.
38What is the typical loss function for a multi-class classification problem like POS Tagging or NMT?
A.Hinge Loss
B.Mean Squared Error (MSE)
C.Absolute Error
D.Categorical Cross-Entropy
Correct Answer: Categorical Cross-Entropy
Explanation:
Cross-entropy loss is the standard for multi-class classification tasks where the model outputs a probability distribution over the vocabulary/tags.
Incorrect! Try again.
39What does GRU stand for?
A.Gradient Rectified Unit
B.Gated Recurrent Unit
C.General Regression Unit
D.Global Recurrent Update
Correct Answer: Gated Recurrent Unit
Explanation:
GRU is a simplified variation of LSTM that merges the cell state and hidden state and uses fewer gates.
Incorrect! Try again.
40The 'Input Gate' in an LSTM is usually controlled by which activation function?
A.Sigmoid
B.Linear
C.Tanh
D.ReLU
Correct Answer: Sigmoid
Explanation:
The gate controller uses a Sigmoid function to output values between 0 (block) and 1 (pass), determining how much new information enters the state.
Incorrect! Try again.
41What visual tool is often used to interpret what an Attention model has learned?
A.Attention Heatmap
B.Scatter Plot
C.Histogram
D.Pie Chart
Correct Answer: Attention Heatmap
Explanation:
Heatmaps visualize the alignment matrix, showing which input words correspond strongly to which output words (e.g., in translation).
Incorrect! Try again.
42Which limitation of RNNs prevents parallelization during training?
A.Sequential dependency of the hidden state
B.Complex loss functions
C.Use of sigmoid functions
D.Large memory footprint
Correct Answer: Sequential dependency of the hidden state
Explanation:
Because the state at time t depends on t-1, computations must happen sequentially, preventing parallel processing on GPUs.
Incorrect! Try again.
43In a sequence model, 'padding' is used to:
A.Remove stopwords
B.Add noise to the data
C.Make all sequences in a batch the same length
D.Increase the learning rate
Correct Answer: Make all sequences in a batch the same length
Explanation:
Since neural networks require fixed-size tensor inputs for batching, shorter sequences are padded (usually with zeros) to match the length of the longest sequence.
Incorrect! Try again.
44Which of these is a 'many-to-one' application of sequence models?
A.Music Generation
B.Sentiment Classification
C.Machine Translation
D.Video Captioning
Correct Answer: Sentiment Classification
Explanation:
Input is a sequence of words (many), output is a single sentiment label (one).
Incorrect! Try again.
45In an NER task, identifying 'Apple' as an Organization rather than a Fruit relies on:
A.The contextual information in the sequence
B.The capitalization
C.The spelling of the word
D.The length of the word
Correct Answer: The contextual information in the sequence
Explanation:
While capitalization helps, the context (e.g., 'Apple released a new phone' vs 'I ate an apple') is crucial for correct entity classification.
Incorrect! Try again.
46Why is the traditional Encoder-Decoder model often described as having 'amnesia'?
A.It struggles to retain information from the beginning of a long sequence at the decoding stage
B.It uses a forget gate
C.It forgets the weights after training
D.It cannot learn new words
Correct Answer: It struggles to retain information from the beginning of a long sequence at the decoding stage
Explanation:
Standard Seq2Seq models rely on the final hidden state. For long sentences, information from the start of the sentence is often diluted or 'forgotten' by the time the vector is formed.
Incorrect! Try again.
47In the attention equation score(h_t, h_s), what are h_t and h_s?
A.Weight and Bias
B.Learning rate and Loss
C.Input and Output Gates
D.Decoder hidden state and Encoder hidden state
Correct Answer: Decoder hidden state and Encoder hidden state
Explanation:
The score function calculates the similarity/compatibility between the current decoder state (h_t) and a specific encoder state (h_s).
Incorrect! Try again.
48Which mechanism allows a model to focus on 'local' parts of the input sequence based on the current decoding step?
A.Dropout
B.Attention Mechanism
C.Batch Normalization
D.Max Pooling
Correct Answer: Attention Mechanism
Explanation:
Attention allows the model to dynamically weigh and focus on specific parts (local areas) of the input sequence relevant to the current prediction.
Incorrect! Try again.
49In sequence labeling, what does the output layer usually consist of?
A.A linear regression layer
B.A Softmax layer over the tag set for each time step
C.A single neuron
D.A clustering algorithm
Correct Answer: A Softmax layer over the tag set for each time step
Explanation:
For tasks like POS or NER, the model outputs a probability distribution over all possible tags for every word in the sequence.
Incorrect! Try again.
50Which of the following best describes the 'Seq2Seq' mapping?
Seq2Seq models map an input sequence of length N to an output sequence of length M, where N and M can differ and vary between samples.
Incorrect! Try again.
Did this save you a night before the exam?
LPU Notes is free, and it stays free. Ads cover part of the server bill.
The rest comes out of a student's own pocket: the domain, the storage,
and keeping the site up through the weeks everyone needs it at once.
The payment button didn't load. An ad blocker or a filtered network is the usual reason.
to try again.
Nothing here is ever locked, and nothing unlocks. Chip in only if it was worth it.
What it pays for →