Unit 3: Deep Learning Sequence Models for NLP - Practice Quiz

1 Why is text commonly treated as sequential data?

2 What is a token in a text sequence?

3 What allows a recurrent neural network to use information from earlier time steps?

4 Which type of input is an RNN especially designed to process?

5 What is the main purpose of an LSTM network?

6 Which LSTM component decides what information should be removed from the cell state?

7 Which two gates are commonly used in a GRU?

8 How does a GRU generally compare with an LSTM?

9 How does a bidirectional RNN process a sequence?

10 What is a key benefit of a bidirectional RNN for text processing?

11 Which task is a common application of sequence modeling?

12 In next-word prediction, what does a sequence model predict?

13 What does sentiment classification usually identify in a text?

14 Which label would a basic sentiment classifier most likely assign to the review "I loved this movie"?

15 What is the goal of text classification?

16 Which task is an example of text classification?

17 What is padding used for when training sequence models in batches?

18 What is supplied as the next decoder input during teacher forcing?

19 What does truncated backpropagation through time limit during training?

20 Which metric measures the proportion of predicted positive cases that are actually positive?

21 Why is word order important when representing a sentence for a sequence model?

22 At time step , which information is typically used by a vanilla RNN to compute its hidden state?

23 An RNN performs well on short sequences but loses information from the beginning of very long sequences. Which problem most directly explains this behavior?

24 What is the main purpose of the forget gate in an LSTM?

25 For a sentiment classifier, why might an LSTM outperform a vanilla RNN when a sentiment cue appears far earlier than the final classification decision?

26 Which statement best describes a key difference between a GRU and an LSTM?

27 A GRU's update gate tends to remain highly open for a time step. What behavior does this usually encourage?

28 Why is a bidirectional RNN useful for named-entity recognition in a completed sentence?

29 Which limitation should be considered when using a bidirectional RNN for real-time speech transcription?

30 Which task is most naturally modeled as sequence-to-sequence learning?

31 In sequence labeling, how does the model output usually differ from document classification?

32 A review says, "The camera is small, but the battery life is excellent." What is the main challenge in assigning an overall sentiment label?

33 Why can a recurrent model be preferable to a bag-of-words classifier for detecting whether a message is a question?

34 A dataset has 95% negative examples and 5% positive examples. Which evaluation choice gives the clearest view of positive-class performance?

35 Why are padding masks used when training on a batch of variable-length sequences?

36 A sequence model overfits its training set while achieving much worse validation performance. Which technique is a reasonable response?

37 During teacher forcing for next-token prediction, what is fed to the decoder at time ?

38 What is a common drawback of using teacher forcing during training but relying only on model predictions during inference?

39 What is the primary purpose of truncated backpropagation through time?

40 If the truncation window is too short, which behavior is most likely?

41 A batch contains right-padded sequences with true lengths . A unidirectional RNN processes all seven time steps without packing or masking, and its hidden state is updated normally for padding tokens. Which representation should be used to classify the second sequence while preserving its intended semantics?

42 For a linearized RNN, , suppose is normal and has spectral radius . Ignoring loss-specific factors, what is the asymptotic behavior of the gradient contribution propagated across recurrent steps?

43 An LSTM uses . Along one cell dimension, assume indirect dependencies through the gates are ignored and for 200 steps. Approximately what fraction of a gradient on reaches through the direct cell-state path?

44 A GRU has input size and hidden size . Each of its reset, update, and candidate computations has an input matrix, a recurrent matrix, and one bias vector. Excluding embeddings and output layers, how many trainable scalar parameters does it have?

45 A bidirectional RNN is trained for token-level prediction using the entire sentence. The system must later emit each prediction immediately as a token arrives in a live stream. What is the central deployment incompatibility?

46 A named-entity recognizer independently predicts BIO tags and frequently outputs illegal transitions such as O I-PER. Which modification most directly models dependencies between adjacent output labels while retaining neural token representations?

47 A document-level sentiment model uses only the final hidden state of a unidirectional RNN. Reviews are truncated to their last 200 tokens, but sentiment-changing premises often occur near the beginning. Which change most directly preserves those premises without increasing the 200-token limit?

48 In a multilabel text classifier, each label is trained with binary cross-entropy. Label A is common and Label B is rare, and their score distributions differ substantially. Which validation procedure is most appropriate when the target metric is macro-?

49 During sequence-model training, scheduled sampling increasingly replaces gold previous tokens with model-generated tokens. Which theoretical issue can remain even if this improves robustness at inference time?

50 A decoder trained with full teacher forcing has low validation loss when conditioned on gold prefixes but produces poor long free-running sequences. Which explanation best identifies the mismatch?

51 A stateful RNN processes a long stream in consecutive chunks of 50 steps. The final hidden value of each chunk initializes the next chunk, but it is detached from the computation graph at every boundary. Which statement is correct?

52 A language model has average negative log-likelihood nats per token on a corpus. What is its token-level perplexity?

53 Two translation systems are evaluated with BLEU. System X produces fluent paraphrases with few exact reference -grams, while System Y copies many reference phrases but occasionally changes critical meanings. Which limitation of BLEU is most directly illustrated?

54 For an LSTM, . Suppose a cell stores useful information with , but is nearly zero. What follows at time ?

55 Consider the GRU convention . If is approximately zero in every dimension, what behavior results?

56 A model mean-pools token representations for text classification. A minibatch is right-padded, and padding vectors are zeros before entering a recurrent encoder. Why must the pooling operation still use a length mask?

57 A bidirectional LSTM has hidden size in each direction and concatenates its forward and backward states at every token. A linear token classifier predicts classes. Ignoring bias, what is the classifier weight matrix shape under the convention ?

58 A binary text classifier is evaluated on 10,000 examples, of which only 100 are positive. It predicts every example as negative. Which metric can appear deceptively strong despite total failure on the positive class?

59 An RNN occasionally produces extremely large gradient norms because products of recurrent Jacobians amplify particular directions. Which intervention directly bounds the magnitude of each parameter update while largely preserving the gradient direction?

60 A BIO-tagged NER prediction identifies the correct entity type but extends the entity by one token beyond the gold boundary. Under strict entity-level micro-, how is this single predicted span counted?