Unit 4: Sequence-to-Sequence Models and Attention Mechanisms - Practice Quiz

1 What is the main role of the encoder in an encoder-decoder architecture?

2 What does the decoder typically produce at each time step?

3 In machine translation, what does a sequence-to-sequence model map?

4 What is the expected output of a sequence-to-sequence text summarization model?

5 What does an attention mechanism allow a decoder to do?

6 Why is attention useful for processing long input sequences?

7 How does soft attention usually combine encoder hidden states?

8 Which function is commonly used to normalize soft attention scores into weights?

9 What does an alignment score measure in an attention-based model?

10 In machine translation, alignment often connects which two elements?

11 Bahdanau attention is also commonly known as which type of attention?

12 Which information does Bahdanau attention use to compute an alignment score?

13 Luong attention is commonly associated with which scoring approach?

14 In global Luong attention, which encoder hidden states may be considered?

15 What is the context vector in an attention-based encoder-decoder model?

16 How does the attention context vector usually change during decoding?

17 What is a reference text in sequence generation evaluation?

18 BLEU primarily evaluates generated text using which type of comparison?

19 ROUGE is most commonly used to evaluate which NLP task?

20 What is a major limitation of a classical sequence-to-sequence model without attention?

21 In an encoder-decoder architecture for machine translation, what is the primary role of the encoder?

22 Why is teacher forcing commonly used when training an encoder-decoder model?

23 A sequence-to-sequence model is used for abstractive summarization. Which behavior best distinguishes abstractive summarization from extractive summarization?

24 During inference, why is beam search often preferred over greedy decoding in a translation model?

25 What problem does attention primarily address in a basic encoder-decoder model?

26 In soft attention, the context vector is typically computed as:

27 Why can soft attention be trained using standard backpropagation?

28 In an attention-based translation model, alignment scores are used to estimate:

29 After applying softmax to alignment scores for one decoding step, what does a larger attention weight indicate?

30 Which description best matches Bahdanau attention?

31 In the original Bahdanau formulation, which decoder state is commonly used to calculate attention scores for target step ?

32 Which scoring function is most directly associated with Luong dot-product attention?

33 What is a key distinction between general Luong attention and dot-product Luong attention?

34 How is the context vector commonly used by an attention-based decoder?

35 Suppose an input has 12 source tokens and the decoder generates 8 target tokens. What is the typical size of the attention matrix for one example?

36 Why should automatic metrics for generated summaries or translations be interpreted alongside human evaluation?

37 A translation receives a high modified n-gram precision but is much shorter than the reference. Which BLEU component helps penalize this output?

38 Why does BLEU use clipped n-gram counts instead of unrestricted candidate n-gram counts?

39 Which ROUGE variant primarily measures recall of overlapping unigrams between a generated summary and a reference summary?

40 ROUGE-L is based primarily on which type of overlap?

41 A bidirectional encoder produces states , while a decoder GRU expects an initial state in . Which initialization preserves information from both encoder directions while satisfying the decoder's dimensional constraint?

42 A summarization decoder is trained with teacher forcing and achieves low token-level cross-entropy, but at inference time an early incorrect token causes increasingly incoherent output. Which train-test discrepancy most directly explains this behavior?

43 Beam search compares hypothesis of length with log-probability and hypothesis of length with log-probability . If the score is average log-probability , which hypothesis is selected?

44 For cross-attention with target length , source length , and hidden dimension , all projected query-key dot products are computed explicitly. Ignoring projection costs, what are the time and attention-matrix space complexities?

45 Let attention weights be and scalar encoder states be . What context value is supplied to a standard soft-attention decoder?

46 A batch contains padded source sequences. Before the alignment softmax, the scores for one example are , where the last two positions are padding. Which operation gives a valid alignment over only real tokens?

47 Which expression correctly represents the additive alignment score used by Bahdanau attention for decoder state and encoder state ?

48 In Luong's multiplicative attention, which scoring function can compare encoder and decoder states of different original dimensions without requiring them to be equal?

49 Consider the common recurrent formulations of Bahdanau and Luong attention. Which timing distinction is accurate?

50 What is the primary modeling effect of input feeding in Luong attention, where the previous attentional vector is concatenated with the next decoder input?

51 An encoder output matrix is , attention weights at step are , and the decoder state is . If and the output layer consumes , what must be the input width of that output layer?

52 A decoder applies soft attention, but the context vector is accidentally detached from the computation graph before the token loss is computed. Assuming no other loss uses attention, which parameters lose the token-loss gradient path?

53 A candidate is the cat the cat and the single reference is the cat is here. Using clipped unigram precision, how many candidate unigram matches are counted and what is the resulting precision?

54 A corpus-level BLEU calculation has candidate length , effective reference length , and geometric mean modified precision . Using when , what is BLEU?

55 A candidate sentence has positive modified unigram, bigram, and trigram precisions but no matching 4-grams. Under unsmoothed BLEU-4 with uniform weights, what score does the sentence receive?

56 A generated summary contains occurrences of a bigram, while the reference contains occurrences of that bigram. All other bigrams have zero overlap, and the reference has bigram tokens in total. What is the clipped ROUGE-2 recall?

57 For candidate tokens A B C D and reference tokens B A C D, the longest common subsequence has length . What are ROUGE-L precision, recall, and for this pair?

58 Two summarization systems obtain nearly identical ROUGE scores, but one frequently introduces unsupported names and dates. Which evaluation addition most directly distinguishes the systems on this failure mode?

59 Why does adding attention usually improve a recurrent encoder-decoder on long inputs without completely eliminating its long-sequence limitations?

60 In a translation model, repeated target phrases arise because the decoder repeatedly attends to the same source positions. Which modification most directly addresses this alignment pathology?