Back to Subject

Unit 2: Word Embeddings and Vector Representations - Practice Quiz

CSE472 — Deep Learning For Natural Language Processing 60 Questions
0 Correct 0 Wrong 60 Left
0/60

1 How does a vector space model represent a word?

Vector space models Easy
A. As an audio signal
B. As a grammar rule
C. As a numerical vector
D. As a database table containing every sentence in the corpus

2 In a vector space model, what does each dimension commonly represent?

Vector space models Easy
A. A complete paragraph
B. A neural network layer used only to store labels
C. A numerical feature
D. A punctuation rule

3 What is a dense word embedding?

Dense word embeddings Easy
A. A table that maps sentences directly to grammar rules
B. A short vector with mostly nonzero values
C. A long vector with mostly zero values
D. A list containing only word frequencies

4 What is one advantage of dense embeddings over one-hot vectors?

Dense word embeddings Easy
A. They assign words identical vectors
B. They preserve the exact position of every word in every document
C. They always require more dimensions
D. They can encode semantic relationships

5 What is Word2Vec primarily used to learn?

Word2Vec Easy
A. Sentence punctuation
B. Document file formats
C. Word embeddings
D. Handwritten rules for parsing every possible sentence

6 Which two architectures are associated with Word2Vec?

Word2Vec Easy
A. PCA and t-SNE
B. GloVe and a large manually constructed dictionary of synonyms
C. CNN and pooling
D. CBOW and Skip-Gram

7 What does CBOW predict during training?

CBOW Easy
A. Context words from a target word
B. The full grammatical structure of an entire document
C. A target word from context words
D. A sentence from one character

8 What does the abbreviation CBOW stand for?

CBOW Easy
A. Combined Batch of Words
B. Continuous Binary Output Weights for language modeling
C. Continuous Bag of Words
D. Context-Based Ordered Words

9 What does a Skip-Gram model predict?

Skip-Gram models Easy
A. Word length from character count
B. A target word from context words
C. A complete translation using only the punctuation marks
D. Context words from a target word

10 If the target word is "coffee", what would Skip-Gram try to learn from a nearby word such as "cup"?

Skip-Gram models Easy
A. That every sentence containing "coffee" must end with "cup"
B. That both words have equal lengths
C. That "cup" must replace "coffee"
D. That "cup" can occur near "coffee"

11 What does GloVe stand for?

GloVe embeddings Easy
A. General Language Output
B. Global Vectors
C. Grouped Vocabulary
D. Globally Validated Linguistic Output Vector Encoding

12 What type of information does GloVe mainly use to learn embeddings?

GloVe embeddings Easy
A. Only alphabetical word order
B. A manually written semantic definition for every word
C. Global word co-occurrence statistics
D. Only sentence punctuation counts

13 In a well-trained embedding space, how are semantically similar words usually positioned?

Capturing semantic similarity Easy
A. Far apart
B. In separate spaces created from the spelling of each word
C. Close together
D. At the origin

14 Which measure is commonly used to compare the similarity of two word vectors?

Capturing semantic similarity Easy
A. The number of documents stored in the training directory
B. Character count
C. Cosine similarity
D. Sentence length

15 Which pair would most likely have high semantic similarity in an embedding space?

Capturing semantic similarity Easy
A. car and quickly
B. car and a detailed rule describing plural noun formation
C. car and banana
D. car and automobile

16 Which word best completes the analogy "king is to queen as man is to ___"?

Analogy relationships Easy
A. child
B. a person who rules a large country from a royal residence
C. woman
D. palace

17 Which vector expression is commonly associated with the analogy "king - man + woman"?

Analogy relationships Easy
A. A vector near "queen"
B. A vector near "castle"
C. A vector representing every royal title in the training corpus
D. A vector near "prince"

18 Why are PCA and t-SNE used with word embeddings?

Visualizing embedding spaces using PCA or t-SNE Easy
A. To translate words into another language
B. To increase the number of dimensions
C. To replace all learned vectors with fixed one-hot encodings
D. To reduce dimensions for visualization

19 What type of method is PCA?

Visualizing embedding spaces using PCA or t-SNE Easy
A. A global co-occurrence model
B. A linear dimensionality reduction method
C. A word prediction architecture
D. A nonlinear method designed exclusively for generating text

20 What is t-SNE especially useful for showing in a two-dimensional embedding plot?

Visualizing embedding spaces using PCA or t-SNE Easy
A. Local clusters of similar words
B. A guaranteed preservation of every global distance in the dataset
C. Exact word frequency counts
D. The original vector dimensions

21 A corpus-based retrieval system represents documents using TF-IDF vectors. Why will the term "photosynthesis" receive a higher weight in a document than the term "the"?

Vector space models Medium
A. It has more characters than most other terms
B. It appears near the beginning of the document
C. It is rare in the document but frequent across the corpus
D. It is frequent in the document but rare across the corpus

22 A vocabulary contains 50,000 words. One model uses one-hot vectors, while another uses 300-dimensional dense embeddings. What is the main representational advantage of the dense embeddings?

Dense word embeddings Medium
A. They assign a unique dimension to every word
B. They encode similarity using far fewer dimensions
C. They guarantee correct meanings for unseen words
D. They eliminate the need to train model parameters

23 After training Word2Vec, the embeddings of "doctor" and "nurse" are close even though the words rarely appear next to each other. Which observation best explains this result?

Word2Vec Medium
A. They tend to occur in similar linguistic contexts
B. They occupy nearby positions in the vocabulary
C. They have nearly identical corpus frequencies
D. They contain a similar number of characters

24 For the sentence "the cat sleeps on the mat" with target word "sleeps" and context window size 2, which input-output pair is used by CBOW?

CBOW Medium
A. Input: "cat", "on"; output: "sleeps", "mat"
B. Input: "the", "cat"; output: "on", "the"
C. Input: "the", "cat", "on", "the"; output: "sleeps"
D. Input: "sleeps"; output: "the", "cat", "on", "the"

25 For the sentence "deep models learn useful features" with center word "learn" and context window size 1, which training pairs are produced by Skip-Gram?

Skip-Gram models Medium
A. ("learn", "models") and ("learn", "useful")
B. ("models", "learn") and ("useful", "learn")
C. ("learn", "deep") and ("learn", "features")
D. ("models", "useful") and ("deep", "features")

26 A Word2Vec model has a vocabulary of one million words. Why is negative sampling commonly used during training?

Word2Vec Medium
A. It removes every infrequent word from the training corpus
B. It converts dense embedding vectors into sparse vectors
C. It fixes the embedding dimensions before optimization
D. It avoids computing a full softmax over the vocabulary

27 Which corpus statistic is used most directly when training GloVe embeddings?

GloVe embeddings Medium
A. Character-level edit distances between words
B. Global word-context co-occurrence counts
C. Sentence labels assigned by annotators
D. Document lengths measured in tokens

28 Two normalized word vectors are and . What is their cosine similarity?

Capturing semantic similarity Medium
A.
B.
C.
D.

29 Which vector expression is typically used to solve the analogy "Paris is to France as Tokyo is to ___"?

Analogy relationships Medium
A.
B.
C.
D.

30 A researcher wants a two-dimensional embedding plot that preserves as much global variance as possible and gives reproducible results without random initialization. Which method is more appropriate?

Visualizing embedding spaces using PCA or t-SNE Medium
A. Negative sampling
B. PCA
C. Skip-Gram
D. t-SNE

31 In a t-SNE plot, two clusters appear far apart. Which conclusion is most defensible?

Visualizing embedding spaces using PCA or t-SNE Medium
A. The clusters are equally distant in the original embedding space
B. The clusters must represent completely unrelated semantic categories
C. Local neighborhoods may be meaningful, but cluster distance may not be
D. Every pairwise distance has been preserved by the projection

32 A Skip-Gram model is retrained with a larger context window. What change is most likely in the learned similarities?

Skip-Gram models Medium
A. Words with identical spellings become the only close neighbors
B. Words with related topics become more likely to be similar
C. Words outside each sentence receive exactly equal vectors
D. Words are represented using fewer embedding dimensions

33 A single pretrained embedding for "bank" is used in both "river bank" and "bank loan". What limitation does this demonstrate?

Dense word embeddings Medium
A. A cosine score cannot compare two word vectors
B. A one-hot vector is required for ambiguous words
C. A dense vector cannot represent frequent vocabulary words
D. A static vector merges multiple senses of a word

34 A nearest-neighbor system uses raw dot products between word embeddings whose norms vary greatly. Long-norm vectors dominate the results. Which change best focuses the comparison on vector direction?

Capturing semantic similarity Medium
A. Replace each vector with its largest coordinate
B. Increase every embedding dimension by the same constant
C. Sort the words by frequency before comparing them
D. Normalize the vectors and use cosine similarity

35 Why does the GloVe objective use a weighting function for word-context co-occurrence counts?

GloVe embeddings Medium
A. To force all co-occurrence counts to become binary
B. To make every vocabulary word equally frequent
C. To limit the influence of extremely common co-occurrences
D. To remove context words that occur only once

36 Two CBOW examples contain the same context words in different orders. If the model averages context embeddings, how will their context representations compare?

CBOW Medium
A. They will be orthogonal because the orders are reversed
B. They will be identical because averaging ignores order
C. They will be sparse because the context has multiple words
D. They will differ because averaging preserves word positions

37 A corpus contains many rare technical terms. Compared with CBOW, why might Skip-Gram produce better representations for those terms?

Skip-Gram models Medium
A. It creates several prediction pairs from each center-word occurrence
B. It predicts the center word from one averaged context
C. It assigns rare terms vectors without any training
D. It replaces every rare term with a common synonym

38 Documents and have cosine similarity , while documents and have cosine similarity . What is the best interpretation?

Vector space models Medium
A. contains more unique words than
B. is necessarily longer than both other documents
C. and contain exactly the same terms
D. and have more similar term distributions

39 Suppose is close to . What relationship has the embedding space most likely captured?

Analogy relationships Medium
A. A country to capital-city relationship
B. A singular-noun to plural-noun relationship
C. A base-verb to present-participle relationship
D. A positive-word to negative-word relationship

40 A researcher has 300-dimensional embeddings for 200,000 words and wants to apply t-SNE efficiently. Which preprocessing strategy is commonly appropriate?

Visualizing embedding spaces using PCA or t-SNE Medium
A. Duplicate each embedding before running the projection
B. Replace each vector with its original corpus frequency
C. Convert every embedding into a one-hot vocabulary vector
D. Use PCA first to reduce to a moderate number of dimensions

41 A corpus contains the terms bank, river, and loan. In a count-based vector space model, bank has high co-occurrence counts with both river and loan, while river and loan rarely co-occur. Which limitation is most directly illustrated?

Vector space models Hard
A. The model cannot compare vectors with cosine similarity
B. The model always assigns identical vectors to synonyms
C. The model cannot represent word frequency
D. The model may conflate unrelated senses of a polysemous word

42 Two word vectors have the same Euclidean norm, but one vector contains a large common-frequency component shared by many words. Which transformation most directly reduces the influence of that shared component when measuring lexical similarity?

Vector space models Hard
A. Use Euclidean distance without normalization
B. Duplicate the common-frequency component
C. Replace each vector with its word frequency
D. Use cosine similarity after centering the vectors

43 A dense embedding has dimension , and its training corpus contains distinct word types. Which statement best explains why the embedding is not merely a compressed one-hot representation?

Dense word embeddings Hard
A. The vectors preserve every original co-occurrence count exactly
B. The vectors are learned so dimensions encode distributed context patterns
C. The embedding guarantees one unique dimension for every word
D. Each word still occupies exactly one independent coordinate

44 A rare word appears only in contexts associated with a frequent synonym. During embedding training, which outcome is most plausible, assuming the rare word receives enough updates to be retained?

Dense word embeddings Hard
A. Its vector is determined only by its spelling
B. Its vector must become orthogonal to the synonym vector
C. Its vector becomes exactly equal to the synonym vector
D. Its vector can approach the synonym vector but remains data- and optimization-dependent

45 In the negative-sampling objective for a positive pair , the model maximizes and adds noise terms. If the noise distribution is changed from uniform to a unigram distribution raised to the power , what is the primary effect?

Word2Vec Hard
A. Frequent words are sampled less often than under uniform sampling
B. The positive pair is replaced by independent positive pairs
C. The noise distribution balances frequent and rare words more effectively
D. Rare words are excluded from the noise distribution

46 Why can two independently trained Word2Vec models have excellent intrinsic similarity scores while their raw vector coordinates differ substantially?

Word2Vec Hard
A. The objective is invariant to certain transformations of the embedding space
B. The models must use different vocabulary encodings
C. Word2Vec stores vectors as unordered frequency lists
D. Similarity scores ignore all relationships among words

47 A CBOW model predicts a center word from the average of its surrounding context vectors. If one context word is highly frequent and semantically uninformative, which modification most directly reduces its disproportionate training influence?

CBOW Hard
A. Increase the context window for every example
B. Remove negative sampling from the objective
C. Apply subsampling to frequent words
D. Replace the average with a one-hot center vector

48 Consider a CBOW model using the sum of context embeddings to predict the target. Compared with a model using the average, what changes when the context window contains twice as many tokens, assuming the embeddings are unchanged?

CBOW Hard
A. The input direction always becomes orthogonal to the original
B. The model receives no additional contextual signal
C. The target vocabulary is automatically doubled
D. The input magnitude generally increases under the sum

49 For a fixed corpus and number of center words, which training pattern usually creates more prediction pairs in Skip-Gram than in CBOW?

Skip-Gram models Hard
A. Only the most frequent context word is retained
B. One context aggregate is predicted from each target
C. Each sentence is represented by one vocabulary vector
D. Each center word predicts multiple context words separately

50 A corpus contains many rare technical terms, and the training budget is limited. Why might Skip-Gram be preferred over CBOW for learning those terms?

Skip-Gram models Hard
A. Skip-Gram represents every word with a separate vocabulary dimension
B. Skip-Gram never uses negative examples
C. Skip-Gram predicts context words from each center word and can provide stronger signals for rare centers
D. Skip-Gram shares one update across all rare terms

51 In Skip-Gram with negative sampling, a positive pair has a dot product that is already very large and positive. What is the likely training behavior for that positive term?

Skip-Gram models Hard
A. Its sigmoid gradient becomes maximal
B. Its sigmoid gradient becomes small
C. Its context vector is removed from the vocabulary
D. Its label changes from positive to negative

52 GloVe uses a weighted least-squares objective over word-context co-occurrence counts. Why is a weighting function useful?

GloVe embeddings Hard
A. It converts every count into a binary label
B. It prevents extremely frequent pairs from dominating while retaining informative counts
C. It removes the need for word and context bias terms
D. It makes all unseen pairs contribute equally

53 The idealized GloVe relation is . Why does the logarithm help model co-occurrence information?

GloVe embeddings Hard
A. It makes the objective independent of vocabulary size
B. It converts multiplicative count relationships into additive relationships
C. It preserves huge count differences without compression
D. It forces all context probabilities to become uniform

54 Suppose two words have identical rows in a co-occurrence matrix but different overall corpus frequencies. Which modeling component can account for part of their difference in GloVe?

GloVe embeddings Hard
A. The word and context bias terms
B. The removal of all zero entries
C. A mandatory orthogonality constraint
D. The t-SNE perplexity parameter

55 Two word vectors have cosine similarity , but their nearest neighbors are dominated by words from the same document genre. What is the most defensible interpretation?

Capturing semantic similarity Hard
A. The words are guaranteed to be synonyms
B. The vectors must have equal Euclidean norms
C. The similarity may reflect shared topical or corpus-specific usage
D. The embedding has no information about word meaning

56 A similarity benchmark contains many rare words, but the embedding was trained on a small corpus. Which evaluation issue is most important to inspect before interpreting a low score?

Capturing semantic similarity Hard
A. Whether every vector has exactly one nonzero coordinate
B. Whether the benchmark contains only antonym pairs
C. Whether cosine similarity always equals Euclidean distance
D. Whether rare words have reliable representations and sufficient counts

57 For the analogy Paris : France :: Tokyo : ?, the standard vector-offset method ranks candidates using . Which failure mode is especially plausible?

Analogy relationships Hard
A. The result is guaranteed to identify a capital city
B. The method cannot perform vector addition
C. The method requires all words to have identical spellings
D. The result can be biased by the individual words' neighborhood frequencies

58 A gender analogy benchmark shows that is not parallel to . Which conclusion is justified?

Analogy relationships Hard
A. The embedding contains no syntactic information
B. The embedding fails to encode every possible relation
C. Cosine similarity cannot be computed for the vocabulary
D. The particular relation is not represented by a single consistent offset

59 A researcher applies PCA to centered word vectors and plots the first two components. Which statement is correct?

Visualizing embedding spaces using PCA or t-SNE Hard
A. PCA removes frequency and syntactic information automatically
B. PCA preserves all pairwise cosine similarities exactly
C. PCA chooses orthogonal directions capturing maximum projected variance
D. PCA guarantees that semantic clusters remain separated

60 Two t-SNE plots of the same embeddings produce different global arrangements after changing the random seed. Which interpretation is most appropriate?

Visualizing embedding spaces using PCA or t-SNE Hard
A. The embedding changed because t-SNE rewrites the original vectors
B. Global distances and orientations should be interpreted cautiously
C. The plot with larger clusters is necessarily more accurate
D. Every local neighborhood in both plots must be identical
Previous MCQ
Unit 1
Next MCQ
Unit 3