Back to Subject

Unit 2: Word Embeddings and Vector Representations - Practice Quiz

CSE472 — Deep Learning For Natural Language Processing 60 Questions
0 Correct 0 Wrong 60 Left
0/60

1 How does a vector space model represent a word?

Vector space models Easy
A. As a numerical vector
B. As a grammar rule
C. As an audio signal
D. As a database table containing every sentence in the corpus

2 In a vector space model, what does each dimension commonly represent?

Vector space models Easy
A. A numerical feature
B. A punctuation rule
C. A neural network layer used only to store labels
D. A complete paragraph

3 What is a dense word embedding?

Dense word embeddings Easy
A. A table that maps sentences directly to grammar rules
B. A long vector with mostly zero values
C. A short vector with mostly nonzero values
D. A list containing only word frequencies

4 What is one advantage of dense embeddings over one-hot vectors?

Dense word embeddings Easy
A. They preserve the exact position of every word in every document
B. They always require more dimensions
C. They assign words identical vectors
D. They can encode semantic relationships

5 What is Word2Vec primarily used to learn?

Word2Vec Easy
A. Handwritten rules for parsing every possible sentence
B. Word embeddings
C. Sentence punctuation
D. Document file formats

6 Which two architectures are associated with Word2Vec?

Word2Vec Easy
A. CBOW and Skip-Gram
B. PCA and t-SNE
C. GloVe and a large manually constructed dictionary of synonyms
D. CNN and pooling

7 What does CBOW predict during training?

CBOW Easy
A. A target word from context words
B. Context words from a target word
C. The full grammatical structure of an entire document
D. A sentence from one character

8 What does the abbreviation CBOW stand for?

CBOW Easy
A. Context-Based Ordered Words
B. Combined Batch of Words
C. Continuous Bag of Words
D. Continuous Binary Output Weights for language modeling

9 What does a Skip-Gram model predict?

Skip-Gram models Easy
A. A target word from context words
B. Word length from character count
C. A complete translation using only the punctuation marks
D. Context words from a target word

10 If the target word is "coffee", what would Skip-Gram try to learn from a nearby word such as "cup"?

Skip-Gram models Easy
A. That "cup" must replace "coffee"
B. That "cup" can occur near "coffee"
C. That both words have equal lengths
D. That every sentence containing "coffee" must end with "cup"

11 What does GloVe stand for?

GloVe embeddings Easy
A. General Language Output
B. Global Vectors
C. Grouped Vocabulary
D. Globally Validated Linguistic Output Vector Encoding

12 What type of information does GloVe mainly use to learn embeddings?

GloVe embeddings Easy
A. Only alphabetical word order
B. Global word co-occurrence statistics
C. A manually written semantic definition for every word
D. Only sentence punctuation counts

13 In a well-trained embedding space, how are semantically similar words usually positioned?

Capturing semantic similarity Easy
A. Close together
B. In separate spaces created from the spelling of each word
C. Far apart
D. At the origin

14 Which measure is commonly used to compare the similarity of two word vectors?

Capturing semantic similarity Easy
A. Cosine similarity
B. The number of documents stored in the training directory
C. Sentence length
D. Character count

15 Which pair would most likely have high semantic similarity in an embedding space?

Capturing semantic similarity Easy
A. car and banana
B. car and a detailed rule describing plural noun formation
C. car and automobile
D. car and quickly

16 Which word best completes the analogy "king is to queen as man is to ___"?

Analogy relationships Easy
A. a person who rules a large country from a royal residence
B. palace
C. child
D. woman

17 Which vector expression is commonly associated with the analogy "king - man + woman"?

Analogy relationships Easy
A. A vector near "prince"
B. A vector representing every royal title in the training corpus
C. A vector near "castle"
D. A vector near "queen"

18 Why are PCA and t-SNE used with word embeddings?

Visualizing embedding spaces using PCA or t-SNE Easy
A. To increase the number of dimensions
B. To translate words into another language
C. To replace all learned vectors with fixed one-hot encodings
D. To reduce dimensions for visualization

19 What type of method is PCA?

Visualizing embedding spaces using PCA or t-SNE Easy
A. A word prediction architecture
B. A linear dimensionality reduction method
C. A nonlinear method designed exclusively for generating text
D. A global co-occurrence model

20 What is t-SNE especially useful for showing in a two-dimensional embedding plot?

Visualizing embedding spaces using PCA or t-SNE Easy
A. Exact word frequency counts
B. Local clusters of similar words
C. The original vector dimensions
D. A guaranteed preservation of every global distance in the dataset

21 A corpus-based retrieval system represents documents using TF-IDF vectors. Why will the term "photosynthesis" receive a higher weight in a document than the term "the"?

Vector space models Medium
A. It has more characters than most other terms
B. It is rare in the document but frequent across the corpus
C. It is frequent in the document but rare across the corpus
D. It appears near the beginning of the document

22 A vocabulary contains 50,000 words. One model uses one-hot vectors, while another uses 300-dimensional dense embeddings. What is the main representational advantage of the dense embeddings?

Dense word embeddings Medium
A. They guarantee correct meanings for unseen words
B. They encode similarity using far fewer dimensions
C. They assign a unique dimension to every word
D. They eliminate the need to train model parameters

23 After training Word2Vec, the embeddings of "doctor" and "nurse" are close even though the words rarely appear next to each other. Which observation best explains this result?

Word2Vec Medium
A. They have nearly identical corpus frequencies
B. They tend to occur in similar linguistic contexts
C. They occupy nearby positions in the vocabulary
D. They contain a similar number of characters

24 For the sentence "the cat sleeps on the mat" with target word "sleeps" and context window size 2, which input-output pair is used by CBOW?

CBOW Medium
A. Input: "the", "cat", "on", "the"; output: "sleeps"
B. Input: "sleeps"; output: "the", "cat", "on", "the"
C. Input: "cat", "on"; output: "sleeps", "mat"
D. Input: "the", "cat"; output: "on", "the"

25 For the sentence "deep models learn useful features" with center word "learn" and context window size 1, which training pairs are produced by Skip-Gram?

Skip-Gram models Medium
A. ("models", "learn") and ("useful", "learn")
B. ("learn", "models") and ("learn", "useful")
C. ("models", "useful") and ("deep", "features")
D. ("learn", "deep") and ("learn", "features")

26 A Word2Vec model has a vocabulary of one million words. Why is negative sampling commonly used during training?

Word2Vec Medium
A. It converts dense embedding vectors into sparse vectors
B. It avoids computing a full softmax over the vocabulary
C. It removes every infrequent word from the training corpus
D. It fixes the embedding dimensions before optimization

27 Which corpus statistic is used most directly when training GloVe embeddings?

GloVe embeddings Medium
A. Sentence labels assigned by annotators
B. Character-level edit distances between words
C. Document lengths measured in tokens
D. Global word-context co-occurrence counts

28 Two normalized word vectors are and . What is their cosine similarity?

Capturing semantic similarity Medium
A.
B.
C.
D.

29 Which vector expression is typically used to solve the analogy "Paris is to France as Tokyo is to ___"?

Analogy relationships Medium
A.
B.
C.
D.

30 A researcher wants a two-dimensional embedding plot that preserves as much global variance as possible and gives reproducible results without random initialization. Which method is more appropriate?

Visualizing embedding spaces using PCA or t-SNE Medium
A. PCA
B. Negative sampling
C. t-SNE
D. Skip-Gram

31 In a t-SNE plot, two clusters appear far apart. Which conclusion is most defensible?

Visualizing embedding spaces using PCA or t-SNE Medium
A. Every pairwise distance has been preserved by the projection
B. Local neighborhoods may be meaningful, but cluster distance may not be
C. The clusters must represent completely unrelated semantic categories
D. The clusters are equally distant in the original embedding space

32 A Skip-Gram model is retrained with a larger context window. What change is most likely in the learned similarities?

Skip-Gram models Medium
A. Words with related topics become more likely to be similar
B. Words outside each sentence receive exactly equal vectors
C. Words with identical spellings become the only close neighbors
D. Words are represented using fewer embedding dimensions

33 A single pretrained embedding for "bank" is used in both "river bank" and "bank loan". What limitation does this demonstrate?

Dense word embeddings Medium
A. A cosine score cannot compare two word vectors
B. A one-hot vector is required for ambiguous words
C. A static vector merges multiple senses of a word
D. A dense vector cannot represent frequent vocabulary words

34 A nearest-neighbor system uses raw dot products between word embeddings whose norms vary greatly. Long-norm vectors dominate the results. Which change best focuses the comparison on vector direction?

Capturing semantic similarity Medium
A. Increase every embedding dimension by the same constant
B. Normalize the vectors and use cosine similarity
C. Replace each vector with its largest coordinate
D. Sort the words by frequency before comparing them

35 Why does the GloVe objective use a weighting function for word-context co-occurrence counts?

GloVe embeddings Medium
A. To remove context words that occur only once
B. To make every vocabulary word equally frequent
C. To limit the influence of extremely common co-occurrences
D. To force all co-occurrence counts to become binary

36 Two CBOW examples contain the same context words in different orders. If the model averages context embeddings, how will their context representations compare?

CBOW Medium
A. They will differ because averaging preserves word positions
B. They will be orthogonal because the orders are reversed
C. They will be identical because averaging ignores order
D. They will be sparse because the context has multiple words

37 A corpus contains many rare technical terms. Compared with CBOW, why might Skip-Gram produce better representations for those terms?

Skip-Gram models Medium
A. It creates several prediction pairs from each center-word occurrence
B. It assigns rare terms vectors without any training
C. It predicts the center word from one averaged context
D. It replaces every rare term with a common synonym

38 Documents and have cosine similarity , while documents and have cosine similarity . What is the best interpretation?

Vector space models Medium
A. and have more similar term distributions
B. is necessarily longer than both other documents
C. and contain exactly the same terms
D. contains more unique words than

39 Suppose is close to . What relationship has the embedding space most likely captured?

Analogy relationships Medium
A. A base-verb to present-participle relationship
B. A singular-noun to plural-noun relationship
C. A country to capital-city relationship
D. A positive-word to negative-word relationship

40 A researcher has 300-dimensional embeddings for 200,000 words and wants to apply t-SNE efficiently. Which preprocessing strategy is commonly appropriate?

Visualizing embedding spaces using PCA or t-SNE Medium
A. Duplicate each embedding before running the projection
B. Replace each vector with its original corpus frequency
C. Use PCA first to reduce to a moderate number of dimensions
D. Convert every embedding into a one-hot vocabulary vector

41 A corpus contains the terms bank, river, and loan. In a count-based vector space model, bank has high co-occurrence counts with both river and loan, while river and loan rarely co-occur. Which limitation is most directly illustrated?

Vector space models Hard
A. The model always assigns identical vectors to synonyms
B. The model cannot compare vectors with cosine similarity
C. The model may conflate unrelated senses of a polysemous word
D. The model cannot represent word frequency

42 Two word vectors have the same Euclidean norm, but one vector contains a large common-frequency component shared by many words. Which transformation most directly reduces the influence of that shared component when measuring lexical similarity?

Vector space models Hard
A. Duplicate the common-frequency component
B. Replace each vector with its word frequency
C. Use cosine similarity after centering the vectors
D. Use Euclidean distance without normalization

43 A dense embedding has dimension , and its training corpus contains distinct word types. Which statement best explains why the embedding is not merely a compressed one-hot representation?

Dense word embeddings Hard
A. Each word still occupies exactly one independent coordinate
B. The vectors are learned so dimensions encode distributed context patterns
C. The vectors preserve every original co-occurrence count exactly
D. The embedding guarantees one unique dimension for every word

44 A rare word appears only in contexts associated with a frequent synonym. During embedding training, which outcome is most plausible, assuming the rare word receives enough updates to be retained?

Dense word embeddings Hard
A. Its vector is determined only by its spelling
B. Its vector must become orthogonal to the synonym vector
C. Its vector becomes exactly equal to the synonym vector
D. Its vector can approach the synonym vector but remains data- and optimization-dependent

45 In the negative-sampling objective for a positive pair , the model maximizes and adds noise terms. If the noise distribution is changed from uniform to a unigram distribution raised to the power , what is the primary effect?

Word2Vec Hard
A. The positive pair is replaced by independent positive pairs
B. The noise distribution balances frequent and rare words more effectively
C. Rare words are excluded from the noise distribution
D. Frequent words are sampled less often than under uniform sampling

46 Why can two independently trained Word2Vec models have excellent intrinsic similarity scores while their raw vector coordinates differ substantially?

Word2Vec Hard
A. Word2Vec stores vectors as unordered frequency lists
B. Similarity scores ignore all relationships among words
C. The objective is invariant to certain transformations of the embedding space
D. The models must use different vocabulary encodings

47 A CBOW model predicts a center word from the average of its surrounding context vectors. If one context word is highly frequent and semantically uninformative, which modification most directly reduces its disproportionate training influence?

CBOW Hard
A. Remove negative sampling from the objective
B. Increase the context window for every example
C. Replace the average with a one-hot center vector
D. Apply subsampling to frequent words

48 Consider a CBOW model using the sum of context embeddings to predict the target. Compared with a model using the average, what changes when the context window contains twice as many tokens, assuming the embeddings are unchanged?

CBOW Hard
A. The input direction always becomes orthogonal to the original
B. The input magnitude generally increases under the sum
C. The target vocabulary is automatically doubled
D. The model receives no additional contextual signal

49 For a fixed corpus and number of center words, which training pattern usually creates more prediction pairs in Skip-Gram than in CBOW?

Skip-Gram models Hard
A. Each sentence is represented by one vocabulary vector
B. One context aggregate is predicted from each target
C. Each center word predicts multiple context words separately
D. Only the most frequent context word is retained

50 A corpus contains many rare technical terms, and the training budget is limited. Why might Skip-Gram be preferred over CBOW for learning those terms?

Skip-Gram models Hard
A. Skip-Gram predicts context words from each center word and can provide stronger signals for rare centers
B. Skip-Gram shares one update across all rare terms
C. Skip-Gram represents every word with a separate vocabulary dimension
D. Skip-Gram never uses negative examples

51 In Skip-Gram with negative sampling, a positive pair has a dot product that is already very large and positive. What is the likely training behavior for that positive term?

Skip-Gram models Hard
A. Its label changes from positive to negative
B. Its sigmoid gradient becomes small
C. Its sigmoid gradient becomes maximal
D. Its context vector is removed from the vocabulary

52 GloVe uses a weighted least-squares objective over word-context co-occurrence counts. Why is a weighting function useful?

GloVe embeddings Hard
A. It converts every count into a binary label
B. It makes all unseen pairs contribute equally
C. It prevents extremely frequent pairs from dominating while retaining informative counts
D. It removes the need for word and context bias terms

53 The idealized GloVe relation is . Why does the logarithm help model co-occurrence information?

GloVe embeddings Hard
A. It forces all context probabilities to become uniform
B. It preserves huge count differences without compression
C. It converts multiplicative count relationships into additive relationships
D. It makes the objective independent of vocabulary size

54 Suppose two words have identical rows in a co-occurrence matrix but different overall corpus frequencies. Which modeling component can account for part of their difference in GloVe?

GloVe embeddings Hard
A. A mandatory orthogonality constraint
B. The removal of all zero entries
C. The word and context bias terms
D. The t-SNE perplexity parameter

55 Two word vectors have cosine similarity , but their nearest neighbors are dominated by words from the same document genre. What is the most defensible interpretation?

Capturing semantic similarity Hard
A. The similarity may reflect shared topical or corpus-specific usage
B. The vectors must have equal Euclidean norms
C. The words are guaranteed to be synonyms
D. The embedding has no information about word meaning

56 A similarity benchmark contains many rare words, but the embedding was trained on a small corpus. Which evaluation issue is most important to inspect before interpreting a low score?

Capturing semantic similarity Hard
A. Whether every vector has exactly one nonzero coordinate
B. Whether cosine similarity always equals Euclidean distance
C. Whether rare words have reliable representations and sufficient counts
D. Whether the benchmark contains only antonym pairs

57 For the analogy Paris : France :: Tokyo : ?, the standard vector-offset method ranks candidates using . Which failure mode is especially plausible?

Analogy relationships Hard
A. The result is guaranteed to identify a capital city
B. The method requires all words to have identical spellings
C. The method cannot perform vector addition
D. The result can be biased by the individual words' neighborhood frequencies

58 A gender analogy benchmark shows that is not parallel to . Which conclusion is justified?

Analogy relationships Hard
A. Cosine similarity cannot be computed for the vocabulary
B. The embedding contains no syntactic information
C. The embedding fails to encode every possible relation
D. The particular relation is not represented by a single consistent offset

59 A researcher applies PCA to centered word vectors and plots the first two components. Which statement is correct?

Visualizing embedding spaces using PCA or t-SNE Hard
A. PCA removes frequency and syntactic information automatically
B. PCA chooses orthogonal directions capturing maximum projected variance
C. PCA guarantees that semantic clusters remain separated
D. PCA preserves all pairwise cosine similarities exactly

60 Two t-SNE plots of the same embeddings produce different global arrangements after changing the random seed. Which interpretation is most appropriate?

Visualizing embedding spaces using PCA or t-SNE Hard
A. Every local neighborhood in both plots must be identical
B. Global distances and orientations should be interpreted cautiously
C. The plot with larger clusters is necessarily more accurate
D. The embedding changed because t-SNE rewrites the original vectors
Previous MCQ
Unit 1
Next MCQ
Unit 3