Unit 6: Generative NLP and LLMs - Subjective Questions

Generative NLP models learn the probability distribution of language and can produce new text. They estimate probabilities such as or the conditional probability .

Key characteristics:

• They generate sequences token by token.
• They learn patterns of grammar, meaning, style, and context from training data.
• Examples include language models, sequence-to-sequence models, and large language models.

Discriminative models directly learn a decision boundary or conditional mapping for tasks such as classification. For example, a sentiment classifier estimates and predicts a label.

The main difference is that generative models can create new content, while discriminative models primarily distinguish between existing classes or make predictions.

An autoregressive language model generates text one token at a time. Given a token sequence , it predicts the probability distribution of the next token .

The probability of a complete sequence can be factorized as:

Generation steps:

• Start with a prompt or beginning-of-sequence token.
• Predict a probability distribution for the next token.
• Select a token using a decoding strategy.
• Append the selected token to the context.
• Repeat until an end token or length limit is reached.

This process allows the model to use previously generated context, but errors early in generation can influence later tokens.

Greedy search selects the highest-probability token at every generation step. If the model assigns probabilities to candidate tokens, greedy search chooses:

Advantages:

• It is simple and computationally efficient.
• It requires very little memory.
• It produces deterministic output when the model is fixed.

Limitations:

• A locally optimal token may lead to a poor complete sequence.
• It cannot revise earlier decisions.
• It may produce repetitive, generic, or incomplete text.
• It often performs poorly when several plausible continuations exist.

Therefore, greedy search is useful for fast generation, but it may not provide the best overall sequence.

Beam search keeps the best partial sequences, called beams, at each generation step. Instead of retaining only one token sequence, it expands every current beam with possible next tokens and retains the sequences with the highest scores.

For a sequence , the log-probability score is:

Using log probabilities avoids numerical underflow and converts multiplication into addition. To reduce the preference for short sequences, length normalization may be used:

where controls the strength of length normalization.

Benefits: Beam search explores more alternatives than greedy search and generally produces higher-probability sequences. Its disadvantages are greater computational cost, possible repetitive output, and a tendency to favor safe or common wording.

Greedy search and beam search are deterministic decoding methods, but they differ in how many alternatives they explore.

Greedy search is appropriate when speed is important. Beam search is preferable when sequence-level quality matters and additional computation is available. Neither method guarantees the globally optimal sequence because search is still limited by the beam width and model probabilities.

Top- sampling restricts the candidate set at each step to the tokens with the highest predicted probabilities. The probabilities of these tokens are then renormalized:

where is the set of top- tokens.

Effect of the value of :

• A small produces focused, predictable, and less diverse text.
• A large allows more creativity but increases the risk of irrelevant or low-quality tokens.
• Setting is equivalent to greedy selection if sampling is deterministic.

Top- sampling avoids selecting from the entire vocabulary, where many very unlikely tokens could create incoherent output. However, a fixed may be too restrictive when the distribution is sharp and too permissive when the distribution is flat.

Nucleus sampling, or top- sampling, selects the smallest set of tokens whose cumulative probability is at least . If the tokens are ordered by decreasing probability, the nucleus is:

The model samples only from after renormalizing the probabilities.

Difference from top- sampling:

• Top- always retains a fixed number of tokens.
• Nucleus sampling retains a variable number of tokens based on the uncertainty of the distribution.
• When the model is confident, the nucleus may be small.
• When the model is uncertain, the nucleus expands to include more alternatives.

Nucleus sampling often gives a better balance between coherence and diversity because it adapts to the shape of the probability distribution.

These decoding controls modify the probability distribution before the next token is selected.

Temperature: The logits are divided by a temperature value :

• sharpens the distribution and produces safer, more predictable output.
• flattens the distribution and increases diversity.
• Very high temperature can produce incoherent text.

Top- sampling: Keeps a fixed number of high-probability candidates.

Top- sampling: Keeps the smallest adaptive candidate set whose cumulative probability reaches .

These methods can be combined, such as applying temperature scaling followed by top- filtering. The appropriate settings depend on whether the task prioritizes factuality, creativity, diversity, or consistency.

An instruction-tuned large language model is a pretrained language model that is further trained to follow natural-language instructions and produce useful responses.

Instruction-tuning process:

• Collect examples containing instructions, inputs, and desired outputs.
• Fine-tune the pretrained model on these supervised demonstrations.
• Often apply preference optimization or human feedback to improve helpfulness and alignment.

Improvements:

• Better understanding of user intent.
• More reliable adherence to requested formats and constraints.
• Improved performance on zero-shot and few-shot tasks.
• More appropriate conversational style.
• Reduced tendency to continue a prompt without answering it.

Instruction tuning does not create knowledge from nothing. It primarily changes how the model uses its learned knowledge and responds to user requests, so it can still make factual errors or fail on ambiguous instructions.

In abstractive summarization, an LLM produces a shorter representation of a source document using new wording rather than copying only source sentences.

Typical behavior:

• It identifies salient topics and events.
• It compresses repeated information.
• It reorganizes content into a coherent narrative.
• It may paraphrase or infer connections between statements.

Advantages:

• The summary can be fluent and concise.
• It can adapt to a requested style or audience.
• It can combine information from different parts of a document.

Risks:

• Hallucinated facts or unsupported conclusions.
• Omission of important details.
• Distortion of numbers, names, dates, or negation.
• Bias inherited from the source or model.

Faithfulness checks, source grounding, and human review are important when summaries are used in high-stakes settings.

Dialogue generation is the task of producing a response conditioned on the conversation history, the current user message, and sometimes a system instruction or external knowledge source.

A high-quality response should have the following properties:

• Relevance: It directly addresses the user's latest request.
• Coherence: It is consistent with earlier turns.
• Informativeness: It provides enough useful detail without unnecessary repetition.
• Conversational appropriateness: It uses a suitable tone and level of formality.
• Consistency: It maintains stable facts, roles, and goals throughout the conversation.
• Safety: It avoids harmful, private, or unsupported content.

Common problems include repetition, topic drift, contradictions, overconfident answers, and failure to ask clarifying questions. Dialogue systems can be improved through instruction tuning, conversation-state tracking, retrieval, safety filters, and preference-based training.

LLMs can support reasoning tasks such as arithmetic, logical deduction, planning, code generation, and multi-step question answering. They generate intermediate steps or use structured procedures to connect premises to a conclusion.

Answer accuracy measures whether the final answer is correct. Reasoning reliability asks whether the process is valid, consistent, and supported by the given information.

A model may produce a correct answer using an invalid explanation, or produce a convincing explanation containing hidden errors. Reliability can be improved through:

• Decomposing complex tasks into smaller subproblems.
• Using tools such as calculators, search systems, or code interpreters.
• Generating multiple solutions and checking agreement.
• Verifying each intermediate step.
• Requiring evidence or citations for factual claims.

Thus, a correct final answer alone is insufficient for evaluating reasoning in safety-critical applications.

For a token sequence , the average negative log-likelihood is:

Perplexity is the exponential of this average loss:

If logarithms with base 2 are used, the equivalent form is:

Lower perplexity means that the model assigns higher probability to the observed sequence. It can be interpreted as the model's average uncertainty, measured in an effective number of equally likely choices. Perplexity comparisons are meaningful only when models use the same evaluation data, tokenization, and preprocessing.

Perplexity measures how well a language model predicts reference tokens, but it does not fully measure the quality of generated responses.

Limitations:

• A low-perplexity model may generate bland or repetitive text.
• It does not directly evaluate factual correctness.
• It may not reflect relevance to a user's instruction.
• It is highly affected by tokenization and the evaluation corpus.
• It does not measure dialogue helpfulness, safety, or emotional appropriateness.
• A generated response can be valid even when it differs substantially from the reference, which may result in a poor perplexity score.

Therefore, perplexity should be combined with task-specific metrics, factuality checks, diversity measures, and human evaluation. It is most useful as a diagnostic measure of language-model fit rather than a complete quality score.

Different metrics capture different properties of generated text.

• BLEU: Measures modified -gram precision, mainly for machine translation. It rewards overlap with a reference but may miss valid paraphrases.
• ROUGE: Measures recall-oriented overlap. ROUGE- uses unigrams, ROUGE- uses bigrams, and ROUGE-L uses the longest common subsequence. It is widely used for summarization.
• METEOR: Uses unigram alignment and can account for stemming and synonyms.
• BERTScore: Compares contextual embeddings between the candidate and reference, allowing semantic similarity beyond exact word overlap.
• Distinct-: Measures the proportion of unique -grams and is used to estimate generation diversity.
• Perplexity: Measures the probability assigned by a language model to a sequence.

No single metric captures fluency, relevance, factuality, coherence, and diversity simultaneously. Evaluation should therefore use several complementary metrics.

Human judgment measures use evaluators to assess qualities that are difficult to capture with automatic metrics.

Common dimensions include:

• Fluency: Whether the response is grammatical and natural.
• Relevance: Whether it answers the task or question.
• Coherence: Whether its ideas are logically connected.
• Factuality: Whether claims are supported and correct.
• Helpfulness: Whether it satisfies the user's goal.
• Safety: Whether it avoids harmful or inappropriate content.

Reliable collection requires clear rubrics, representative test examples, evaluator training, randomized presentation, and blind comparison when possible. Researchers often use Likert-scale ratings, pairwise preference judgments, or ranking tasks. Agreement statistics such as Cohen's kappa or Krippendorff's alpha can measure consistency between evaluators. Multiple evaluators and adjudication reduce the impact of individual subjectivity.

Dialogue quality is difficult to evaluate because a response can be acceptable in several different ways and its quality depends on the conversation context.

Major challenges:

• There may be multiple valid responses to the same message.
• A response can sound fluent while being factually wrong.
• Quality may depend on previous turns rather than the final response alone.
• Different users prefer different tones and levels of detail.
• Evaluators may reward confident or persuasive language.
• Long conversations make evaluation expensive and inconsistent.

A robust evaluation protocol should provide the complete dialogue context, define separate criteria for relevance, factuality, coherence, helpfulness, and safety, and use multiple independent raters. Pairwise comparisons can be more reliable than absolute scores, but they can still be affected by position bias and evaluator preferences.

Explainability is the ability to understand and communicate why an LLM produced a particular output or behavior. It is important for debugging, auditing, trust, fairness, and safety.

Important approaches include:

• Feature attribution: Estimates which input tokens influenced the output using methods such as gradients or perturbation tests.
• Attention analysis: Examines attention patterns, although attention weights alone are not a complete explanation.
• Counterfactual analysis: Changes parts of the input and observes how the output changes.
• Probing: Trains auxiliary models to test whether specific information is represented internally.
• Activation and circuit analysis: Studies internal neurons, features, or computational pathways.
• Example-based explanations: Retrieves training or similar examples that help explain the response.

Explanations should be validated because a plausible natural-language explanation may not accurately describe the model's actual internal computation.

Plausibility means that an explanation sounds reasonable to a human reader. Faithfulness means that the explanation accurately reflects the factors and processes that caused the model's output.

An LLM may generate a plausible explanation after producing an answer, even when that explanation was not used to reach the answer. This creates a risk of explanation fabrication.

Faithful explanation tests include:

• Removing or changing supposedly important input evidence and checking whether the output changes.
• Comparing explanations with attribution or intervention methods.
• Testing whether the stated reasoning steps are logically valid.
• Measuring sensitivity to relevant and irrelevant information.

A good explainability method should be both understandable and causally informative. Human readability is valuable, but it cannot substitute for evidence that the explanation corresponds to the model's actual behavior.

A hallucination occurs when an LLM generates content that is unsupported, false, or inconsistent with the available context while presenting it as if it were reliable.

Types:

• Factual hallucination: The response contains a false claim about the real world, such as an incorrect date or citation.
• Contextual hallucination: The response contradicts or ignores information provided in the prompt or source document.
• Fabricated content: The model invents entities, references, quotations, events, or details that have no supporting basis.

Causes include:

• Training objectives that reward likely text rather than truth.
• Missing, outdated, or ambiguous knowledge.
• Exposure bias during autoregressive generation.
• Prompts that encourage unsupported completion.
• Excessive sampling randomness.
• Poor retrieval or failure to use retrieved evidence.

Hallucination is especially dangerous in medical, legal, financial, and scientific applications.