Exam NCA-GENL Topic 3 Question 91 Discussion

Perplexity is the primary metric used to evaluate the quality of text generated by language models, as emphasized in NVIDIA's Generative AI and LLMs course. Perplexity measures how well a language model predicts a sequence of tokens, with lower values indicating better performance, as the model is less
"surprised" by the data. It is calculated as the exponentiated average negative log-likelihood of the tokens in a test set, reflecting the model's ability to assign high probabilities to correct sequences. In generative tasks, perplexity is widely used because it directly assesses the model's fluency and coherence. Option B, Precision, and Option C, Recall, are metrics for classification tasks, not text generation. Option D, Accuracy, is also irrelevant for evaluating generative quality, as it applies to categorical predictions. The course notes:
"Perplexity is a key metric for evaluating language models, measuring how well the model predicts text sequences, with lower perplexity indicating higher-quality generation." References: NVIDIA Building Transformer-Based Natural Language Processing Applications course; NVIDIA Introduction to Transformer-Based Natural Language Processing.