Exam NCA-GENL Topic 6 Question 3 Discussion

When training an LLM with a dataset larger than available RAM, using a memory-mapped file is an effective alternative, as discussed in NVIDIA's Generative AI and LLMs course. Memory-mapped files allow the system to access portions of the dataset directly from disk without loading the entire dataset into RAM, enabling efficient handling of large datasets. This approach leverages virtual memory to map file contents to memory, reducing memory bottlenecks. Option A is incorrect, as moving large datasets in and out of GPU memory via PCI bandwidth is inefficient and not a standard practice for dataset storage. Option C is wrong, as discarding data reduces model quality and is not a scalable solution. Option D is inaccurate, as eliminating semantically equivalent sentences is a specific preprocessing step that does not address memory constraints.
The course states: "Memory-mapped files enable efficient training of LLMs on large datasets by accessing data from disk without loading it fully into RAM, overcoming memory limitations." References: NVIDIA Building Transformer-Based Natural Language Processing Applications course; NVIDIA Introduction to Transformer-Based Natural Language Processing.