Theoretical Quiz: Hugging Face & Open-Source Models

Question 1

What does the .gguf file format provide that standard Hugging Face safetensors directories do not?

• A) Automatic translation of Python to C++.
• B) A single unified binary containing quantized weights, tokenizer vocabulary, and hyperparameter metadata optimized for memory-mapped CPU/GPU execution with llama.cpp.
• C) It guarantees 100% training convergence in 1 epoch.
• D) It bypasses commercial licensing checks.

Answer: B
Explanation: GGUF packs all model components and quantization data into a single file designed for fast mmap loading.

Question 2

What is the difference between a Base model and an Instruct model on Hugging Face?

• A) Base models can only run on Linux, while Instruct models can run on Windows.
• B) Base models are trained only on next-token prediction, whereas Instruct models have undergone Supervised Fine-Tuning (SFT) and RLHF to follow conversational prompts and instructions.
• C) Base models have fewer parameters than Instruct models.
• D) Instruct models cannot be quantized.

Answer: B
Explanation: Base models simply autocomplete text; Instruct models are specifically aligned to act as helpful chat assistants and follow multi-turn instructions.

Question 3

Why does 4-bit AWQ or GGUF quantization allow a 70B parameter model to run on a machine with 48GB VRAM?

• A) It removes 50% of the layers from the neural network.
• B) It compresses each weight from 16 bits (2 bytes) to 4 bits (0.5 bytes), reducing weight memory from ~140 GB down to ~35-40 GB while retaining attention layers intact.
• C) It offloads all calculations to the cloud.
• D) It doubles the token generation speed by skipping attention calculations.

Answer: B
Explanation: 4-bit quantization reduces memory per parameter by 4x, making high-parameter models fit onto consumer or workstation GPUs.