Theoretical Quiz: AI for Data Engineering
Evaluate your architectural understanding of incorporating generative models into the Modern Data Stack.
Question 1
Why is dynamic schema pruning critical when implementing enterprise Text-to-SQL systems across large data warehouses?
- A) It speeds up the database query execution time on the warehouse engine.
- B) It prevents prompt context window overflow, reduces token costs, and minimizes LLM hallucination of irrelevant columns.
- C) It automatically creates missing indexes on the queried tables.
- D) It encrypts the schema metadata before sending it over the network.
Answer: B
Explanation: Including hundreds of irrelevant tables in a prompt confuses LLMs and increases hallucination risks. Dynamic pruning selects only the tables relevant to the user query.
Question 2
What is the primary operational advantage of using OpenAI / Anthropic Batch APIs for ETL transformations over real-time completions?
- A) Batch APIs guarantee zero token usage.
- B) Batch APIs offer a 50% discount on API costs and provide dedicated high-throughput capacity without strict real-time rate limit spikes.
- C) Batch APIs automatically fix SQL syntax errors without LLM calls.
- D) Batch APIs run exclusively on local GPUs.
Answer: B
Explanation: Batch endpoints process async non-time-critical workloads within 24 hours at half the cost and with much higher throughput limits.
Question 3
How does using Pydantic / Instructor improve reliability in AI data pipelines compared to raw string prompting?
- A) It speeds up the GPU hardware clock.
- B) It guarantees deterministic, type-checked JSON schemas with automatic validation, retrying requests automatically when validation fails.
- C) It bypasses API rate limits.
- D) It makes embedding models obsolete.
Answer: B
Explanation: Pydantic validates data types, ranges, and required fields, retrying the prompt with targeted error feedback if the LLM output violates constraints.