Snowflake Architecture

Snowflake uses a patented hybrid three-layer architecture designed to combine the advantages of shared-disk and shared-nothing storage structures.

graph TD
    UI[Cloud Services: Optimization, Security, Metadata]
    UI --> CW1[Virtual Warehouse A - Compute]
    UI --> CW2[Virtual Warehouse B - Compute]
    CW1 --> CS[Centralized Hybrid Columnar Storage]
    CW2 --> CS

The Three Architecture Layers

1. Database Storage (Centralized)

When data is loaded into Snowflake, it is reorganized into a proprietary columnar, optimized, compressed format called micro-partitions.

• Immutable Storage: Micro-partitions are immutable, enabling safe concurrency, Time Travel, and Fail-Safe.
• Automatic Clustering: Data is clustered based on natural load orders or specified clustering keys.
• Cloud Object Storage: Backed by standard cloud storage services (AWS S3, Azure Blob, or Google Cloud Storage).

2. Query Processing (Compute)

Query execution is handled by MPP (Massively Parallel Processing) compute clusters known as Virtual Warehouses.

• Isolation: Each warehouse is isolated from others; they do not share compute resources and have no resource contention.
• Dynamic Scaling: Warehouses can be resized dynamically (e.g. Small to X-Large) or auto-scaled (multi-cluster) instantly.
• Ephemeral Caching: Warehouses cache data on SSDs locally to speed up subsequent queries.

3. Cloud Services (The Brain)

A collection of stateless services running across multiple availability zones coordinating operations. It manages:

• Metadata: Micro-partition locations, stats, transaction logs.
• Security & Access: User authentication, RBAC authorization, end-to-end data encryption.
• Query Optimization: Parser, cost-based optimizer, and execution plan generation.

This content is reserved for Premium Members.