Databricks Asset Bundles (Declarative Automation Bundles)
Declarative Automation Bundles (formerly known as Databricks Asset Bundles) provide an infrastructure-as-code (IaC) approach to managing Databricks projects. They allow you to define Databricks resources—such as jobs, pipelines, and ML models—as source files alongside your data engineering and machine learning code.
Why Use Bundles?
- Infrastructure as Code (IaC): Bundles let you describe your entire Databricks project in YAML templates, ensuring your infrastructure is version-controlled and reproducible.
- CI/CD Integration: Because bundles define your project configuration and resources in code, they are ideal for automating deployments across development, staging, and production environments using tools like GitHub Actions or Azure DevOps.
- Collaboration: Multiple contributors can easily work on the same project using standard software engineering practices like code review and automated testing.
- Databricks CLI: Bundles are deeply integrated with the Databricks CLI, allowing you to validate, build, and deploy your assets with simple commands like
databricks bundle deploy.
Bundle Structure
A typical bundle includes:
databricks.yml: The main configuration file defining the project, environments, and resources.- Source code: Python files, notebooks, or SQL scripts containing your business logic.
- Tests: Unit tests or integration tests to validate your code before deployment.
Find this content helpful?
☕ Buy me a coffee