This is a collaborative post from Databricks and dbt. We thank Florian Eiden, Staff Product Manager at dbt, for his contributions.
dbt is a data transformation framework that enables data teams to collaboratively model, test and document data in data warehouses. It is popular with Databricks customers, who use it to transform and prepare data in the lakehouse. Many of these customers including Conde Nast, Red Ventures, Loft and Aktify also use dbt Cloud to develop, test, schedule, and investigate dbt data models through a collaborative web-based UI.
Today, we are happy to announce that dbt Cloud has added support for dbt-databricks, a purpose-built dbt adapter for Databricks. This adapter lets dbt Cloud users take advantage of Unity Catalog’s three-level namespace, enabling clean separation of tables and views managed by dbt models as well as data governance using standard SQL. The adapter also includes great out-of-the-box performance and smart defaults, which means less tuning and toil for data teams.
All new Databricks projects created in dbt Cloud will automatically use the new dbt-databricks adapter. Existing projects need to be migrated. We have published a handy migration guide so you can safely test and migrate your project.
We are excited about the partnership between Databricks and dbt Cloud. We would love to hear your feedback on Databricks Community or in the #db-databricks-and-spark dbt community Slack channel.
