Big Data News Hubb
Advertisement
  • Home
  • Big Data
  • News
  • Contact us
No Result
View All Result
  • Home
  • Big Data
  • News
  • Contact us
No Result
View All Result
Big Data News Hubb
No Result
View All Result
Home Big Data

Orchestrate the Lakehouse: Announcing Support for Refreshing SQL Queries, Dashboards and Alerts in Databricks Workflows

admin by admin
November 12, 2022
in Big Data


Databricks Workflows is the highly available, fully-managed orchestrator for all your data, analytics, and AI workloads. Today, we are happy to announce the public preview of the new SQL task type, which lets you easily author, schedule, inspect, and operate workflows that refresh Databricks SQL queries, dashboards and alerts. You can combine this new task type with existing robust support for notebooks, Python scripts and dbt projects to orchestrate the full power of the lakehouse.

Real-world workflows orchestrate many different activities across the lakehouse. Consider a workflow that ingests data into Delta Lake and transforms it using a Delta Live Table. Once this data is updated, you can now instruct Databricks to refresh one or more dashboards or execute SQL queries. You can also run alerts to check various data quality dimensions and send notifications to Slack or Teams if specified conditions are met. Orchestrating these SQL tasks inside the workflow where data is prepared has several benefits: not only is the workflow easier to reason about, it is also easy to repair and pick up where it failed.

A workflow refreshing multiple queries and a dashboard

In order to create a Databricks SQL task, simply go to Databricks Workflows and select “SQL” from the Type dropdown:

 Creating a SQL task to refresh a query
Creating a SQL task to refresh a query

You also benefit from the observability and stability benefits that come with our jobs scheduler such as managing schedules and inspecting previous runs.

Run details for a SQL task
Run details for a SQL task

For pricing and availability in your environment, see the Databricks SQL pricing page (AWS, Azure).



Source link

Previous Post

Learn more about Apache Flink and Amazon Kinesis Data Analytics with three new videos

Next Post

Podcast Highlights: “Intel on AI”: Learning with AI

Next Post

Podcast Highlights: "Intel on AI": Learning with AI

Recommended

Your guide to AWS Analytics at re:Invent 2022

November 6, 2022

LaunchDarkly’s journey from ingesting 1 TB to 100 TB per day with Amazon Kinesis Data Streams

December 15, 2022

Air Force Looks to AI to Help Maintain Bombers, ICBMs

November 27, 2022

Don't miss it

News

Global Benefits of International Courier Services

March 21, 2023
Big Data

An early look at the labor market impact potential of large language models

March 21, 2023
Big Data

Unlocking Hidden Value From Production Line Data

March 20, 2023
Big Data

Fine-Tuning Large Language Models with Hugging Face and DeepSpeed

March 20, 2023
News

Accelerating revenue growth with real-time analytics: Poshmark’s journey

March 20, 2023
News

Big Data Career Notes: March 2023 Edition

March 20, 2023

big-data-footer-white

© 2022 Big Data News Hubb All rights reserved.

Use of these names, logos, and brands does not imply endorsement unless specified. By using this site, you agree to the Privacy Policy and Terms & Conditions.

Navigate Site

  • Home
  • Big Data
  • News
  • Contact us

Newsletter Sign Up

No Result
View All Result
  • Home
  • Big Data
  • News
  • Contact us

© 2022 Big Data News Hubb All rights reserved.