Big Data News Hubb
Advertisement
  • Home
  • Big Data
  • News
  • Contact us
No Result
View All Result
  • Home
  • Big Data
  • News
  • Contact us
No Result
View All Result
Big Data News Hubb
No Result
View All Result
Home Big Data

Orchestrate the Lakehouse: Announcing Support for Refreshing SQL Queries, Dashboards and Alerts in Databricks Workflows

admin by admin
November 12, 2022
in Big Data


Databricks Workflows is the highly available, fully-managed orchestrator for all your data, analytics, and AI workloads. Today, we are happy to announce the public preview of the new SQL task type, which lets you easily author, schedule, inspect, and operate workflows that refresh Databricks SQL queries, dashboards and alerts. You can combine this new task type with existing robust support for notebooks, Python scripts and dbt projects to orchestrate the full power of the lakehouse.

Real-world workflows orchestrate many different activities across the lakehouse. Consider a workflow that ingests data into Delta Lake and transforms it using a Delta Live Table. Once this data is updated, you can now instruct Databricks to refresh one or more dashboards or execute SQL queries. You can also run alerts to check various data quality dimensions and send notifications to Slack or Teams if specified conditions are met. Orchestrating these SQL tasks inside the workflow where data is prepared has several benefits: not only is the workflow easier to reason about, it is also easy to repair and pick up where it failed.

A workflow refreshing multiple queries and a dashboard

In order to create a Databricks SQL task, simply go to Databricks Workflows and select “SQL” from the Type dropdown:

 Creating a SQL task to refresh a query
Creating a SQL task to refresh a query

You also benefit from the observability and stability benefits that come with our jobs scheduler such as managing schedules and inspecting previous runs.

Run details for a SQL task
Run details for a SQL task

For pricing and availability in your environment, see the Databricks SQL pricing page (AWS, Azure).



Source link

Previous Post

Learn more about Apache Flink and Amazon Kinesis Data Analytics with three new videos

Next Post

Podcast Highlights: “Intel on AI”: Learning with AI

Next Post

Podcast Highlights: "Intel on AI": Learning with AI

Recommended

Brief History of LLMs – insideBIGDATA

July 18, 2023

Heard on the Street – 2/9/2023

February 9, 2023

BWH Hotels scales enterprise business intelligence adoption while reducing costs with Amazon QuickSight

June 3, 2023

Don't miss it

Big Data

“Above the Trend Line” – Your Industry Rumor Central for 9/29/2023

September 30, 2023
Big Data

Ballard Power Systems RDU (Remote Diagnostics Unit) Visualization Platform for Interactive At-Scale Industrial IoT Streaming Analytics

September 30, 2023
News

Process and analyze highly nested and large XML files using AWS Glue and Amazon Athena

September 30, 2023
News

Rethinking ‘Open’ for AI

September 30, 2023
News

Embracing the New Era of Online Education

September 30, 2023
Big Data

Unleashing the Power of AI in Paid Search Marketing: Insights from Industry Expert

September 29, 2023
big-data-footer-white

© Big Data News Hubb All rights reserved.

Use of these names, logos, and brands does not imply endorsement unless specified. By using this site, you agree to the Privacy Policy and Terms & Conditions.

Navigate Site

  • Home
  • Big Data
  • News
  • Contact us

Newsletter Sign Up

No Result
View All Result
  • Home
  • Big Data
  • News
  • Contact us

© 2022 Big Data News Hubb All rights reserved.