At Databricks, we offer maximal flexibility for choosing compute for ETL and ML/AI workloads. Staying true to the theme of flexibility, we announce support for additional instance types with Photon on Azure, including default VMs. These choices allow customers to improve performance while reducing the total cost of ownership (TCO). The newly supported VMs include Standard_DS3_v2, which has been added to the growing list of supported VMs. With these instance types, Photon reduces your TCO, making data engineering, analysis, and ML/AI workloads faster. So you pay less overall for performance.
Improving Price-Performance with Photon Acceleration
Since the GA of Photon, the Databricks engineering team has focused on broadening expression and operator coverage in Photon with each release. We have worked on improving the memory utilization of expressions and operators along with the ability to spill when there is memory pressure. With these improvements and optimizations, Photon can now be leveraged against general purpose VMs without losing stability. Photon can be used on a wide spectrum of workloads ranging from raw data processing to optimized execution of queries on highly curated and structured data from BI dashboards. The expansion of supported instance types improves price-performance across all clouds.
Let’s look at the data before and after enabling Photon with Databricks Runtime 12.2 using the newly supported instance types.
Summary of Results
Photon consistently delivers price-performance advantages across TPC-DS and TPC-H derived benchmarks. You see clear price-performance benefits and workload acceleration. TCO gains improved between 2.1 and 2.6x, while the performance boost measured 3.4x to 4.3x using standard benchmarks.
What’s Next for Instance Type Support?
We continue to invest in Photon to lower our customers’ total costs. Stay tuned as we aim to bring Photon support to a broader range of compute-optimized instances.
Want to learn more about how customers are leveraging the Photon to reduce their costs by more than 30%? Check out this customer-written blog from Vizio.
