We are excited to announce that PrivateLink and using customer-managed keys (CMK) for encryption are now Generally Available (GA) for Databricks on AWS! We know that data is your most valuable asset, and the GA of these two key security features will deliver additional control and protection of your data – at rest and in transit – on the Databricks Lakehouse Platform.
PrivateLink and customer-managed keys are two of the most sought after features for customers in highly regulated industries such as Financial Services and Health and Life Sciences. With general availability, customers can leverage PrivateLink and customer-managed keys in environments that require a GA guarantee, extending the benefits of the Databricks Lakehouse Platform to even their most sensitive use cases.
This blog will highlight the benefits of using PrivateLink and CMK for Databricks on AWS, including how to get started with these features today.
Secure your data with AWS PrivateLink
Many customers want the guarantee of private networking to ensure that their users can access data without exposing traffic to a public network. AWS PrivateLink provides a private network route from one AWS environment to another. Now, Databricks customers on AWS can configure PrivateLink between Databricks users and the control plane and between the control plane and the data plane. Using PrivateLink for Databricks on AWS provides the following benefits:
- End-to-end private networking: With PrivateLink, you can set up Databricks workspaces that route traffic privately from your users to your data and back again. Routing traffic on private networks substantially reduces the risk of accidental misconfiguration or traffic inspection by very advanced attackers.
- Data exfiltration protection: PrivateLink endpoints grant access to specific resources, allowing you to tightly control network access. In the event of a security incident within your network, only the mapped resource would be accessible, significantly reducing the attack surface for data exfiltration.
- Meet compliance requirements: With PrivateLink, you can set up a secure perimeter around your data to only be processed in trusted private networks. This helps you to meet compliance requirements for even your most sensitive workloads.
Protect your data at rest with customer-managed keys
Databricks encrypts customer content at rest by default within our control plane, but some customers may prefer the ability to use customer-managed keys for added control. With AWS Key Management Service (AWS KMS), Databricks customers can now bring their encryption keys to protect data in managed services and workspace storage, such as notebooks, secrets, Databricks SQL queries, Databricks SQL query history, and EBS volumes.
Using customer-managed keys for Databricks on AWS provides the following benefits:
- More control over your data: Because you manage the key needed to decrypt your data, you have overall control over how and when it can be used. If you delete or revoke access to your key, it isn’t possible for Databricks (or anyone else) to decrypt that data.
- Greater reassurance in the event of a compromise: Like all of the best security teams in the world, we hope for the best but plan for the worst. In the event of a security compromise, you can simply revoke access to your CMK and, with it, our ongoing access to your data.
- Enforce your own rotation policies: If you use a platform-managed key (PMK), the owner rotates the key per their compliance policy. With a CMK you can rotate the key as per your compliance policy.
- Monitor access: As well as greater control, you have visibility over how and when your key is being used. You can use cloud-native monitoring solutions to track the use of your CMK and detect any unauthorized attempts to access your data.
Getting Started with PrivateLink and CMK on Databricks
PrivateLink and customer-managed keys are available on the Enterprise pricing tier of Databricks on AWS. For step-by-step instructions on configuring these features for your Databricks workspaces on AWS, refer to our documentation (PrivateLink | CMK).
Please visit our Security and Trust Center for more information about Databricks security practices and features available to customers.
