For customers on the E2 Platform, Databricks has a feature that allows them to use AWS PrivateLink to provision secure private workspaces by creating VPC endpoints to both the front-end and back-end interfaces of the Databricks infrastructure. The front-end VPC endpoint ensures that users connect to the Databricks web application, REST APIs and JDBC/ODBC interface over their private network. The back-end VPC endpoints ensure that clusters in their own managed VPC connect to the secure cluster connectivity relay and REST APIs over the AWS network backbone.
We previously covered how customers can leverage AWS Route 53 Outbound resolver endpoints to allow workspaces deployed on their own VPC to resolve custom hostnames that can be hosted on customer managed DNS servers. When using PrivateLink for front-end, the workspace URL will need to resolve to the private IP of the PrivateLink interface in order to enable access to the workspace via a private connectivity (from on-premises or other connected VPCs).
In this blog we are going to show how to leverage Route 53 Inbound Endpoints to enable DNS name resolution of workspaces with PrivateLink enabled for the front-end interface. We will also demonstrate how customers using Terraform for managing workspace deployments can add this configuration to their pipeline and automatically make private workspaces accessible over a private network.