Skip to main content

Databricks Databricks

Databricks is a unified analytics platform that provides a collaborative environment for data scientists, data engineers, and analysts to work together on data projects.

Creating a Databricks connection​

Using Service Principal​

To create the connection you need:

  • An API Address
  • A Client ID
  • A Client Secret

Obtaining the credentials​

Step 1: Create a service principal​

You can create a service principal directly in your Databricks account or from a Databricks workspace. If you have identity federation enabled on your workspaces, Databricks recommends creating the service principal in the account and assigning it to workspaces. If you do not have identity federation enabled and you want to use the service principal at the workspace level, you must create your service principal from a workspace.

Identity federation enabled​

To add a service principal to the account using the account console:

  1. As an account admin, log in to the account console.
  2. Navigate to User management.
  3. On the Service principals tab, click Add service principal.
note

In order to call Databricks account-level APIs, on the Roles tab, turn on Account admin.

You can now assign your service principal to identity federated workspaces.

  1. In the account console sidebar, click Workspaces.
  2. Click your workspace name.
  3. On the Permissions tab, click Add permissions.
  4. Search for and select the service principal, assign the permission level (workspace User or Admin), and click Save.
Identity federation disabled​
  1. As a workspace admin, log in to the Databricks workspace.
  2. Click your username in the top bar of the Databricks workspace and select Settings.
  3. Click on the Identity and access tab.
  4. Next to Service principals, click Manage.
  5. Click Add service principal.
  6. Click the drop-down arrow in the search box and then click Add new.
  7. Enter a name for the service principal and click Add.
info

The service principal is added to both your workspace and the Databricks account.

Step 2: Assign workspace-level permissions to the Databricks service principal​

  1. If the admin console for your workspace is not already opened, click your username in the top bar and click Settings.
  2. Click on the Identity and access tab.
  3. Next to Service principals, click Manage.
  4. Click the name of your Databricks service principal to open its settings page.
  5. On the Configurations tab, check the box next to each entitlement that you want your Databricks service principal to have for this workspace, and then click Update.
  6. On the Permissions tab, grant access to any Databricks users, service principals, and groups that you want to manage and use this Databricks service principal.

Step 3: Create an OAuth secret for a service principal​

Before you can use OAuth to authenticate to Databricks, you must first create an OAuth secret, which can be used to generate OAuth access tokens. A service principal can have up to five OAuth secrets. Account admins and workspace admins can create an OAuth secret for a service principal.

Account admins​
  1. As an account admin, log in to the account console.
  2. Click User management.
  3. On the Service principals tab, select your service principal.
  4. Under OAuth secrets, click Generate secret.
  5. Copy the displayed Secret and Client ID, and then click Done. (The client ID is the same as the service principal’s application ID)

Workspace admins​

  1. If the admin console for your workspace is not already opened, click your username in the top bar and click Settings.
  2. Click the Identity and access tab.
  3. Next to Service principals, click Manage.
  4. Click the name of your Databricks service principal to open its settings page.
  5. Navigate to the Secrets tab.
  6. Under OAuth secrets, click Generate secret.
  7. Copy the displayed Secret and Client ID, and then click Done. (The client ID is the same as the service principal’s application ID)

Creating your connection​

  1. In the Blink platform, navigate to the Connections page > Add connection. A New Connection dialog box opens displaying icons of external service providers available.
  2. Select the Databricks icon. A dialog box with name of the connection and connection methods appear.
  3. (Optional) Edit the name of the connection. At a later stage you cannot edit the name.
  4. Select Service Principal as the method to create the connection.
  5. Fill in the parameters:
    • The API Address
    • The Client ID
    • The Client Secret
  6. (Optional) Click Test Connection to test it.
  7. Click Create connection. The new connection appears on the Connections page.