Authenticate with Microsoft Entra service principals

Microsoft Entra service principal authentication uses the credentials of a Microsoft Entra service principal to authenticate. To create and manage service principals for Azure Databricks, see:

• Service principals
• Provision a service principal by using Terraform

To configure Microsoft Entra service principal authentication with Azure Databricks, you must set the following associated environment variables, .databrickscfg fields, Terraform fields, or Config fields:

• The Azure Databricks host.

  • For account operations, specify https://accounts.azuredatabricks.net.
  • For workspace operations, Databricks recommends specifying the per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net and explicitly assigning the Microsoft Entra service principal to the workspace. Alternatively, specify the Azure resource ID. This approach requires Contributor or Owner permissions on the Azure resource, or a custom role with specific Azure Databricks permissions.

• For account operations, the Azure Databricks account ID.

• The Azure resource ID.

• The tenant ID of the Microsoft Entra service principal.

• The client ID of the Microsoft Entra service principal.

• The client secret of the Microsoft Entra service principal.

To perform Microsoft Entra service principal authentication with Azure Databricks, integrate the following within your code, based on the participating tool or SDK:

Environment variables

To use environment variables for a specific Azure Databricks authentication type with a tool or SDK, see Authorize access to Azure Databricks resources or the tool's or SDK's documentation. See also Environment variables and fields for unified authentication and the Authentication method priority.

For account-level operations, set the following environment variables:

• DATABRICKS_HOST, set to the value of your Azure Databricks account console URL, https://accounts.azuredatabricks.net.
• DATABRICKS_ACCOUNT_ID
• ARM_TENANT_ID
• ARM_CLIENT_ID
• ARM_CLIENT_SECRET

For workspace-level operations, set the following environment variables:

• DATABRICKS_HOST, set to the value of your Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net.
• ARM_TENANT_ID
• ARM_CLIENT_ID
• ARM_CLIENT_SECRET

Databricks recommends using DATABRICKS_HOST and explicitly assigning the Microsoft Entra service principal to the workspace. Alternatively, use DATABRICKS_AZURE_RESOURCE_ID with the Azure resource ID. This approach requires Contributor or Owner permissions on the Azure resource, or a custom role with specific Azure Databricks permissions.

Configuration profiles

Create or identify an Azure Databricks configuration profile with the following fields in your .databrickscfg file. If you create the profile, replace the placeholders with the appropriate values. To use the profile with a tool or SDK, see Authorize access to Azure Databricks resources or the tool's or SDK's documentation. See also Environment variables and fields for unified authentication and the Authentication method priority.

For account-level operations, set the following values in your .databrickscfg file. In this case, the Azure Databricks account console URL is https://accounts.azuredatabricks.net:

[<some-unique-configuration-profile-name>]
host                = <account-console-url>
account_id          = <account-id>
azure_tenant_id     = <azure-service-principal-tenant-id>
azure_client_id     = <azure-service-principal-application-id>
azure_client_secret = <azure-service-principal-client-secret>

For workspace-level operations, set the following values in your .databrickscfg file. In this case, the host is the Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net:

[<some-unique-configuration-profile-name>]
host                = <workspace-url>
azure_tenant_id     = <azure-service-principal-tenant-id>
azure_client_id     = <azure-service-principal-application-id>
azure_client_secret = <azure-service-principal-client-secret>

Databricks recommends using host and explicitly assigning the Microsoft Entra service principal to the workspace. Alternatively, use azure_workspace_resource_id with the Azure resource ID. This approach requires Contributor or Owner permissions on the Azure resource, or a custom role with specific Azure Databricks permissions.

Databricks CLI

For the Databricks CLI, do one of the following:

• Set the environment variables as specified on the Environment tab.
• Set the values in your .databrickscfg file as specified on the Profile tab.

Environment variables always take precedence over values in your .databrickscfg file.

See also Microsoft Entra ID service principal authentication.

Databricks Connect

For Databricks Connect, you can either:

• Use a config profile: Set workspace-level values in your .databrickscfg file as described on the Profile tab. Also set the cluster_id to your workspace instance URL.
• Use environment variables: Set the same values as shown on the Environment tab. Also set the DATABRICKS_CLUSTER_ID to your workspace instance URL.

Values in .databrickscfg take precedence over environment variables.

To initialize Databricks Connect with these settings, see Compute configuration for Databricks Connect.

Visual Studio Code extension

For the Databricks extension for Visual Studio Code, do the following:

1. Set the values in your .databrickscfg file for Azure Databricks workspace-level operations as specified on the Profile tab.
2. In the Configuration pane of the Databricks extension for Visual Studio Code, click Configure Databricks.
3. In the Command Palette, for Databricks Host, enter your per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net, and then press Enter.
4. In the Command Palette, select your target profile's name in the list for your URL.

For more details, see Set up authorization for the Databricks extension for Visual Studio Code.

Terraform

For account-level operations, for default authentication:

provider "databricks" {
  alias = "accounts"
}

For direct configuration, replace the placeholders with values from the console or some other configuration store, such as HashiCorp Vault. See also Vault Provider. In this case, the Azure Databricks account console URL is https://accounts.azuredatabricks.net:

provider "databricks" {
  alias               = "accounts"
  host                = <your-account-console-url>
  account_id          = <your-account-id>
  azure_tenant_id     = <your-azure-tenant-id>
  azure_client_id     = <your-azure-client-id>
  azure_client_secret = <your-azure-client-secret>
}

For workspace-level operations, for default authentication:

provider "databricks" {
  alias = "workspace"
}

For direct configuration, replace the placeholders with values from the console or some other configuration store, such as HashiCorp Vault. See also Vault Provider. In this case, the host is the Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net:

provider "databricks" {
  alias               = "workspace"
  host                = <your-workspace-url>
  azure_tenant_id     = <your-azure-tenant-id>
  azure_client_id     = <your-azure-client-id>
  azure_client_secret = <your-azure-client-secret>
}

Databricks recommends using host and explicitly assigning the Microsoft Entra service principal to the workspace. Alternatively, use azure_workspace_resource_id with the Azure resource ID. This approach requires Contributor or Owner permissions on the Azure resource, or a custom role with specific Azure Databricks permissions.

For more information about authenticating with the Databricks Terraform provider, see Authentication.

Databricks SDK for Python

For account-level operations, for default authentication:

from databricks.sdk import AccountClient

a = AccountClient()
# ...

For direct configuration, replace the function calls with code that gets values from the console or some other configuration store, such as Azure KeyVault. In this case, the Azure Databricks account console URL is https://accounts.azuredatabricks.net:

from databricks.sdk import AccountClient

a = AccountClient(
  host                = get_account_console_url(),
  account_id          = get_account_id(),
  azure_tenant_id     = get_azure_tenant_id(),
  azure_client_id     = get_azure_client_id(),
  azure_client_secret = get_azure_client_secret()
)
# ...

For workspace-level operations, for default authentication:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient()
# ...

For direct configuration, replace the function calls with code that gets values from the console or some other configuration store, such as Azure KeyVault. In this case, the host is the Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient(
  host                = get_workspace_url(),
  azure_tenant_id     = get_azure_tenant_id(),
  azure_client_id     = get_azure_client_id(),
  azure_client_secret = get_azure_client_secret()
)
# ...

Databricks recommends using host and explicitly assigning the Microsoft Entra service principal to the workspace. Alternatively, use azure_workspace_resource_id with the Azure resource ID. This approach requires Contributor or Owner permissions on the Azure resource, or a custom role with specific Azure Databricks permissions.

For more information about authenticating with Databricks tools and SDKs that use Python and that implement Databricks unified authentication, see:

• Set up the Databricks Connect client for Python
• Set up authorization for the Databricks extension for Visual Studio Code
• Authenticate the Databricks SDK for Python with your Azure Databricks account or workspace

Databricks SDK for Java

For account-level operations, for default authentication:

import com.databricks.sdk.AccountClient;
// ...
AccountClient a = new AccountClient();
// ...

For direct configuration, replace the function calls with code that gets values from the console or some other configuration store, such as Azure KeyVault. In this case, the Azure Databricks account console URL is https://accounts.azuredatabricks.net:

import com.databricks.sdk.AccountClient;
import com.databricks.sdk.core.DatabricksConfig;
// ...
DatabricksConfig cfg = new DatabricksConfig()
  .setHost(getAccountConsoleUrl())
  .setAccountId(getAccountId())
  .setAzureTenantId(getAzureTenantId())
  .setAzureClientId(getAzureClientId())
  .setAzureClientSecret(getAzureClientSecret())
AccountClient a = new AccountClient(cfg);
// ...

For workspace-level operations, for default authentication:

import com.databricks.sdk.WorkspaceClient;
// ...
WorkspaceClient w = new WorkspaceClient();
// ...

For direct configuration, replace the function calls with code that gets values from the console or some other configuration store, such as Azure KeyVault. In this case, the host is the Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net:

import com.databricks.sdk.WorkspaceClient;
import com.databricks.sdk.core.DatabricksConfig;
// ...
DatabricksConfig cfg = new DatabricksConfig()
  .setHost(getWorkspaceUrl())
  .setAzureTenantId(getAzureTenantId())
  .setAzureClientId(getAzureClientId())
  .setAzureClientSecret(getAzureClientSecret())
WorkspaceClient w = new WorkspaceClient(cfg);
// ...

Databricks recommends using setHost and explicitly assigning the Microsoft Entra service principal to the workspace. Alternatively, use setAzureWorkspaceResourceId with the Azure resource ID. This approach requires Contributor or Owner permissions on the Azure resource, or a custom role with specific Azure Databricks permissions.

For more information about authenticating with Databricks tools and SDKs that use Java and that implement Databricks unified authentication, see:

• Set up the Databricks Connect client for Scala (the Databricks Connect client for Scala uses the included Databricks SDK for Java for authentication)
• Authenticate the Databricks SDK for Java with your Azure Databricks account or workspace

Databricks SDK for Go

For account-level operations, for default authentication:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
a := databricks.Must(databricks.NewAccountClient())
// ...

For direct configuration, replace the function calls with code that gets values from the console or some other configuration store, such as Azure KeyVault. In this case, the Azure Databricks account console URL is https://accounts.azuredatabricks.net:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
a := databricks.Must(databricks.NewAccountClient(&databricks.Config{
  Host:              getAccountConsoleUrl(),
  AccountId:         getAccountId(),
  AzureTenantId:     getAzureTenantId(),
  AzureClientId:     getAzureClientId(),
  AzureClientSecret: getAzureClientSecret(),
}))
// ...

For workspace-level operations, for default authentication:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
w := databricks.Must(databricks.NewWorkspaceClient())
// ...

For direct configuration, replace the function calls with code that gets values from the console or some other configuration store, such as Azure KeyVault. In this case, the host is the Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
w := databricks.Must(databricks.NewWorkspaceClient(&databricks.Config{
  Host:              getWorkspaceUrl(),
  AzureTenantId:     getAzureTenantId(),
  AzureClientId:     getAzureClientId(),
  AzureClientSecret: getAzureClientSecret(),
}))
// ...

Databricks recommends using Host and explicitly assigning the Microsoft Entra service principal to the workspace. Alternatively, use AzureWorkspaceResourceId with the Azure resource ID. This approach requires Contributor or Owner permissions on the Azure resource, or a custom role with specific Azure Databricks permissions.

For more information about authenticating with Databricks tools and SDKs that use Go and that implement Databricks client unified authentication, see Authenticate the Databricks SDK for Go with your Azure Databricks account or workspace.

Additional resources

• Databricks unified authentication
• Authorize service principal access to Azure Databricks with OAuth
• Service principals
• Provision a service principal by using Terraform