Share via

Service principals

A service principal is a specialized identity in Azure Databricks designed for automation and programmatic access. Service principals provide secure, API-only access to Azure Databricks resources for automated tools, scripts, and CI/CD platforms, without relying on individual user credentials.

For how to manage service principals, see Manage service principals.

Note

This page assumes your workspace has identity federation enabled, which is the default for most workspaces. For information about legacy workspaces without identity federation, see Legacy workspaces without identity federation.

What is a service principal?

Service principals give automated tools and scripts API-only access to Azure Databricks resources, providing greater security than using user accounts. You can grant and restrict a service principal's access to resources in the same way as you can an Azure Databricks user. For example, you can grant a service principal the account admin or workspace admin role, grant access to data using Unity Catalog, or add a service principal as a member to a group.

You can grant Azure Databricks users, service principals, and groups permissions to use a service principal. This allows users to run jobs as the service principal, instead of as their identity, which prevents jobs from failing if a user leaves your organization or a group is modified.

Databricks and Microsoft Entra ID service principals

Service principals can either be Azure Databricks managed service principals or Microsoft Entra ID managed service principals.

Azure Databricks managed service principals can authenticate to Azure Databricks using Databricks OAuth authentication and personal access tokens. Microsoft Entra ID managed service principals can authenticate to Azure Databricks using Databricks OAuth authentication and Microsoft Entra ID tokens. For more information on authentication for service principals, see Manage tokens for a service principal.

Azure Databricks managed service principals are managed directly within Azure Databricks. Microsoft Entra ID managed service principals are managed in Microsoft Entra ID, which requires additional permissions. Databricks recommends that you use Azure Databricks managed service principals for Azure Databricks automation and that you use Microsoft Entra ID managed service principals in cases where you must authenticate with Azure Databricks and other Azure resources at the same time.

To create a Azure Databricks managed service principal, skip this section and continue reading with Who can manage and use service principals?.

To use Microsoft Entra ID managed service principals in Azure Databricks, an admin user must create a Microsoft Entra ID application in Azure. To create a Microsoft Entra ID managed service principal, see Authenticate with Microsoft Entra service principals.

Common use cases

Service principals are ideal for automation scenarios like the following, where you need secure, reliable programmatic access to Databricks resources:

Use case Example
CI/CD pipelines Deploy notebooks, libraries, and configurations automatically as part of your continuous integration and deployment workflows.
Scheduled jobs Run ETL pipelines, data processing jobs, and automated reports on a schedule without depending on individual user accounts.
Cross-system integrations Connect external applications and services to Databricks for data ingestion, transformation, or analytics.
Automated testing Execute integration tests and validate data pipelines as part of your testing framework.
Infrastructure as code Provision and manage Databricks resources using tools like Terraform, ARM templates, or Databricks Asset Bundles.

Who can manage and use service principals?

To manage service principals in Azure Databricks, you must have one of the following roles:

Role Capabilities
Account admins
  • Add service principals to the account and assign them admin roles.
  • Assign service principals to workspaces.
  • Automatically have service principal manager role on all service principals in the account.
Workspace admins
  • Add service principals to an Azure Databricks workspace.
  • Assign them the workspace admin role.
  • Manage access to objects and functionality in the workspace.
  • Automatically have service principal manager role on service principals they create.
Service principal managers
  • Manage roles on a service principal.
  • Assign the service principal manager role to other users.
Service principal users
  • Run jobs as the service principal, using its identity instead of the job owner's identity.

Note

  • The creator of a service principal automatically becomes the service principal manager.
  • Users with the service principal manager role do not inherit the service principal user role. If you want to use the service principal to execute jobs, you need to explicitly assign yourself the service principal user role, even after creating the service principal.
  • When the RestrictWorkspaceAdmins setting is set to ALLOW ALL, workspace admins can create tokens on behalf of any service principal in their workspace. See Restrict workspace admins.

For information on how to grant the service principal manager and user roles, see Roles for managing service principals.

Sync service principals to your Azure Databricks account from your Microsoft Entra ID tenant

You can sync Microsoft Entra ID service principals automatically from your Microsoft Entra ID tenant to your Azure Databricks account using automatic identity management. Databricks uses Microsoft Entra ID as the source, so any changes to users or group memberships are respected in Azure Databricks. Automatic identity management is enabled by default for accounts created after August 1, 2025. See Sync users and groups automatically from Microsoft Entra ID.

SCIM provisioning does not support syncing service principals.

Additional resources

  • Last updated on