Edit

Share via


Configure change event streaming (preview)

Applies to: SQL Server 2025 (17.x) Azure SQL Database

This article describes how to configure the change event streaming (CES) feature introduced in SQL Server 2025 (17.x) and Azure SQL Database.

Note

Change event streaming is currently in preview for:

During preview, this feature is subject to change. For current supportability, see Limitations.

Overview

To configure and use change event streaming, follow these steps:

  1. Use an existing or create a new Azure Event Hubs namespace and Event Hubs instance. The Event Hubs instance receives events.
  2. Enable change event streaming for a user database.
  3. Create an event stream group. With this group, configure the destination, credentials, message size limits, and partitioning schema.
  4. Add one or more tables to the event stream group.

Each step is described in detail in the following sections of this article.

Prerequisites

To configure change event streaming, you need the following resources, permissions, and configuration:

Configure Azure Event Hubs

To learn how to create Azure Event Hubs, review Create an event hub using the Azure portal.

Azure Event Hubs access control

Configure access control for your SQL resource to Azure Event Hubs. Microsoft Entra authentication is the most secure method but is currently only supported by Azure SQL Database for CES. While using a shared access policy is supported by both Azure SQL Database and SQL Server 2025, use it only in Azure SQL Database if Microsoft Entra authentication isn't an option.

Shared access policies provide authentication and authorization to Azure Event Hubs. Each shared access policy needs a name, an access level (Manage, Send, or Listen), and a resource binding (Event Hubs namespace or a specific Event Hub instance). Instance level policies offer more security by following the principle of least privilege. Both SQL Server 2025 and Azure SQL Database support this method. However, use Microsoft Entra authentication whenever possible with Azure SQL Database, as it provides better security.

If you use a shared access policy for authentication and authorization, clients sending data to an Azure Event Hub need to provide the name of the policy they want to use, along with either a SAS token generated from the policy or the policy's Service key.

SAS tokens have a security advantage over service keys: If the client is compromised, the SAS token is only valid until it expires, and the compromised client can't create new SAS tokens. In contrast, service keys don't automatically expire. A compromised client with a service key can generate new SAS tokens by using the key.

To configure streaming to Azure Event Hubs with the AMQP protocol (the default native Azure Event Hubs protocol), create or reuse a shared access policy with Send permission and generate a SAS token. You can generate the token programmatically with any programming or scripting language. The example in this article shows how to generate a SAS token from a new or existing policy by using a PowerShell script.

Note

For improved security, using Microsoft Entra based access control whenever possible is strongly recommended. If Microsoft Entra based access control isn't possible and you're using shared access policies, use SAS token authentication instead of service key-based authentication whenever possible. Best practices for SAS tokens include defining an appropriate minimally required access scope, setting a short expiration date, and rotating the SAS key regularly. For key-based authentication, rotate keys periodically. Store all secrets securely by using Azure Key Vault or a similar service.

Install required modules

To manage Azure Event Hubs resources with PowerShell scripts, you need the following modules:

  • Az PowerShell module
  • Az.EventHub PowerShell module

The following script installs the required modules:

Install-Module -Name Az -AllowClobber -Scope CurrentUser -Repository PSGallery -Force
Install-Module -Name Az.EventHub -Scope CurrentUser -Force

If you already have the required modules and want to update them to the latest version, run the following script:

Update-Module -Name Az -Force
Update-Module -Name Az.EventHub -Force

Connect to Azure

You can either use Azure Cloud Shell or sign in and set your subscription context.

To run with Azure Cloud Shell, review Sign in to Azure.

Define a policy

To create the SAS token, you need a policy with Send rights. You can either:

  • Create a new policy

    Or

  • Use an existing policy

Note

The policy must have Send rights.

Create SAS token for a new or existing policy

When you create a new policy, make sure it has the Send right. If you use an existing policy, check that it has the Send right.

The following script creates a new policy or gets an existing one, then generates a full SAS token in an HTTP authorization header format.

Replace values in angle brackets (<value>) with values for your environment.

function Generate-SasToken {
$subscriptionId = "<Azure-Subscription-ID>"
$resourceGroupName = "<Resource-group-name>"
$namespaceName = "<Azure-Event-Hub-Namespace-name>"
$eventHubName = "<Azure-Event-Hubs-instance-name>"
$policyName = "<Policy-name>"
$tokenExpireInDays = "<number-of-days-token-will-be-valid>"

# Modifying the rest of the script is not necessary.

# Login to Azure and set Azure Subscription.
Connect-AzAccount

# Get current context and check subscription
$currentContext = Get-AzContext
if ($currentContext.Subscription.Id -ne $subscriptionId) {
    Write-Host "Current subscription is $($currentContext.Subscription.Id), switching to $subscriptionId..."
    Set-AzContext -SubscriptionId $subscriptionId | Out-Null
} else {
    Write-Host "Already using subscription $subscriptionId."
}

# Try to get the authorization policy (it should have Send rights)
$rights = @("Send")
$policy = Get-AzEventHubAuthorizationRule -ResourceGroupName $resourceGroupName -NamespaceName $namespaceName -EventHubName $eventHubName -AuthorizationRuleName $policyName -ErrorAction SilentlyContinue

# If the policy does not exist, create it
if (-not $policy) {
    Write-Output "Policy '$policyName' does not exist. Creating it now..."

    # Create a new policy with the Manage, Send and Listen rights
    $policy = New-AzEventHubAuthorizationRule -ResourceGroupName $resourceGroupName -NamespaceName $namespaceName -EventHubName $eventHubName -AuthorizationRuleName $policyName -Rights $rights
    if (-not $policy) {
        throw "Error. Policy was not created."
    }
    Write-Output "Policy '$policyName' created successfully."
} else {
    Write-Output "Policy '$policyName' already exists."
}

if ("Send" -in $policy.Rights) {
    Write-Host "Authorization rule has required right: Send."
} else {
    throw "Authorization rule is missing Send right."
}

$keys = Get-AzEventHubKey -ResourceGroupName $resourceGroupName -NamespaceName $namespaceName -EventHubName $eventHubName -AuthorizationRuleName $policyName

if (-not $keys) {
    throw "Could not obtain Azure Event Hub Key. Script failed and will end now."
}
if (-not $keys.PrimaryKey) {
    throw "Could not obtain Primary Key. Script failed and will end now."
}

# Get the Primary Key of the Shared Access Policy
$primaryKey = ($keys.PrimaryKey)
Write-Host $primaryKey

## Check that the primary key is not empty.

# Define a function to create a SAS token (similar to the C# code provided)
function Create-SasToken {
    param (
        [string]$resourceUri, [string]$keyName, [string]$key
    )

$sinceEpoch = [datetime]::UtcNow - [datetime]"1970-01-01"
    $expiry = [int]$sinceEpoch.TotalSeconds + ((60 * 60 * 24) * [int]$tokenExpireInDays) # seconds since Unix epoch
    $stringToSign = [System.Web.HttpUtility]::UrlEncode($resourceUri) + "`n" + $expiry
    $hmac = New-Object System.Security.Cryptography.HMACSHA256
    $hmac.Key = [Text.Encoding]::UTF8.GetBytes($key)
    $signature = [Convert]::ToBase64String($hmac.ComputeHash([Text.Encoding]::UTF8.GetBytes($stringToSign)))
    $sasToken = "SharedAccessSignature sr=$([System.Web.HttpUtility]::UrlEncode($resourceUri))&sig=$([System.Web.HttpUtility]::UrlEncode($signature))&se=$expiry&skn=$keyName"
    return $sasToken
}

# Construct the resource URI for the SAS token
$resourceUri = "https://$namespaceName.servicebus.windows.net/$eventHubName"

# Generate the SAS token using the primary key from the new policy
$sasToken = Create-SasToken -resourceUri $resourceUri -keyName $policyName -key $primaryKey

# Output the SAS token
Write-Output @"
-- Generated SAS Token --
$sasToken
-- End of generated SAS Token --
"@
}

Generate-SasToken

The output of the previous command should look like the following text:

-- Generated SAS Token --
SharedAccessSignature sr=https%3a%2f%YourEventHubNamespace.servicebus.windows.net%2fYourEventHub&sig=xxxxxxxxxxxxxxxxxxxxxxx&se=2059133074&skn=SharedKeyNameIsHERE
-- End of generated SAS Token --

Copy the entire SAS token value (the line that starts with SharedAccessSignature) to use it later when you configure CES, such as the following example:

SharedAccessSignature sr=https%3a%2f%YourEventHubNamespace.servicebus.windows.net%2fYourEventHub&sig=xxxxxxxxxxxxxxxxxxxxxxx&se=2059133074&skn=SharedKeyNameIsHERE

Enable and configure change event streaming

To enable and configure change event streaming, change the database context to the user database and then follow these steps:

  1. If it's not already configured, set the database to the full recovery model.
  2. Create a master key and a database scoped credential.
  3. Enable event streaming.
  4. Create the event stream group.
  5. Add one or more tables to the event stream group.

The examples in this section demonstrate how to enable CES for the AMQP protocol and the Apache Kafka protocol:

The following are sample parameter values for the examples in this section:

  • @stream_group_name = N'myStreamGroup'

  • @destination_location = N'myEventHubsNamespace.servicebus.windows.net/myEventHubsInstance' - this value is the FQDN of the specific Azure Event Hubs and instance name.

  • @partition_key_scheme = N'None' - (default) partitions are chosen round robin. Other possible options are:

    • StreamGroup - partitioning by the stream group
    • Table - partitioning by table
    • Column - partitioning by columns
  • [optional, if Shared Access policies via Service Key are used]

    • Primary or secondary key value taken from the Shared access policy: Secret = 'BVFnT3baC/K6I8xNZzio4AeoFt6nHeK0i+ZErNGsxiw='
  • EXEC sys.sp_add_object_to_event_stream_group N'myStreamGroup', N'dbo.myTable'

  • @max_message_size_kb = 256 - 256 KB is the default maximum message size, but this value should be aligned with the limits of the destination Azure Event Hubs.

Example: Stream to Azure Event Hubs via AMQP protocol

The examples in this section show how to stream change events to Azure Event Hubs by using the AMQP protocol. AMQP is the default native Azure Event Hubs protocol.

The example in this section uses a SAS token to authenticate to your Azure Event Hubs instance through the AMQP protocol. If Microsoft Entra authentication isn't available, use a SAS token instead of a service key value for improved security.

Replace values in angle brackets (<value>) with values for your environment.

USE <database name>;

-- Create the Master Key with a password.
CREATE MASTER KEY ENCRYPTION BY PASSWORD = '<Password>';

CREATE DATABASE SCOPED CREDENTIAL <CredentialName>
    WITH IDENTITY = 'SHARED ACCESS SIGNATURE',
    SECRET = '<Generated SAS Token>' -- Be sure to copy the entire token. The SAS token starts with "SharedAccessSignature sr="

EXEC sys.sp_enable_event_stream

EXEC sys.sp_create_event_stream_group
    @stream_group_name =      N'<EventStreamGroupName>',
    @destination_type =       N'AzureEventHubsAmqp',
    @destination_location =   N'<AzureEventHubsHostName>/<EventHubsInstance>',
    @destination_credential = <CredentialName>,
    @max_message_size_kb =    <MaxMessageSize>,
    @partition_key_scheme =   N'<PartitionKeyScheme>'

EXEC sys.sp_add_object_to_event_stream_group
    N'<EventStreamGroupName>',
    N'<SchemaName>.<TableName>'

Example: Stream to Azure Event Hubs via Apache Kafka protocol

The examples in this section show how to stream change events to Azure Event Hubs by using the Apache Kafka protocol.

The example in this section uses a SAS token to authenticate to your Azure Event Hubs instance through the Apache Kafka protocol. If Microsoft Entra authentication isn't available, use a SAS token instead of a service key value for improved security.

Replace values in angle brackets (<value>) with values for your environment.

USE <database name>

-- Create the Master Key with a password.
CREATE MASTER KEY ENCRYPTION BY PASSWORD = '<Password>'

CREATE DATABASE SCOPED CREDENTIAL <CredentialName>
    WITH IDENTITY = 'SHARED ACCESS SIGNATURE',
    SECRET = '<Event Hubs Namespace – Primary or Secondary connection string>'

EXEC sys.sp_enable_event_stream

EXEC sys.sp_create_event_stream_group
    @stream_group_name =      N'<EventStreamGroupName>',
    @destination_type =       N'AzureEventHubsApacheKafka',
    @destination_location =   N'<AzureEventHubsHostName>:<port>/<EventHubsInstance>', -- myEventHubsNamespace.servicebus.windows.net:9093/myEventHubsInstance
    @destination_credential = <CredentialName>,
    @max_message_size_kb =    <MaxMessageSize>,
    @partition_key_scheme =   N'<PatitionKeyScheme>'

EXEC sys.sp_add_object_to_event_stream_group
    N'<EventStreamGroupName>',
    N'<SchemaName>.<TableName>'

View CES configuration and function

In sys.databases, is_event_stream_enabled = 1 indicates that change event streaming is enabled for the database.

The following query returns all databases with change event streaming enabled:

SELECT *
FROM sys.databases
WHERE is_event_stream_enabled = 1;

In sys.tables, is_replicated = 1 indicates a table is streamed, and sp_help_change_feed_table provides information about the table group and table metadata for change event streaming.

The following query returns all tables with change event streaming enabled, and provides metadata information:

SELECT name,
       is_replicated
FROM sys.tables;

EXECUTE sp_help_change_feed_table
    @source_schema = '<schema name>',
    @source_name = '<table name>';

CES stored procedures, system functions, and DMVs

The following table lists the stored procedures, system functions, and DMVs that you can use to configure, disable, and monitor change event streaming:

System object Description
Configure CES
sys.sp_enable_event_stream Enables CES for the current user database.
sys.sp_create_event_stream_group Creates a stream group, which is a streaming configuration for a group of tables. The stream group also defines the destination and related details (such as authentication, message size, partitioning). The stream_group_id is automatically generated and displayed for the end user when the procedure completes.
sys.sp_add_object_to_event_stream_group Adds a table to the stream group.
Disable CES
sys.sp_remove_object_from_event_stream_group Removes a table from the stream group.
sys.sp_drop_event_stream_group Drops the stream group. The stream group must not be in use.
sys.sp_disable_event_stream Disables CES for the current user database.
Monitor CES
sys.dm_change_feed_errors Returns delivery errors.
sys.dm_change_feed_log_scan_sessions Returns information about log scan activity.
sys.sp_help_change_feed_settings Provides the status and information of configured change event streaming.
sys.sp_help_change_feed Monitors the current configuration of the change stream.
sys.sp_help_change_feed_table_groups Returns metadata that is used to configure change event streaming groups.
sys.sp_help_change_feed_table Provides the status and information of the streaming group and table metadata for change event streaming.

Limitations

Change event streaming (CES) has the following limitations:

Azure SQL Database

The following limitations apply when using CES with Azure SQL Database:

  • Extended Event (xEvent) debugging in Azure SQL Database isn't currently available.

Server-level and general limitations

  • CES isn't supported on SQL Server 2025 (17.x) on Linux.
  • CES emits events only for data changes from INSERT, UPDATE, and DELETE DML statements.
  • CES doesn't handle schema changes (DDL operations), which means it doesn't emit events for DDL operations. However, DDL operations aren't blocked, so if you execute them, the schema of subsequent DML events reflects the updated table structure. You're expected to gracefully handle events with the updated schema.
  • Currently, CES doesn't stream data that exists in a table before CES is enabled. Existing data isn't seeded, or sent as a snapshot, when CES is enabled.
  • When JSON is the specified output format, large event messages might be split at approximately 25% of the configured maximum message size per stream group. This limitation doesn't apply to the binary output type.
  • If a message exceeds the Azure Event Hubs message size limit, the failure is currently only observable through Extended Events. CES xEvents are currently only available in SQL Server 2025, and not Azure SQL Database.
  • Renaming tables and columns configuired for CES isn't supported. Renaming a table or column fails. Database renames are allowed.
  • Microsoft Entra authentication for CES isn't currently available in SQL Server 2025.

Database-level limitations

Table-level limitations

  • A table can belong to only one streaming group. You can't stream the same table to multiple destinations.

  • You can only configure user tables for CES. CES doesn't support streaming system tables.

  • You can configure up to 4,096 stream groups. Each stream group can include up to 40,000 tables.

  • Online index operations are not supported

  • While CES is enabled on a table, you can't add or drop a primary key constraint on that table.

  • ALTER TABLE SWITCH PARTITION isn't supported on tables configured for CES.

  • TRUNCATE TABLE isn't supported on tables enabled for CES.

  • CES doesn't support tables that use any of the following features:

    • Clustered columnstore indexes
    • Temporal history tables or ledger history tables
    • Always Encrypted
    • In-memory OLTP (memory-optimized tables)
    • Graph tables
    • External tables

Column-level limitations

  • CES doesn't support the following data types. Streaming skips columns of these types:
    • geography
    • geometry
    • image
    • json
    • rowversion / timestamp
    • sql_variant
    • text / ntext
    • vector
    • xml
    • User-defined types (UDT)

Permissions in the source database

  • For row-level security, CES emits changes from all rows, regardless of user permissions.
  • Dynamic data masking doesn't apply to data sent through CES. Data is streamed unmasked, even if masking is configured.
  • CES doesn't emit events related to object-level permission changes (for example, granting permissions to specific columns).