Edit

Share via

Add an Azure Data Explorer database source to an eventstream (preview)

This article shows you how to add an Azure Data Explorer database (DB) source to a Microsoft Fabric eventstream. 

Azure Data Explorer is a fully managed, high-performance platform that delivers real-time insights from massive streaming data. You can use Microsoft Fabric eventstreams to connect to an Azure Data Explorer database, stream the data from its tables, and route them to various destinations within Fabric.

Prerequisites

  • Access to the Fabric premium workspace with Contributor or higher permissions.
  • An active Azure subscription.
  • An Azure Data Explorer cluster with at least one database deployed. Ensure that the cluster is publicly accessible and not restricted by a firewall or a virtual network.
  • The required permissions to access the Azure Data Explorer cluster.

Add an Azure Data Explorer DB as a source

If you haven't added any source to your eventstream yet, select the Use external source tile.

Screenshot that shows the selection of the tile for using an external source.

If you're adding the source to an already published eventstream, switch to Edit mode. On the ribbon, select Add source > External sources.

Screenshot that shows selections for adding external sources.

On the Select a data source page, search for Azure Data Explorer DB. On the Azure Data Explorer DB tile, select Connect.

Screenshot that shows the selection of Azure Data Explorer as the source type in the wizard for getting events.

Configure and connect to an Azure Data Explorer DB

  1. On the Configure connection settings page, select New connection.

    Screenshot that shows the page for configuration connection settings.

  2. In the Connection settings section of the popup window, follow these steps:

    1. For Cluster, enter the URI of your Azure Data Explorer cluster.
    2. The Database and Table name boxes in the cloud connection are optional. Values entered in these boxes are ignored in this step. You can specify them in the next step of the wizard.
    3. For Connection name, enter a name for the connection to the Azure Data Explorer cluster. 
    4. For Authentication kind, only Organizational account is currently supported.
    5. Select Connect.

    Screenshot that shows connection settings and credentials.

  3. On the Configure connection settings page, follow these steps if you didn't specify the database and tables in the connection settings earlier:

    1. In the Database box, enter the name of your database.

    2. In the Enter table name(s) box, enter either a single table name or multiple table names separated by commas.

      Screenshot that shows completed connection settings.

  4. If you're using a real-time hub, follow these steps. Otherwise, move to the next step.

    1. In the Source details section, select the Fabric workspace where you want to save the eventstream.

    2. For Eventstream name, enter a name for the eventstream. The wizard creates an eventstream with the selected Azure Database Explorer table as a source.

    3. The Stream name value is automatically generated for you by appending -stream to the name of the eventstream. You can see this stream on the real-time hub's All data streams page when the wizard finishes.

      Screenshot that shows the section for source details in the Azure Data Explorer connection settings.

  5. If you're using the eventstream editor to add an Azure Data Explorer database table as a source to an eventstream, select the pencil button under Source name, and then enter a source name.

    Screenshot that shows the pencil button for adding a source name.

  6. Select Next at the bottom of the page.

  7. On the Review + connect page, review the settings, and then select Connect (Fabric real-time hub) or Add (Fabric eventstream).

    Screenshot that shows the page for reviewing details and connecting an Azure Data Explorer connector.

View an updated eventstream

  1. Confirm that the Azure Data Explorer DB source is added to your eventstream on the canvas in the Edit mode. To implement this newly added Azure Data Explorer DB source, select Publish.

    Screenshot that shows the editor with the Publish button selected.

  2. Your Azure Data Explorer DB source is available for visualization in the Live view. 

    Screenshot that shows the editor in the live view.

    Note

    When the Azure Data Explorer DB streaming connector starts, it captures only new table changes. Historical table data isn't streamed into the eventstream.

Limitations

The Azure Data Explorer streaming connector captures table changes by using a database cursor to query the differences between two ingestion times. However, each query result is limited to a maximum of 64 MB or 500,000 records. For details, see the Azure Data Explorer query limits. If your Azure Data Explorer database has a high ingestion rate that exceeds this limit, the connector might fail.

Related content

  • Last updated on