Enable external threat detection and protection for Copilot Studio custom agents (preview)

[This article is prerelease documentation and is subject to change.]

Custom agents created in Copilot Studio are secure by default. They include built-in protection against various threats, such as user prompt-injection attacks (UPIA) and cross-domain prompt injection Attacks (XPIA). At runtime, the agent blocks attacks of these types, reducing the risk of data exfiltration.

To further increase the monitoring capabilities and security of custom agents, Copilot Studio lets organizations configure external threat detection systems for enhanced oversight. These tools operate during the agent's runtime, continuously evaluating agent activity. If the system detects any suspicious tools or actions, it can intervene to block them from executing. This threat detection provides an extra layer of real-time protection and compliance enforcement.

How it works

An external threat detection system is set up as a web service, exposing a REST API with a threat detection endpoint. A secure connection is configured between the agent and the endpoint. At runtime, every time the orchestrator considers invoking a tool, it sends relevant data about the proposed tool use to the threat detection endpoint for evaluation. The threat detection system analyzes the data and returns a decision to either allow or block the tool invocation.

If the threat detection system detects a security issue during an agent's operation, the agent immediately stops processing, and notifies the user that their message is blocked. On the other hand, if the system approves the operation, the agent proceeds seamlessly, with no visible effect or interruption for the user.

Options for setting up external threat detection

Copilot Studio supports a flexible "bring your own protection" approach. Organizations have the freedom to integrate security solutions that best fit their unique requirements.

Options include:

• Develop your own custom monitoring tools, or have someone develop them for you. For more information on how to set up the system endpoint so that your agent can connect to it, see Build a runtime threat detection system for Copilot Studio agents
• Apply a robust enterprise solution by Microsoft Defender
• Use products from other trusted security providers

What data is shared with the threat detection provider?

Once you configure a connection to a threat detection system, the agent shares data with the external security provider during its run. The agent communicates with the service whenever it considers invoking a tool. This data sharing ensures efficient decision-making by the configured system, without degrading the experience of your agent's users.

The high-level data shared with the system includes:

• The user's recent prompt and the latest history of chat messages exchanged between the agent and the user.
• Outputs of previous tools used by the agent.
• Conversation metadata: Identity of the agent, the user who interacts with it, the user's tenant, and the trigger that triggered it (where applicable).
• The tool the agent wants to invoke, including agent-generated reasoning of why this tool was selected and the proposed inputs and values.

Prerequisites

Before you begin, you need:

• An external threat detection service set up to evaluate agent tool use requests. The service needs to expose a REST API endpoint. For the setup on the Copilot Studio side of the integration, you need the base URL for the security provider web service. This article refers to this URL as the endpoint. The agent sends requests for threat detection to APIs at this base URL. You should receive this URL from your security provider.
• A Microsoft Entra tenant where you can register an application for authentication between the agent and the threat detection service.
• A user with a Power Platform Administrator role to configure a connection between the agent and the external threat detection system for both the individual environment level, and the environment group level.

Configure an external threat detection system

The process to configure an external threat detection system for your agent has two steps:

1. Configure a Microsoft Entra application.
2. Configure threat detection in Power Platform admin center.

Step 1: Configure a Microsoft Entra application

There are two paths you can take to configure a Microsoft Entra application:

• Option A: Configure using PowerShell script (recommended)
• Option B: Configure manually using Azure portal

Option A: Configure using PowerShell (Recommended)

You can use a provided PowerShell script to automate the creation and configuration of your Microsoft Entra application.

Prerequisites for PowerShell Configuration

• Windows PowerShell 5.1 or later
• Sufficient permissions to create application registrations in your Microsoft Entra tenant
• The base URL of the threat detection web service. The URL is referred to as the Endpoint in the script parameters that follow. You should receive this URL from your security provider.
• Your organization's Microsoft Entra tenant ID. The tenant ID is referred to as the TenantId in the script parameters that follow.

Download and prepare the script

Download the Create-CopilotWebhookApp.ps1 script.

Script parameters

The script accepts the following parameters:

Execute the script

To create the application:

1. Open Windows PowerShell as an administrator.

2. Go to the directory containing the script.

3. Execute the following script, replacing the placeholder values for TenantId, Endpoint, DisplayName, and FICName with your own parameters:

   .\Create-CopilotWebhookApp.ps1 `
   -TenantId "11111111-2222-3333-4444-555555555555" `
   -Endpoint "https://provider.example.com/threat_detection/copilot" `
   -DisplayName "Copilot Security Integration - Production" `
   -FICName "ProductionFIC"
   

The interactive script runs in the command line. The script outputs the App ID of the created Microsoft Entra application. You need the App ID later when configuring threat detection in Power Platform admin center.

Option B: Configure manually using Azure portal

Prerequisites for manual configuration

• Your organization's Microsoft Entra tenant ID. The tenant ID is referred to as the tenantId in the instructions that follow.
• Sufficient permissions to create application registrations in your Microsoft Entra tenant.
• The base URL of the threat detection web service you're using. This is referred to as the endpoint in the following instructions. You should receive this URL from your security provider.

Register a Microsoft Entra application

Follow these steps to create a Microsoft Entra application registration. The application is used to secure authentication between the agent and the threat detection web service. See Register an application in Microsoft Entra ID to learn how to create such an app.

1. Sign in to Azure portal and navigate to the Microsoft Entra ID page.
2. Under App registrations, select New registration.
3. Provide a name and select Accounts in this organizational directory only (Single tenant) as the supported account type.
4. Register the app.
5. After the app is created, copy the App ID. You need the App ID later when configuring threat detection in Power Platform admin center.

Authorize the Microsoft Entra application with your provider of choice

The agent uses Federated Identity Credentials (FIC) as a secure, secret-less authentication method for exchanging data with the threat detection system provider. Follow these steps to configure FIC for your Microsoft Entra application. For more information, see Configure a user-assigned managed identity to trust an external identity provider.

1. Open the Azure portal and go to App registrations. Select the application you created in step 1 above.

2. In the sidebar, select Manage > Certificates & secrets > Federated credentials.

3. Select Add credential.

4. In the Federated credentials scenario drop-down, select Other issuer.

5. Fill the fields according to these instructions:

   • Issuer: Enter the following URL, replacing {tenantId} with your organization's Microsoft Entra tenant ID: https://login.microsoftonline.com/{tenantId}/v2.0

   • Type: Select Explicit subject identifier.

   • Value: Input a string structured as follows: /eid1/c/pub/t/{base 64 encoded tenantId}/a/m1WPnYRZpEaQKq1Cceg--g/{base 64 encoded endpoint}

     Perform base64 encoding for your organization's Microsoft Entra tenant ID and the base URL of the threat detection web service. Replace the placeholder {base 64 encoded tenantId} with the base64-encoded value of your tenant ID, and the placeholder {base 64 encoded endpoint} with the base64-encoded base URL.

     To get the base64 encoding of your tenant ID and endpoint URL, use the following PowerShell script. Make sure to replace the placeholder values "11111111-2222-3333-4444-555555555555" and "https://provider.example.com/threat_detection/copilot" with your actual values for tenant ID and endpoint URL:

     # Encoding tenant ID
     $tenantId = [Guid]::Parse("11111111-2222-3333-4444-555555555555")
     $base64EncodedTenantId = [Convert]::ToBase64String($tenantId.ToByteArray()).Replace('+','-').Replace('/','_').TrimEnd('=')
     Write-Output $base64EncodedTenantId
     
     # Encoding the endpoint
     $endpointURL = "https://provider.example.com/threat_detection/copilot"
     $base64EncodedEndpointURL = [Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes($endpointURL)).Replace('+','-').Replace('/','_').TrimEnd('=')
     Write-Output $base64EncodedEndpointURL
     

   • Name: Choose a descriptive name.

6. Select the Add button.

Step 2: Configure the threat detection system

Next, you need to configure the threat detection system in Power Platform admin center to connect your agent to the external security provider.

Prerequisites for configuring threat detection in Power Platform admin center

• The App ID of the Microsoft Entra application you created in the previous step.
• The endpoint link provided by your external monitoring system's provider. The endpoint link is the same base URL endpoint you use when configuring the Microsoft Entra application.
• A user with a Power Platform Administrator role to configure the connection.
• Perform any other steps required by your security provider to authorize your registered application. You should consult your provider's documentation (as applicable) for any specific onboarding and authorization steps.

To configure the threat detection system in Power Platform admin center, follow these steps:

1. Sign in to the Power Platform admin center.
2. On the side navigation, select Security and then select Threat detection. The Threat detection page opens.
3. Select Additional threat detection. A pane opens.
4. Select the environment for which you want to enhance agent protection and select Set up. A pane opens.
5. Select Allow Copilot Studio to share data with a threat detection provider.
6. Under Azure Entra App ID, enter the App ID of the Microsoft Entra application you created previously.
7. Enter the Endpoint link provided by your external monitoring system's provider. The endpoint link is the same base URL endpoint you use when configuring the Microsoft Entra application.
8. Under Set error behavior, define the system's default behavior for when the threat detection system doesn't respond in time or responds with an error. By default, this is set to Allow the agent to respond, but you can also chose the Block the query option to further reduce risk.
9. Select Save.

Troubleshooting

Here's some information on issues that might occur and how to handle them.

Power Platform admin center threat detection configuration issues

The following table describes common errors that might happen when you select Save in the previous step, and how to handle these errors:

For more error details, select Copy error info.

Common Microsoft Entra and authentication issues

Here are some other common issues that might occur with your Microsoft Entra app and authentication.

Microsoft Entra application doesn't exist

Example: Failed to acquire token: AADSTS700016: Application with identifier '55ed00f8-faac-4a22-9183-9b113bc53dd4' wasn't found in the directory 'Contoso'. This can happen if the application isn't installed by the administrator of the tenant or consented to by any user in the tenant. You might have sent your authentication request to the wrong tenant.

How to handle: Make sure the application ID provided is correct and exists in Azure.

No FIC configured on the app

Example: Failed to acquire token: A configuration issue is preventing authentication—check the error message from the server for details. You can modify the configuration in the application registration portal. See https://aka.ms/msal-net-invalid-client for details. Original exception: AADSTS70025: The client '57342d48-0227-47cd-863b-1f4376224c21'(Webhooks test) has no configured federated identity credentials.

How to handle: The app provided doesn’t have any FIC configured on it. Follow the documentation and configure FIC accordingly.

Invalid FIC Issuer

Example: Failed to acquire token: A configuration issue is preventing authentication—check the error message from the server for details. You can modify the configuration in the application registration portal. See https://aka.ms/msal-net-invalid-client for details. Original exception: AADSTS7002111: No matching federated identity record found for presented assertion issuer 'https://login.microsoftonline.com/262d6d26-0e00-40b3-9c2f-31501d4dcbd1/v2.0'. Make sure the federated identity credential Issuer is 'https://login.microsoftonline.com/{tenantId}/v2.0'.

How to handle: No FIC with the expected issuer was found on the app. Open your FIC configuration and set the issuer to https://login.microsoftonline.com/{tenantId}/v2.0 (filling in your tenant ID).

Invalid FIC Subject

Example: Failed to acquire token: A configuration issue is preventing authentication—check the error message from the server for details. You can modify the configuration in the application registration portal. See https://aka.ms/msal-net-invalid-client for details. Original exception: AADSTS7002137: No matching federated identity record found for presented assertion subject '/eid1/c/pub/t/Jm0tJgAOs0CcLzFQHU3L0Q/a/iDQPIrayM0GBBVzmyXgucw/aHR0cHM6Ly9jb250b3NvLnByb3ZpZGVyLmNvbeKAiw'. Make sure the federated identity credential Subject is '/eid1/c/pub/t/{tenantId}/a/iDQPIrayM0GBBVzmyXgucw/aHR0cHM6Ly9jb250b3NvLnByb3ZpZGVyLmNvbeKAiw'.

How to handle: No FIC with the expected subject is found on the app. Open your FIC configuration and set the subject to the expected value as suggested in the error (fill in your tenant ID). Make sure there are no extra whitespaces or blank lines in the subject fields.

App isn't allowlisted with provider (Microsoft Defender specific)

Example: The application ID in your authentication token doesn't match the registered application for webhook access. Ensure you're using the correct application credentials.

How to handle: Application isn't allowlisted with the provider. Refer to the provider documentation to grant the app webhook access.

Disconnect the protection by the threat detection system

If you no longer want the threat detection system to monitor your agent, follow these steps:

1. Sign in to the Power Platform admin center.
2. On the side navigation, select Security and then select Threat detection. The Threat detection page opens.
3. Select Additional threat detection. A pane opens.
4. Select the environment for which you want to turn off enhanced agent protection and select Set up. A pane opens.
5. Unselect Allow Copilot Studio to share data with your selected provider.
6. Select Save.