Copilot Studio Kit overview

Copilot Studio Kit is a toolkit designed to enhance your experience with Copilot Studio. The Power Customer Advisory Team (Power CAT) created and maintain the kit to equip makers with tools to build, test, and optimize custom agents. Whether you're refining prompts, analyzing performance, or customizing webchat experiences, Copilot Studio Kit helps you create intelligent and high-performing custom agents.

Test capabilities

Copilot Studio Kit allows makers to configure agents, tests, and test sets, and use them to batch test their custom agents. Test runs produce detailed results, including latencies, observed responses, and run-level aggregates. The test types include response match, attachment match, topic match, and generative answers.

Learn more: Enhance agent testing

Conversation KPIs

Conversation KPIs (key performance indicators) help makers track and analyze custom agent performance. Conversation KPIs complement Copilot Studio's built-in analytics by providing aggregated data in Dataverse. This data makes it easier to understand conversation outcomes.

Learn more: Track agent performance with Conversation KPIs

SharePoint synchronization

SharePoint synchronization lets makers periodically sync selected content from SharePoint locations to custom agent knowledge bases as files.

Learn more: Improve agent responses with SharePoint synchronization

Prompt Advisor

Prompt Advisor helps makers create well-structured prompts to improve user input. Makers enter a prompt to get a confidence score, a detailed analysis, and suggested optimized prompts that they can refine iteratively.

Learn more: Refine prompts with Prompt Advisor

Webchat Playground

Webchat Playground helps makers customize the appearance and behavior of the copilot agent webchat, including the colors, fonts, thumbnails, and more.

Learn more: Customize webchat appearance with Webchat Playground

Adaptive Cards Gallery

Adaptive Cards Gallery offers makers Adaptive Card templates for various scenarios. The templates demonstrate the extensibility of Adaptive Card visuals and behavior, and include examples of agent-side implementation and dynamic data binding.

Learn more: Enhance user engagement with Adaptive Cards Gallery

Agent Inventory

Agent Inventory provides administrators with a tenant-wide view of all custom agents in their organization. It shows details about the features they're using, authentication mode, knowledge sources, orchestration type, and more. Agent Inventory includes a dashboard, and you can export the data for use in other applications.

Learn more: Monitor agents using Agent Inventory

Agent Review Tool

The Agent Review Tool is a solution analysis tool that checks your agents for any potential issues or anti-patterns that might negatively affect performance or security. After analysis, the Agent Review Tool presents the findings in an easy-to-interpret format, with severity and details on how to address each issue.

Learn more: Analyze agents using the Agent Review Tool

Conversation Analyzer

Conversation Analyzer helps makers analyze conversation transcripts of their custom agents using custom prompts to obtain actionable insights.

Learn more: Analyze conversations transcripts

Agent Value

The Agent Value component is a modular tool for classifying Copilot Studio agents by type, behavior, and value. It helps organizations understand each agent's role, align agents with strategic goals, and measure the value each agent delivers. Results are presented visually on the Agent Value dashboard.

Learn more: Measure agent value

Automated testing using pipelines in Power Platform

This feature allows users to automate the testing and deployment of Copilot Studio custom agents using Power Platform Pipelines. The goal is to ensure that agents are automatically validated through test runs before they're deployed to target environments (production). By integrating Power Automate flows with Dataverse and the Copilot Studio Kit, this approach introduces a quality gate into the deployment process. This method supports continuous delivery practices and enhances the overall governance of the deployment lifecycle.

Learn more: Automate testing and deployment of agents using pipelines in Power Platform

Next step