Understand Copilot for Microsoft Fabric Data Factory

Copilot for Data Factory in Microsoft Fabric is an AI-based assistant that supports common data integration workflows. It enables you to create, transform, and manage dataflows and pipelines using natural language inputs.

Copilot for Dataflow Gen2

Copilot for Dataflow Gen2 assists with several tasks that often challenge users. For example, many people struggle with writing complex queries or modifying existing ones to fit new requirements. With Query generation, you can describe what you need in natural language, and Copilot will create or update queries for you. If you need sample data or want to reference existing queries, Copilot helps speed up data preparation.

When it comes to Data transformation, defining the right steps to filter, aggregate, or reshape data can be confusing, especially for those new to Mashup code. Copilot allows you to specify your transformation needs in plain language, then automatically generates the required Mashup code. This removes the guesswork and helps you achieve accurate results without deep coding expertise.

Understanding complex queries is another common pain point. With Code explanation, Copilot breaks down the generated Mashup code, explaining the logic and purpose behind each transformation step. This feature helps you build confidence in your dataflows, learn best practices, and work smarter by focusing on your data goals rather than technical details.

By using these Copilot features, you can address common obstacles in data preparation and transformation when working with Dataflow Gen2.

Here's an example of how you can interact with Copilot in the Dataflow Gen2 editor:

We’ll explore this topic in more detail in a later module.

Copilot for Data Pipelines

Copilot for Data Pipelines addresses common challenges in pipeline creation and management. Many users find it difficult to design complex pipelines from scratch, especially when they are unsure which activities to include or how to configure them. With Pipeline generation, you can describe your desired workflow in natural language—for example, "Copy data from Azure SQL to a Lakehouse and send a notification when complete"—and Copilot will generate the necessary Data Pipeline activities. This can reduce initial setup effort.

Troubleshooting errors in pipelines is another area where users often struggle, particularly when error messages are unclear or solutions are not obvious. The Error troubleshooting feature in Copilot provides explanations of pipeline errors and guidance to resolve issues. For instance, if a pipeline fails due to a missing dataset, Copilot will identify the problem and suggest steps to fix it.

Understanding the structure and relationships within a complex pipeline can also be challenging, especially for those new to Data Factory. With Pipeline summarization, Copilot can generate an overview of your pipeline, describing the activities involved and how they interact. This can make it easier to review, share, and optimize your pipelines.

Here's an example of how you can interact with Copilot in the Data Factory pipeline editor:

Benefits

By using Copilot for Data Factory, you can perform data integration tasks and focus on deriving value from your data. Here are some areas where it can help:

• Efficiency: Automates parts of creating and managing dataflows and pipelines.
• Accessibility: Supports both citizen and professional data engineers via natural language inputs.
• Error assistance: Provides suggestions for pipeline errors.
• Collaboration: Provides pipeline summaries and code explanations for sharing.