Azure Data Factory Dataflow error handling for Dynamics 365 CRM application as sink
In Azure Data Factory Dataflow with sink type as Inline and Inline dataset type as Dynamics 365, there is no option for Continue on Error. The Dataflow and Pipelines fails on the first error from Dynamics 365 CRM application. The Error Tab in sink only…
Azure Data Factory
Is this a documentation error?
On the webpage https://learn.microsoft.com/en-us/azure/data-factory/connector-oracle?tabs=data-factory, I think the bold text here "The number of bytes the connector can fetch in a single network round trip. E.g., ArraySize=10485760. Larger values…
Azure Data Factory
Urgent Help Needed – Error After Upgrading Oracle Linked Service to 2.0
I’m running into a critical issue after switching from the Oracle Linked Service 1.0 to 2.0. Everything worked fine before the upgrade. Now my pipeline fails on the Sink side with the following error: Failure happened on 'Sink' side.…
Azure Data Factory
Dataflow works fine in DEV Azure Data Factory , but fails with errors in PROD
I am working in an environment of two ADFs (PROD and DEV), which are supposed to be identical copies. I have a bunch of copy activities running fine on both, but three (a,b,c) of them fail in PROD while running fine in DEV. a.-c. - copy jobs from (SAP)…
Azure Data Factory
ADF Related login issue
having a ADF pipeline which is trying to connect with oracle data base while clicking on test connection connectivity us getting successful but while debugging the pipeline its getting failed with below error details: Operation on target Copy MOC PTW…
Azure Data Factory
Oracle DB to ADLS using ADF
Hello - I have a requirement to create a data archival for on-prem oracle db. The DB contains attachments like excel,email etc. stored in DB storage as oracle Blobs for pega application. The data will be accessed by business users and we need an…
Azure Data Factory
Need help installation of OLE DB drivers to SSIS IR for DB2 connect
Hi Team, I am trying to run an SSIS package in Azure Data Factory using an SSIS Integration Runtime. The SSIS package uses IBM DB2 as the source and SQL MI as the destination. However, the package execution is failing due to a DB2 connection…
Azure Data Factory
Precision loss when migrating data from SQL Server to Snowflake using ADF via CSV staging (but not with Direct Copy)
I’m migrating SQL Server tables to Snowflake using Azure Data Factory. Direct Copy from SQL Server to Snowflake works, but I can’t use it because of the 100 MB row size limit. That forces me to take an intermediate staging approach. Approach 1: Direct…
Azure Data Factory
ADF Amazon Redshift linked service produces SSL error
I have a Amazon Redshift v1.0 linked service in Azure Data Factory that connects successfully, when I test the connection. When the service is used in a dataset, it fails to fetch data with 27809 error code: Failed while reading data from source during…
Azure Data Factory
Datafactory Self hosted integration runtime auto updates
Hi, We have enabled auto-update for our self-hosted integration runtime. Initially, we deployed it with version '5.48.9076.1'. Over time, it was updated to '5.55.9315.1' via auto-update with our interventions. However, we've noticed that the latest…
Azure Data Factory
third-party SFTP with ADF
We are trying to connect Azure Data Factory to a third-party SFTP endpoint provided by Bloomberg. The connection fails when using the Azure Integration Runtime (AutoResolve IR). Is it possible to connect to Bloomberg SFTP using Azure IR + IP…
Azure Data Factory
In Azure Data Factory, what version of JDBC driver is used to connect to Snowflake?
In Data Factory, what version of JDBC driver is used to connect to Snowflake via linked service using Integration Runtime in a Managed VNet (not self-hosted)? What is the schedule to update the JDBC driver for Snowflake connection when a new version is…
Azure Data Factory
OAuth 2.o grant-type: Authorization Code
Hi, I am trying to create a pipeline that interacts with an endpoint that uses OAuth 2.0; however, it requires a grant_type=authorization_code, which I can only assume is granted once the admin user logs in and retrieves it. My question is how have…
Azure Data Factory
Feature Request: Add Gate Condition for ADF scheduled Trigger
Hi Team, I would like to propose an enhancement for Azure Data Factory (ADF) triggers: the ability to add a Gate Condition before a trigger executes. Currently, ADF triggers (such as schedule triggers) do not allow conditional logic beyond basic…
Azure Data Factory
Files Stored in respective folder
Hello Team, There are multiple files stored in 3 folder in SFTP Location.The pipeline is schduled for every 2 hours. My queries is if any news files is not present on the schdule time then the pipeline must not run. If any new files are present in the…
Azure Data Factory
We are facing an issue while copying data from Oracle to Azure SQL using ADF Copy Activity.
We are facing an issue while copying data from Oracle to Azure SQL using ADF Copy Activity. The pipeline fails or produces incorrect numeric values due to high-precision decimals in Oracle. Details: Source (Oracle): TABLE_FIN Target (Azure SQL):…
Azure Data Factory
Why do I have to have GIT
Why do I have to have GIT to use Data Factory? I currently use TFS. Why can't that be used, or my Azure Storage container or.... I hate GIT
Azure Data Factory
Slow ADF Copy Performance When Transferring Oracle Tables Containing CLOB Columns
I am facing a performance issue in ADF when copying data from an Oracle source table that contains a CLOB column. Although the table size is only around 1 GB, the copy operation takes almost 2 hours to complete. However, if I convert the CLOB column to a…
Azure Data Factory
Email Notifcation in ADF Pipline
Hello Team, I am using the Web activity to received the email notification in my outlook.So i have mentioned this formula @json(concat( '{', '"message": "ERP-DW-UAT TRANSACTION_PARTITION INPROGRESS",', …
Azure Data Factory
ADF Amazon S3 Linked Service - SSL failure
I am trying to create a Linked Service to an Amazon S3 bucket in Azure Data Factory but getting a Server Certificate error. How can I disable Server Certificate Validation when using an Amazon S3 Linked Service? Error: The file operation is failed. A…